Information processing system, information processing method, and program

Abstract

Provided are an information processing system, an information processing method, and a program that use a machine learning model with a more improved technique.SOLUTION: The information-processing method includes a S15 of inputting a prompt to a machine-learning generative model. When an instruction for causing the generation model to generate data is input, the generation model 14 outputs the data generated in response to the instruction. The prompt includes one draft of a notification text to be transmitted to the one or more user terminals and a modification instruction to modify the one draft. The information-processing method also includes a S16 of acquiring, from the generative model, a draft obtained by the generative model modifying one original draft in accordance with the modification instruction, and a S17 of inputting the draft to the machine-learning prediction model 26. When a draft is input, the prediction model outputs a prediction result of an index indicating a reaction of the user when the draft is received.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] In recent years, it has been proposed to use machine learning models to create copy for use in advertisements. For example, Patent Document 1 discloses a method for generating a machine learning model for generating copy by fine-tuning an open-source pre-learning model. This machine learning model is trained to output second copy that is expected to have a high click-through rate (CTR) by using first copy that has a click-through rate (CTR) equal to or higher than a predetermined value as training data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-182309 Summary of the Invention [Problem to be solved by the invention]

[0004] While machine learning models are used in a variety of fields, not just in creating advertising copy, there is still much room for improvement in how they are used. The present disclosure aims to provide an information processing system, an information processing method, and a program that can utilize machine learning models in an improved manner. [Means for solving the problem]

[0005] An information processing system according to one embodiment of the present disclosure includes a memory that stores a plurality of instructions and at least one processor, wherein the at least one processor executes the plurality of instructions to input a prompt to a machine learning generative model, wherein the generative model is configured to, upon input of an instruction to generate data to the generative model, output data generated in accordance with the instruction, the prompt including a draft of a notification message to be sent to one or more user terminals and a modification instruction to modify the draft; obtain from the generative model a draft obtained by modifying the draft in accordance with the modification instruction; and input the draft to a machine learning predictive model, wherein, upon input of the draft, the predictive model is configured to output a predicted result of an indicator indicating a user's reaction when the draft is received; and obtain the predicted result for the draft output by the predictive model.

[0006] An information processing method according to one embodiment of the present disclosure is an information processing method executed by at least one processor, comprising: inputting a prompt to a machine learning generative model, wherein the generative model is configured to output data generated in response to input of an instruction to cause the generative model to generate data, the prompt including a draft of a notification message to be sent to one or more user terminals and a correction instruction to correct the draft; obtaining from the generative model a draft obtained by correcting the draft in accordance with the correction instruction; and inputting the draft to a machine learning predictive model, wherein the predictive model is configured to output a predicted result of an indicator indicating a user's reaction when the draft is received, when the draft is input.

[0007] A program according to one embodiment of the present disclosure causes at least one processor to: input a prompt to a machine learning generative model, wherein the generative model is configured to output data generated in response to input of an instruction to cause the generative model to generate data, the prompt including a draft of a notification message to be sent to one or more user terminals and a correction instruction to correct the draft; obtain from the generative model a draft obtained by correcting the draft in accordance with the correction instruction; and input the draft to a machine learning predictive model, wherein the predictive model is configured to output a predicted result of an indicator indicating a user's reaction when the draft is received, when the draft is input. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram of an information processing system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a notification screen displayed on a user terminal. [Figure 3] FIG. 3 is a diagram illustrating an example of a notification display screen displayed on a user terminal. [Figure 4] FIG. 4 is a diagram illustrating an example of an operation screen of the notification generating application when generating a notification. [Figure 5] FIG. 5 is a diagram illustrating an example of an operation screen for managing notifications of the notification generating application of FIG. [Figure 6] FIG. 6 is a diagram illustrating a detailed screen transitioned from the operation screen of FIG. [Figure 7] FIG. 7 is a diagram illustrating a method for generating a draft of a notice. DETAILED DESCRIPTION OF THE INVENTION

[0009] Examples of an information processing system 11, an information processing method, and a program will be described with reference to Figures 1 to 7. The present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[0010] [Information Processing Systems] The information processing system 11 shown in FIG. 1 includes an information processing device 20 for transmitting various notifications to a user terminal 60 used by a user. The information processing device 20 is operated by a notification distributor. Notifications transmitted from the information processing device 20 are received by one or more (usually multiple) user terminals 60. Each user views the notifications via an application (hereinafter sometimes simply referred to as an "app") installed on the user terminal 60 that the user uses.

[0011] The notification is expressed to the user as a "notice." The notification includes a notification text, which includes information about special offers, information about new features and updates of the app, or "notices from the store." The special offers include campaign information from the administrator of the information processing device 20, and the "notices from the store" include advertisements requested by any advertiser. The administrator of the information processing device 20 may be the provider of the app for receiving the notifications.

[0012] The advertising client (advertiser) is, for example, a seller who sells products or services in a store. The store may be a brick-and-mortar store that sells products or services face-to-face, or an online shop that operates an EC (Electronic Commerce) site. The client operates the client terminal 50 to send a draft of the advertisement to the information processing device 20.

[0013] The information processing device 20 performs information processing for generating a notification based on the draft of the advertisement received from the requester terminal 50. More specifically, the information processing device 20 generates a draft of the notification using the generative model 14 based on the draft of the notification (e.g., advertisement) received from the requester terminal 50. Thereafter, the information processing device 20 generates a notification to be sent to the user terminal 60 based on the draft of the notification.

[0014] The generative model 14 is configured to output the data generated in response to an instruction to generate some data into the generative model 14. The draft of the notification is not limited to the draft of the advertisement received from the client terminal 50, but may be, for example, a draft of the advertisement created by a contractor who has been contracted to create the advertisement, or a draft of the notification created by an administrator.

[0015] The generative model 14 may be a language model capable of editing sentences or creating new ones. The language model may be a natural language processing model trained using a large amount of text data. The generative model 14 may be a general-purpose large-scale language model applicable to any natural language processing task, such as information extraction, text summarization, text generation, or question and answering. The language model is configured to, when input with a prompt including an instruction, create a sentence according to the instruction and output it as a completion.

[0016] In the present disclosure, an example will be described in which the information processing device 20 uses the generative model 14, which is a general-purpose language model that is not included in the information processing system 11. Alternatively, the information processing device 20 may be provided with the generative model 14, or the information processing system 11 may be provided with a computer that stores the generative model 14.

[0017] A general-purpose language model can generate sentences according to instructions, but it is not clear whether the sentences are appropriate. For example, in the case of an advertisement copy, if the user who reads the copy responds well, the advertisement is deemed to be highly effective and appropriate. An indicator of user response or advertising effectiveness (also simply referred to as an "advertising indicator") is, for example, the click-through rate (CTR) of the link included in the advertisement copy. The indicator may be, but is not limited to, user engagement, which indicates the degree to which the user participated or engaged with the advertisement (notification).

[0018] The information processing system 11 includes a prediction model 26 for predicting an index value related to a notification message. For example, the prediction model 26 is configured to, when one example notification message is input, output a prediction result of an index indicating a user's reaction when the one example notification message is received.

[0019] When the information processing device 20 acquires the original draft of the notification text, it causes the generative model 14 to create a revised draft of the original draft. Furthermore, the information processing device 20 inputs the draft into the prediction model 26 and acquires the predicted value of the index as the prediction result from the prediction model 26. The index is not limited to the click rate, but may also be, for example, the display rate (view rate) of the delivered notification, GMS (Gross Merchandise Sales), the number of installations or launches of the advertised app, or the number of views of the linked video.

[0020] The information processing system 11 may include at least one server 30. The at least one server 30 may include one or more of a web server that provides an e-commerce site, a processing server that provides electronic payment services, and a management server that provides a point program. In this example, an example will be described in which the information processing system 11 includes one web server that provides an e-commerce site.

[0021] The information processing system 11 may include a learning model generation device 40. The learning model generation device 40 is configured to generate a prediction model 26. When the information processing system 11 (the information processing device 20 in this example) includes a plurality of prediction models 26, the information processing system 11 may include a plurality of learning model generation devices 40 for generating the plurality of prediction models 26. In this example, an example will be described in which the information processing system 11 includes one learning model generation device 40 for generating a prediction model 26 that predicts a click-through rate.

[0022] [Requester's device] The requester terminal 50 used by the requester may be realized as a computer including at least one processor 51, at least one memory 52, and a communication interface (IF) 53. For convenience, the following describes a case where the requester terminal 50 includes one processor 51 and one memory 52. ​​The requester terminal 50 may be a mobile terminal such as a smartphone or a tablet.

[0023] The communication IF 53 enables communication with other devices via a network. The requester terminal 50 may include an input device 54 and an output device 55, or may be externally connected. The input device 54 may include, for example, a keyboard and a mouse, and the output device 55 may be, for example, a display. The requester terminal 50 may also include a touch panel, which is an input / output device.

[0024] A client creates a draft of an advertisement for, for example, a brick-and-mortar store or an online shop. The content and format of the advertisement may be specified by the notification distributor. For example, the format of the advertisement delivered to the user by the notification may include a headline title, a body text, an image, and a link. The image in the present disclosure is a headline image displayed together with the title, but is not limited to this, and may also be an image included in the body text or an advertising image replacing the body text. The link is included in at least one of the title, the body text, or the image.

[0025] [User device] The user terminal 60 may be realized as a computer including at least one processor 61, at least one memory 62, and a communication IF 63. For convenience, the following describes a case where the user terminal 60 includes one processor 61 and one memory 62. The communication IF 63 enables communication with other devices via a network.

[0026] The user terminal 60 may be a mobile terminal such as a smartphone or a tablet. The user terminal 60 may include a display 64, which is an output device. The display 64 may include a touch panel, which is an input device.

[0027] The memory 62 stores programs and various data executed by the processor 61. The programs include applications for receiving notifications transmitted from the information processing device 20. The processor 61 executes processes based on the programs to realize various functions.

[0028] The application for receiving the notification may be, for example, an electronic payment application (hereinafter referred to as a "Pay App") for making electronic payments such as code payments, a points management application for managing points, or a shopping application for using an EC site. In any case, the notification is created in a format for display on any application installed on the user terminal 60. Below, an example of receiving a notification via a Pay App installed on the user terminal 60 will be described.

[0029] The pay app may display a notification link button 70 in the header of the payment screen 67 that displays the payment code 66, for example, to notify the user that a notification has been received. The notification link button 70 may be an icon or text. The notification link button 70 allows the user to know that a new notification has been received or that there are unread notifications. For example, in FIG. 1, the number in the badge displayed over the notification link button 70 (bell mark) indicates the number of unread notifications. The user can view the contents of the received notification by operating the notification link button 70.

[0030] [Web Server] The server 30 that provides the EC site may be realized as a computer including at least one processor 31, at least one memory 32, and a communication IF 33. For convenience, the following describes a case where the server 30 includes one processor 31 and one memory 32. The communication IF 33 enables communication with other devices via a network.

[0031] The memory 32 stores a program 34 and a database 35 executed by the processor 31. The program 34 includes an application and an operating system. The processor 31 executes processing based on the program 34 to realize various functions.

[0032] In this example, the server 30 provides a marketplace-type e-commerce site where multiple businesses or stores are located, but the e-commerce site may also be an online shop operated for a single business or store. The e-commerce site may allow electronic payments through processing by the server 30 or another server, and points may be awarded in a point program according to the payment amount.

[0033] The database 35 may include seller data. The seller data may include multiple seller records for each of multiple sellers. Each seller record may include, for example, a seller ID as an identifier, a seller name, a store name, a store ID, a store address, an account on the e-commerce site, an email address, authentication information, a payment receiving account, a store terminal ID, an electronic payment history, a sales history on the e-commerce site, and a point award history, but is not limited to these. The store terminal may be a cashless payment terminal or a point-of-sale (POS) register integrated with a payment terminal.

[0034] The seller data may further include the product (or service) offered, the category of the product (or service), the brand, and the item. Examples of categories include, but are not limited to, women's fashion, men's fashion, kids' and baby products, daily necessities, cosmetics, diet, health, home appliances, sports, outdoors, housing, pets, and hobbies. The categories may match the categories offered by the e-commerce site.

[0035] The database 35 may include customer data related to customers who use the e-commerce site. The customer data may include, for example, multiple customer records for each of multiple customers, with a customer ID as an identifier. Each customer record may include, for example, but is not limited to, name, address, account, email address, electronic payment history, purchase history on the e-commerce site, point acquisition history, and point usage history. The payment history and purchase history may include, for example, but is not limited to, the name of the store where the payment was made, date and time, purchased items, the category of purchased items, and purchase amount.

[0036] [Information processing device] The information processing device 20 may be realized as a computer including, for example, at least one processor 21, at least one memory 22, and a communication IF 23. For convenience, the following describes a case where the information processing device 20 includes one processor 21 and one memory 22. The communication IF 23 enables communication with other devices via a network.

[0037] The memory 22 stores a program 24 executed by the processor 21 and a database 25. The program 24 includes an application and an operating system. The processor 21 performs various functions by executing processes based on the program 24. The memory 22 may also store a prediction model 26 generated by a learning model generation device 40.

[0038] The database 25 may include destination data relating to multiple users who are candidate recipients of the notification, client data relating to the client of the advertisement, and notification data relating to the notification to be delivered. The information processing device 20 may acquire data held by the server 30 via a network. Data items of the destination data may match some or all of the customer data included in the database 35. The destination data may include some or all of the customer data. Data items of the client data may match some or all of the seller data included in the database 35. The client data may include some or all of the seller data.

[0039] The notification data may include, for example, multiple notification records for each of multiple notifications, with a notification ID as the record identifier. Each notification record may include, for example, but is not limited to, the requester's name, category, title, distribution start date and time, distribution frequency, and status. Categories may include, for example, "Special Offers," "Information about New App Features and Updates," and "Store Announcements." Hereinafter, notifications of "Special Offers" and "Store Announcements" that include advertisements are also referred to as "Advertisement Notifications." The status may include, for example, the status of the distribution, such as "Distributing," "Draft," "Pending," or "Completed."

[0040] The notification data may include a delivery log of the notification, including the delivery destination, delivery start date and time, and the number of deliveries, and may also include performance data on user engagement with delivered notifications. The performance data may include at least one indicator of advertising effectiveness, such as the number of impressions (number of times the notification is displayed), the number of clicks, and the click-through rate, as user engagement with the advertising notification. The performance data may also include the content of each notification, such as advertising content. The advertising content may include, for example, a title, a body text, a website destination URL, and an image.

[0041] [Learning model generation device] The information processing system 11 may include a learning model generation device 40 for generating the prediction model 26. The learning model generation device 40 may be realized as a computer including at least one processor 41, at least one memory 42, and a communication IF 43. For convenience, the following describes a case where the learning model generation device 40 includes one processor 41 and one memory 42. The communication IF 43 enables communication with other devices via a network.

[0042] The memory 42 stores a program 44 executed by the processor 41. The program 44 includes an application and an operating system. The processor 41 performs processing based on the program 44 to realize various functions. The memory 42 may store a learning model 45 and a learning dataset 46.

[0043] The data set 46 may include notification data (particularly, delivery logs and performance data) acquired from the information processing device 20. The data set 46 is not limited to notifications transmitted via the pay app, but may also include delivery logs and performance data of notifications transmitted via other apps (for example, a points management app or a shopping app) or notifications transmitted by email.

[0044] The dataset 46 may include training data, validation data, and test data. The dataset 46 may include seller data and customer data acquired from the server 30. The learning model generation device 40 may edit the customer data to include user data for multiple user groups having multiple attributes different from each other. The dataset 46 may include the user data edited in this way.

[0045] For example, when a user to whom the data is to be delivered matches a customer included in the customer data, the learning model generation device 40 may cluster the multiple customers included in the customer data into multiple user groups based on one or more common attributes that each of the customers has. User attributes include, but are not limited to, residential area, age group, gender, or annual income.

[0046] User groups may be clustered in advance so that they can be the target of advertisement notifications. For this clustering, one or more personas, which are fictitious user images, may be set. The persona profile can be set by combining multiple attributes, such as a "female" in her 20s or 30s.

[0047] The training model 45 may utilize an algorithm such as logistic regression or gradient boosting. The training model 45 may also be a pre-trained model. In this case, the pre-trained model may be generated by transfer learning or fine-tuning the pre-trained model using the dataset 46.

[0048] The learning model generation device 40 may use performance data of advertisements as teacher data (training data) and train the learning model 45 so that when an advertisement copy (at least one of the title and the body copy) is input, the learning model 45 outputs a click-through rate as a predicted value. The prediction model 26 trained in this manner is configured to output a click-through rate predicted by an original advertisement copy when the original advertisement copy is input, for example. The learning model 45 may be retrained periodically or irregularly based on additional performance data.

[0049] The learning model generation device 40 trains the learning model 45 using the dataset 46, thereby generating a learned model. The learned model generated in this manner is called a prediction model 26. The prediction model 26 generated by the learning model generation device 40 is provided to the information processing device 20 and stored in the memory 22.

[0050] The prediction model 26 may be configured to be trained using user data for a plurality of users (e.g., customers of an e-commerce site), including a plurality of attribute values ​​for each of the plurality of users, and to output a prediction result according to the user's attributes. When the attributes of the user to be delivered are input to such a prediction model 26 along with one example notification (draft notification), the prediction model 26 outputs a prediction result according to the user's attributes.

[0051] The learning model generation device 40 may train the learning model 45 using user data of a user group having one or more common attributes, thereby generating a prediction model 26 that outputs a prediction result specialized for that user group. That is, the learning model generation device 40 may generate multiple prediction models 26 corresponding to each of multiple user groups. In this case, by changing the prediction model 26 to be used depending on the attributes of the users to whom the data is to be distributed, it is possible to obtain prediction results according to the user attributes.

[0052] [Terminal Device] The person in charge of creating the notification can create a draft of the notification using the terminal device 80. The person in charge of using the terminal device 80 may be a client who creates a draft of the advertisement, a contractor who undertakes to create the advertisement, a distributor who distributes the notification, or an operator of the information processing device 20, but is not limited to these.

[0053] The terminal device 80 may be realized as a computer including, for example, at least one processor 81, at least one memory 82, and a communication IF 83. For convenience, the following describes a case in which the terminal device 80 includes one processor 81 and one memory 82. The communication IF 83 enables communication with other devices via a network.

[0054] The memory 82 stores a program 84 that is executed by at least one processor 81. The program 84 includes one or more applications and an operating system. The processor 81 executes processing based on the program 84 to realize various functions.

[0055] The terminal device 80 may include an input device 85 and a display 86, or may be externally connected to them. The input device 85 may include, for example, a keyboard and a mouse. The terminal device 80 may also be equipped with a touch panel, which is an input / output device.

[0056] The one or more applications stored in memory 22 include a notification generation application. Hereinafter, the notification generation application will be referred to as a "drafting tool." A person in charge can use the drafting tool to generate a draft of a notification. In addition to a notification generation function for generating a draft of a notification, the drafting tool may also have a notification management function for managing the sending of notifications.

[0057] [Displaying notifications on user devices] When the notification link button 70 displayed on the user terminal 60 shown in Fig. 1 is operated, a notification screen 71 shown in Fig. 2 is displayed. The notification screen 71 may display a list of notifications, or may display a category list 72 as shown in Fig. 2. The category list 72 includes a plurality of list elements 73 indicating notification categories (in Fig. 2, "Special Offers," "New Features / Updates," and "Notices from Stores").

[0058] The category list 72 may include an indication of whether there are new or unread notifications. For example, in Figure 2, the number of unread notifications is displayed at the right end of each list element 73. When one of the list elements 73 is selected, the announcements (notifications) for the selected category are displayed.

[0059] 3 illustrates a notification display screen 75 that displays a notification. The notification display screen 75 includes, for example, a title box 76 that displays a title, an image area 77 for displaying an image, and a body box 78 that displays the body text. In the present disclosure, the image area 77 is displayed between the title and the body text.

[0060] [Draft Tool] 4 shows an example of an operation screen 100 that is displayed on the display 86 when the draft tool is launched on the terminal device 80. The operation screen 100 is merely an example, and the design can be changed as desired. The operation screen 100 may include a navigation bar 101, a sidebar 102, and a main column 103. The navigation bar 101 includes a tab 104 for displaying switching between a notification generation function and a notification management function.

[0061] The sidebar 102 may include, for example, a notification management button 105 that is operated when managing the delivery of notifications, and a notification generation button 106 that is operated when generating notifications. Fig. 4 shows the operation screen 100 when the notification generation button 106 is operated.

[0062] When the notification generation button 106 is operated, a notification information area 110 showing notification information and a notification generation area 120 for generating a notification may be displayed in the main column 103. The notification information area 110 may include one or more category buttons 111 (three radio buttons in this example) for selecting a category of the notification.

[0063] When any of the category buttons 111 is selected, a selection field 112 may be displayed in the notification information area 110. For example, when the "Notices from Stores" category button 111 is selected and the selection field 112 is operated, a drop-down list of multiple stores (stores or sellers that have requested advertising) may be displayed. The notification information area 110 may include a date and time input field 113 for inputting the start date and time of notification distribution. When the date and time input field 113 is operated, a date picker for inputting a date from a calendar and a time picker for setting a time may be displayed.

[0064] The notification generation area 120 may include a title input field 121 for inputting a title, and an image specification field 122 for specifying an image (here, a header image). The notification generation area 120 may further include an image display field 123 for displaying the specified image. The notification generation area 120 may also include a URL input field 124 for inputting a link destination of an external browser to be included in the body of the notification.

[0065] The notification generation area 120 includes a text input field 125 for inputting text. When the person in charge inputs text into the text input field 125, a rephrasing button 126 may be displayed in the text input field 125 or the notification generation area 120. The rephrasing button 126 is operated when causing the generative model 14 to generate one or more drafts based on the text input into the text input field 125. The rephrasing button 126 may be displayed in advance in the text input field 125 or the notification generation area 120, regardless of whether text has been input.

[0066] When the paraphrase button 126 is operated, the terminal device 80 transmits the original draft input in the text input field 125 to the information processing device 20. Upon receiving the original draft, the information processing device 20 transmits a draft generated by the generative model 14 based on the original draft to the terminal device 80. The information processing device 20 may transmit the prediction result of the prediction model 26 for the draft (for example, a predicted value of the click rate) to the terminal device 80 together with the draft.

[0067] The information processing device 20 may transmit one draft for one original draft to the terminal device 80, or may transmit multiple drafts for one original draft to the terminal device 80. When transmitting multiple drafts to the terminal device 80, the information processing device 20 may transmit multiple prediction results for each of the multiple drafts to the terminal device 80.

[0068] The notification generation area 120 may include a button (not shown) for specifying the number of drafts to be generated or an input field (not shown) for inputting the number. In this case, the information processing device 20 transmits the specified number of drafts for one original draft to the terminal device 80 together with the corresponding prediction results.

[0069] In response to the operation of the paraphrase button 126, the terminal device 80 displays a draft display field 127 in the notification generation area 120 and displays one or more drafts received from the information processing device 20 in the draft display field 127. The terminal device 80 may also display, in the draft display field 127, the prediction results for each draft together with the one or more drafts.

[0070] The draft display field 127 may be displayed in advance in the notification generation area 120 regardless of whether the paraphrase button 126 is operated. When multiple drafts are displayed in the draft display field 127, selection buttons 128 corresponding to each of the multiple drafts may be displayed in the draft display field 127.

[0071] Even when the terminal device 80 receives a plurality of drafts and a plurality of prediction results corresponding thereto, the processor 81 may display, among the plurality of drafts, one with the best prediction result in the draft display field 127. Alternatively, the processor 81 may display some of the plurality of drafts in the draft display field 127. For example, the processor 81 may display, among the plurality of drafts, only drafts whose predicted advertising indicators are equal to or greater than a specified target value (for example, 60%, 70%, 80%, or 90%) in the draft display field 127.

[0072] The prediction result for the draft may be displayed in the draft display field 127 not as the predicted value of the advertising indicator itself, but as a category of a predetermined numerical range (e.g., less than 40%, 40 to 70%, 70 to 100%). The category of the numerical range may be set, for example, taking into consideration the prediction accuracy of the prediction model 26. Furthermore, instead of displaying the prediction result as the value of a specific indicator (e.g., click-through rate), the prediction result may be displayed as an indicator of a higher concept such as "user engagement."

[0073] When the person in charge operates the selection button 128 to select one from one or more drafts, the selected draft replaces the original draft that was previously input and is displayed in the main text input field 125. Therefore, the selection button 128 can also be said to be a replacement button. Even when only one draft is displayed in the draft display field 127, when the person in charge operates the selection button 128, the draft is replaced with the original draft and is displayed in the main text input field 125.

[0074] In this example, the original text of the main text is replaced with a draft and the predicted results of the draft are displayed, but in other examples, instead of or in addition to the main text, the original text of the advertisement title may be replaced with a draft and the predicted results of the draft may be displayed.

[0075] The person in charge can further edit the draft that has been replaced by the original draft. For example, if the main text contains information such as the start date of an advertising campaign or the conditions for obtaining a benefit, such information does not need to be revised. Therefore, the person in charge can first enter only the sentence that they want to rephrase in the main text input field 125, replace it with the draft as necessary, and then add the information to complete the main text. In this way, the notification text generated by the generative model 14 may be part of the main text (or title).

[0076] Although not shown, the operation screen 100 may include one or more operation buttons for selecting target users to whom a notification is to be sent. For example, the one or more selection buttons may include a button for selecting a predefined user group to send a notification to, or a button for specifying attributes of the user group. The user group may be clustered in advance so as to match the prediction target of the prediction model 26. The attributes of the user group include, but are not limited to, the user's age group (e.g., teens, twenties, thirties, etc.), gender, and region (e.g., prefecture).

[0077] Although not shown, the operation screen 100 may include a save button for saving the notification generated as described above, and a cancel button for terminating the process without saving the notification. When the person in charge operates the save button, the content input and operation performed by the person in charge via the operation screen 100 is transmitted as notification data from the terminal device 80 to the information processing device 20 via the network. Upon receiving the transmitted notification data, the information processing device 20 may assign a notification ID to the notification data and store it in the memory 22 as part of the database 25.

[0078] Although not shown, the operation screen 100 may include a review button for confirming the display of the notification generated as described above. When the person in charge operates the review button, the entire image of the notification, including the title, image, and text, may be displayed in the main column 103 or in a separate window, as shown in FIG.

[0079] Although not shown, the draft tool may have a function for sending a draft of the generated notification to an approver or requester of notification delivery in order to obtain approval for delivery of the notification generated as described above. The sending and approval of the draft may be realized by the function of the draft tool or by the function of a separate communication application such as email.

[0080] 5 illustrates an example of the operation screen 100 when the notification management button 105 is operated. When the notification management button 105 is operated, the terminal device 80 requests the information processing device 20 to send notification data to be displayed on the operation screen 100. In response to the request from the terminal device 80, the information processing device 20 transmits the notification data stored in the memory 22 to the terminal device 80. When the processor 81 of the terminal device 80 receives the notification data, it displays it in the main column 103 as a notification list 130.

[0081] A plurality of notification records for each of the generated notifications may be displayed in the main column 103, with the notification ID used as a record identifier. Each notification record may include, for example, but is not limited to, the source of distribution (e.g., the name of the store that requested the advertisement), category, title, distribution date and time (scheduled distribution date and time if not yet distributed, or distribution start date and time if already distributed), status, and last updated date and time.

[0082] While a notification is being created and awaiting approval, its status becomes "Draft." If the approver or requester requests a change to the notification content (if approval is denied), the status becomes "Pending." Once approval is granted, the "Draft" status changes to "Waiting for Delivery." By selecting a notification record displayed in the main column 103, its detailed information may be displayed in the main column 103 or in a separate window.

[0083] 6 shows an example of a details screen 131 that displays detailed information about a notification record. The details screen 131 may include a details field 132 that displays the notification title, text, link destination, and image, as well as the distribution status. The details field 132 may include, for example, the status, the date and time the notification was generated, the date and time the distribution started, the date and time the last update was made, and the distribution destination, but is not limited to these.

[0084] The details screen 131 for a notification record whose status is "distributing" or "completed" may include a performance column 133. The performance column 133 may include performance data regarding user engagement, as well as the number of deliveries to date. Performance data may include, but is not limited to, the number of impressions (number of times the notification has been displayed), the number of clicks, and the click rate. Regarding the number of impressions, for example, the notification may not be considered to have been displayed when the notification screen 71 in FIG. 2 is displayed, but may be considered to have been displayed when the notification display screen 75 in FIG. 3 is displayed.

[0085] The details screen 131 may include an edit button 134 and may also include a distribution button 135. If the status is distribution in progress, the distribution button 135 may be operated to stop distribution, accompanied by a display of "distribution stop." If the status is distribution waiting, the distribution button 135 may be operated to start distribution, accompanied by a display of "distribution start." When the distribution start operation is performed in this way, the information processing device 20 starts distribution of a notification to the user terminal 60, the set distribution destination, at the set distribution start date and time.

[0086] The edit button 134 is operated when editing a notification record. By operating the edit button 134, the person in charge can display the operation screen 100 shown in FIG. 4 and edit the notification record. For example, for a notification whose status is pending, the person in charge can change the draft to another draft as an editing task. Furthermore, even if a notification is currently being distributed, if the expected user engagement is not being achieved, the person in charge can revise the advertisement copy and then redistribute the notification.

[0087] [How notifications are generated and delivered] The information processing method of the present disclosure will be described with reference to Fig. 7. In particular, Fig. 7 shows a method for generating a draft of a notification message, which is one of the information processing methods of the present disclosure. The processor 21 of the information processing device 20 and the processor 81 of the terminal device 80 cooperate with each other to execute a plurality of instructions, thereby realizing the information processing method. The plurality of instructions for realizing the information processing method are contained in a program 24 stored in memory 22 and a program 84 stored in memory 82.

[0088] In step S11, processor 81 acquires a draft of the notification text (e.g., advertising text) that the person in charge has input into main text input field 125 of the draft tool. Then, in step S12, when the person in charge operates rephrasing button 126, processor 81 transmits a rephrasing request together with the draft input in step S11 to information processing device 20.

[0089] In step S13, the processor 21 acquires the original draft by receiving the original draft and the paraphrase request transmitted by the terminal device 80. In step S14, the processor 21 generates a prompt to be input to the generative model 14 based on the paraphrase request.

[0090] At least some of the information used to generate the prompt may be included in the paraphrase request. For example, the paraphrase request may include data entered in notification information area 110 (e.g., notification category, advertising requester, or distribution start date and time).

[0091] The prompt may include, for example, one draft of the notification (the body of the advertisement in this example) and a revision instruction for revising the one draft. Alternatively, the revision instruction may include an instruction for generating multiple drafts of the one draft. The prompt may be generated in advance before the process of FIG. 7 is started.

[0092] The modification instructions include instructions for modifying the first draft so that the value of the index (e.g., click rate) changes in a direction that improves user response, i.e., so that user engagement improves. For example, the modification instructions may include instructions for modifying the first draft so that the value of the index approaches or exceeds a specified target value (e.g., 60%, 70%, 80%, or 90%). The modification instructions may not include a target value, but may simply include instructions to make a modification to improve the index value.

[0093] The correction instruction may include multiple target indicators. At least one of the multiple indicators may be the same as the prediction indicator output by the prediction model 26. For example, if the advertising indicator and the prediction indicator are click rates, the correction instruction may include multiple targets such as "a first target is to increase the click rate to 70% or more, and a second target is to increase the GMS (Gross Merchandise Sales) of the target store."

[0094] If the notification text to be rephrased includes an advertisement copy about a store, the correction instructions may include supplemental information for generating the advertisement copy. The supplemental information may include store information about the advertiser or the store being advertised. The store may be a member store of an electronic payment service or a store that operates on an e-commerce site. In this case, the processor 21 may obtain store information about the store from the database 35 of the server 30. The store information may include, for example, but is not limited to, the products (or services) offered, the genres, brands, and items of the products (or services) offered.

[0095] The correction instruction may include, as supplemental information, at least some of the following: the content of the campaign related to the advertisement, the timing of the campaign, the app in which the notification will be displayed (e.g., a pay app), and the attributes of the user who will receive the notification (e.g., age, gender, and residential area). For example, the supplemental information may include that a store is a member of an electronic payment service and that the user who will receive the notification is a user of the electronic payment service. Processor 21 may acquire such supplemental information from the notification data stored in memory 22.

[0096] In step S15, processor 21 inputs the generated prompt to generative model 14. Then, in step S16, processor 21 acquires a completion output by generative model 14. The completion includes one or more drafts that generative model 14 has modified from one original draft in accordance with the modification instructions. Therefore, in step S16, processor 21 acquires one or more drafts.

[0097] In step S17, processor 21 inputs the one or more drafts acquired in step S16 into prediction model 26. In the following step S18, processor 21 acquires prediction results of indicators for each of the one or more drafts output by prediction model 26. If processor 21 acquires multiple drafts in step S16, processor 21 may input the drafts one by one into prediction model 26 and acquire corresponding prediction results one by one, thereby acquiring multiple prediction results corresponding to the multiple drafts.

[0098] If the prediction model 26 is configured to output a prediction result according to the attributes of the user, the processor 21 may input the attributes of the user who is to receive the notification message, along with the draft, to the prediction model 26. In this case, in step S18, the processor 21 obtains a prediction result according to the attributes of the user.

[0099] In step S19, the processor 21 transmits one or more drafts generated by the generative model 14 to the terminal device 80, together with one or more corresponding prediction results output by the prediction model 26. When the terminal device 80 receives the drafts and prediction results transmitted by the information processing device 20, the processor 81 displays the received drafts and prediction results in the draft display field 127, i.e., on the display 86 of the terminal device 80.

[0100] [Effects of the present disclosure] The draft display field 127 of the draft tool displays one or more drafts that the generative model 14 has paraphrased based on one original draft of the notification text. The draft display field 127 displays each draft along with a predicted result of an indicator that indicates the user's reaction when receiving a notification containing the text of that draft. Therefore, the person in charge can judge the quality of the paraphrased draft based on the predicted result. As a result, the person in charge can select a more appropriate one from the multiple drafts and adopt the selected draft as the notification text.

[0101] When the person in charge judges that the draft generated by the generative model 14 is appropriate based on the prediction results of the predictive model 26 and operates the selection button 128, the selected draft is adopted as the notification text. The person in charge can also modify the selected draft as necessary. Thereafter, when the person in charge operates the distribution button 135, the processor 21 sends a notification related to the corresponding notification record to one or more user terminals 60. As a result, a notification including the selected draft as the notification text is sent to the user terminal 60.

[0102] Since the generation model 14 may be a general-purpose language model, there is no need to generate a dedicated learning model for the purpose of generating notification sentences. Therefore, even if the notification category or advertising content changes, the desired notification can be generated simply by changing the prompt.

[0103] [Effects of this disclosure] According to the present disclosure, the following effects can be achieved. (1) A draft notification message can be generated by the generative model 14 modifying the original notification message. Furthermore, the predictive model 26 outputs a prediction result of an index showing the user's reaction to the draft, allowing the person in charge to determine whether the draft generated by the generative model 14 is appropriate. In this way, the information processing system 11, information processing method, and program disclosed herein can utilize machine learning models (particularly the generative model 14) in an improved manner.

[0104] (2) The generative model 14 generates multiple drafts for one original draft simply by changing the correction instructions included in the prompt. The predictive model 26 then outputs multiple prediction results for each of the multiple drafts using the same criteria. This allows the person in charge to easily compare the multiple drafts to determine which one is more appropriate.

[0105] (3) The processor 21 of the information processing device 20 transmits multiple drafts and multiple prediction results to the terminal device 80, so that the person in charge can generate the notification message using the terminal device 80 that he or she uses.

[0106] (4) After the person in charge selects one of the multiple drafts as the notification to be sent or modifies the selected draft, the information processing device 20 transmits a notification including the selected notification to one or more user terminals 60. By including the modified notification in the notification in this manner, improved user engagement with the notification can be expected. Furthermore, by retraining the prediction model 26 based on performance data for the notifications delivered in this manner and evaluating the drafts using the retrained prediction model 26, continuous improvement in user engagement can be expected.

[0107] (5) By displaying the one of the multiple drafts that has the best prediction result from the prediction model 26 in the draft display field 127 of the terminal device 80, the person in charge can be saved the trouble of selecting one from the multiple drafts.

[0108] (6) By including store information about the store that will be advertising in the revision instructions, the generative model 14 can generate a draft advertising copy that is suitable for advertising that store. (7) The correction instructions include supplemental information for generating advertising copy, which allows the generative model 14 to generate a more appropriate draft based on the supplemental information. For example, if the supplemental information includes information that the advertised store is a member of an electronic payment service and that the user receiving the notification is a user of the electronic payment service, the generative model 14 can generate a draft that is appropriate for users of the electronic payment service.

[0109] (8) By displaying the advertising text included in the notification of the electronic payment application (Pay App) on the notification display screen 75, it is possible to effectively show advertisements for affiliated stores of the electronic payment service to users of the electronic payment service.

[0110] (9) By using the prediction model 26 that outputs prediction results according to the attributes of the user, the information processing device 20 can present the appropriate prediction results for each user to the person in charge. As a result, the person in charge can select a draft that is suitable for the attributes of the user to whom the draft is to be distributed.

[0111] (10) The prediction model 26 can output a predicted value for an advertising indicator indicating user engagement for the input draft, allowing the person in charge to select a draft notification that will improve user engagement.

[0112] (11) The prediction model 26 can output a predicted value of the click rate for the input draft, which allows the person in charge to select a draft for the notification that will increase the click rate.

[0113] (12) If the correction instructions include instructions to correct the original proposal 1 so that the value of the index is improved, the generative model 14 can be made to generate a draft that will further improve the prediction results of the predictive model 26.

[0114] (13) If the correction instructions include instructions to correct the draft so that the value of the indicator approaches or exceeds a specified target value, the generative model 14 can generate an appropriate draft.

[0115] This embodiment can be modified as follows: This embodiment and the following modifications can be combined and implemented within the scope of technical compatibility. [Example of modifying the generative model] The generative model 14 may be configured to output a draft image obtained by correcting the original image in accordance with the correction instructions when a prompt including an original image (not limited to text) and correction instructions for correcting the original image is input. This generative model 14 makes it possible to obtain the corrected advertisement image in addition to the advertisement title and body text. Alternatively, the information processing system 11 may include the generative model 14, which is a language model for generating draft text, and another generative model suitable for generating images.

[0116] In this case, the information processing device 20 may include a prediction model configured to, when an example image is input, output a prediction result of an index indicating a user's reaction when the example image is received. The information processing device 20 can obtain the prediction result for the draft image output by the prediction model.

[0117] [Example 1 of changing the prediction model] The information processing system 11 may include a plurality of prediction models 26 configured to output prediction results for a plurality of different indices. For example, the information processing device 20 may include a first prediction model that outputs a predicted value of a click-through rate (first prediction result) and a second prediction model that outputs a predicted value of a GMS (second prediction result). In this case, at least one of the first prediction result and the second prediction result may be displayed in the draft display field 127.

[0118] [Example 2 of changing the prediction model] The prediction model 26 is not limited to predicting advertising metrics, but is preferably a metric that indicates user engagement with notifications. For example, if the notification is about a new feature of an app, the metric may be the number or rate of use of the new feature, or if the notification is a request for an update, the metric may be the rate at which the update is performed.

[0119] [Example of change instructions] The correction instructions included in the prompt may include a goal different from the predictive indicator of the predictive model 26. For example, if the predictive indicator is a click-through rate, the goal included in the correction instructions may be "increase the number of visitors to the target store," "increase the number of installs of the Pay App," or "increase traffic to the linked destination."

[0120] [Example of changing notification generation method] The modification instruction may not include a draft that is the basis for the modification, but may include a generation instruction for causing the generative model 14 to generate a new draft. That is, the generative model 14 may newly generate a draft notification instead of modifying the draft from the requester. In this case, step S11 may be omitted, and step S12 may be changed to sending a request to generate a draft. When causing the generative model 14 to create a new draft, the prompt may include one or more example sentences of the notification text to be generated, or it may not include any example sentences.

[0121] [Application change example 1] The draft tool may be installed in the information processing device 20. In this case, the person in charge can operate the information processing device 20 to use the draft tool.

[0122] [Application change example 2] The draft tool may be a native application that is installed and used on the terminal device 80, or may be a web application that runs on a web browser.

[0123] [Application change example 3] The draft tool may also have a function of displaying the prediction results of the prediction model 26 for the original draft entered by the person in charge. In this case, when the information processing device 20 receives the original draft, it transmits the prediction results for the original draft as well as the prediction results for the draft to the terminal device 80. The terminal device 80 may display the prediction results received from the information processing device 20, for example, in the text input field 125. According to this modified example, the person in charge can determine whether to replace the original draft with the draft by comparing the prediction results for the original draft and the prediction results for the draft.

[0124] [Example of notification changes] The notification message is not limited to being displayed via an app on the user terminal 60, but may be included in any electronic medium, such as an email, a chat, or a web advertisement. Furthermore, the notification message may be printed on paper or the like.

[0125] [Examples of changes to flow charts and configuration diagrams] The flow diagrams and block diagrams of the present disclosure illustrate the architecture, functionality, and operation of devices, systems, methods, and programs according to embodiments of the present disclosure. Each step included in these flow diagrams and each component included in the block diagrams may correspond to a portion of a program including one or more instructions for implementing a logical function. In other embodiments, some of the illustrated steps may be omitted, other steps may be included, the order of steps may be different, or some steps may be executed simultaneously. Furthermore, a flow diagram described as a series of actions may be executed in several separate steps, or multiple flow diagrams may be executed consecutively or in conjunction with each other. Furthermore, in other embodiments, some of the illustrated components may be omitted, other components may be included, or the arrangement of components may be changed. Furthermore, the functions implemented by these steps and components may be implemented in hardware, software, or a combination of hardware and software.

[0126] [Example of changes to information processing systems] The information processing system 11 may be realized as a single information processing device 20 (for example, a computer), or may be distributed across multiple devices (for example, computers) or subsystems that cooperate to execute programs.

[0127] [Server change example] When the information processing system 11 includes multiple servers 30, each server 30 may have a database related to the services provided by that server 30. These databases may be linked to each other based on one or more common data items or common identifiers (e.g., a user ID in the case of a user database). For example, multiple user databases may be linked to each other based on one or more common data items (user name and date of birth) or a common user ID. The multiple servers 30 may include, for example, but are not limited to, a web server that provides an e-commerce site, a processing server that provides electronic payment services, or a management server that provides a point program.

[0128] [Memory change example] The memories 22, 32, 42, 52, 62, and 82 included in the devices of the present disclosure are computer-readable storage media, including non-transitory computer-readable media, such as, but not limited to, ROM, hard disk, storage, removable media, flash memory, memory stick, optical media, magneto-optical media, and CD-ROM.

[0129] [Example of changing the processor] The one or more processors 21, 31, 41, 51, 61, 81 included in the apparatus of the present disclosure may include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a neural network processing unit (NPU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), an application-specific integrated circuit (ASIC), other processors including a general-purpose processor, or any combination thereof designed to perform the functions described herein.

[0130] [Network change example] Communication between the devices or systems may be performed according to well-known communication protocols over one or more communication networks, which may be, for example, but not limited to, an intranet, the Internet, a local area network, a wide area network, a wireless network, a wired network, a virtual network, a software-defined network, or any other type of network or combination thereof.

[0131] [Example of changing communication interface] The communication IFs 23, 33, 43, 53, 63, and 83 realize a function that allows one device to communicate with other devices via a communication network. The communication IF 23 may be, for example, but is not limited to, a local area network (LAN), Wi-Fi (registered trademark), Bluetooth (registered trademark), near field communication (NFC), or other wireless communication IFs.

[0132] The following are some aspects that can be understood based on the above-described embodiment and modifications. [1] a memory for storing a plurality of instructions; at least one processor; wherein the at least one processor executes the plurality of instructions to inputting a prompt into a machine learning generative model, the generative model being configured to output data generated in response to input of an instruction to cause the generative model to generate data, the prompt including a draft of a notification message to be sent to one or more user terminals and a correction instruction to correct the draft; Acquiring a draft from the generative model in which the generative model has modified the first original draft in accordance with the modification instruction; inputting the draft into a machine learning predictive model, the predictive model being configured to, upon input of the draft, output a predicted result of an indicator showing a user's reaction when receiving the draft; obtaining the prediction result for the draft output by the prediction model; An information processing system configured to perform the above.

[0133] [2] the modification instructions include instructions for generating a plurality of drafts from the one original draft; the at least one processor: Obtaining from the generative model a plurality of drafts generated by the generative model in accordance with the modification instructions; inputting the plurality of drafts into the forecasting model; obtaining a plurality of prediction results for each of the plurality of drafts output by the predictive model; configured to: The information processing system according to [1] above.

[0134] [3] the at least one processor: The method is configured to transmit the plurality of drafts together with the plurality of prediction results for each of the plurality of drafts to a terminal device used by a person in charge of creating the notification text. The information processing system according to [2] above.

[0135] [4] the at least one processor is configured to send a notification to the one or more user terminals; The notification includes, as the notification text, one draft selected by the person in charge from among the plurality of drafts, or includes, as the notification text, a text obtained by amending the one draft by the person in charge; The information processing system according to [2] or [3] above.

[0136] [5] the at least one processor: and displaying one of the plurality of drafts with the best prediction result on a terminal device used by a person in charge of creating the notification. The information processing system according to any one of [2] to [4] above.

[0137] [6] The notice includes an advertisement for one store, the at least one processor is configured to obtain store information for the one store; The correction instruction includes store information about the one store. The information processing system according to any one of [1] to [5] above.

[0138] [7] The notice includes an advertisement for one store, The correction instruction includes supplemental information for generating the advertisement copy, The supplemental information includes information that the first store is a member store of an electronic payment service and that the user is a user of the electronic payment service. The information processing system according to any one of [1] to [6] above.

[0139] [8] the at least one processor: configured to send a notification including the notification text to the one or more user terminals, the notification being formatted for display in an application installed on each of the one or more user terminals; The format includes a title and a body of the notification; The draft is at least a part of the title or the body of the document; The information processing system according to any one of [1] to [7] above.

[0140] [9] the prediction model is configured to be trained using user data for a plurality of users, the user data including a plurality of attribute values ​​for each of the plurality of users, and thereby to output a prediction result according to the attributes of the users; the at least one processor: inputting attributes of users who are expected to receive the notification message together with the draft into the prediction model; acquiring the prediction result output by the prediction model according to the attributes of the user; configured to: The information processing system according to any one of [1] to [8] above.

[0141]

[10] The indicator is an indicator of user engagement. The information processing system according to any one of [1] to [9] above.

[0142]

[11] the at least one processor: configured to send a notification including the notification text to the one or more user terminals; The notice includes a website link; The metric is a click-through rate. The information processing system according to any one of [1] to

[10] above.

[0143]

[12] The correction instructions include instructions for correcting the first draft so that the value of the index is improved. The information processing system according to any one of [1] to

[11] above.

[0144]

[13] The correction instructions include instructions for correcting the first draft so that the value of the index approaches a specified target value or exceeds the target value. The information processing system according to any one of [1] to

[12] above.

[0145]

[14] 1. An information processing method executed by at least one processor, comprising: inputting a prompt into a machine learning generative model, the generative model being configured to output data generated in response to input of an instruction to cause the generative model to generate data, the prompt including a draft of a notification message to be sent to one or more user terminals and a correction instruction to correct the draft; Acquiring a draft from the generative model in which the generative model has modified the first original draft in accordance with the modification instruction; inputting the draft into a machine learning predictive model, the predictive model being configured to, upon input of the draft, output a predicted result of an indicator showing a user's reaction when receiving the draft; An information processing method, including:

[0146]

[15] At least one processor has inputting a prompt into a machine learning generative model, the generative model being configured to output data generated in response to input of an instruction to cause the generative model to generate data, the prompt including a draft of a notification message to be sent to one or more user terminals and a correction instruction to correct the draft; Acquiring a draft from the generative model in which the generative model has modified the first original draft in accordance with the modification instruction; inputting the draft into a machine learning predictive model, the predictive model being configured to, upon input of the draft, output a predicted result of an indicator showing a user's reaction when receiving the draft; A program to perform the following. [Explanation of symbols]

[0147] 11...information processing system, 14...generative model, 20...information processing device, 21,31,41,51,61,81...processor, 22,32,42,52,62,82...memory, 23,33,43,53,63,83...communication IF, 24,34,44,84...program, 25,35...database, 26...prediction model, 30...server, 40...learning model generation device, 45...learning model, 46...dataset, 50...requester terminal, 54...input device, 55...output device, 60...user terminal, 64...display, 67...payment screen, 70...notification link button, 71...notification screen, 72...category list, 73...list element, 75...notification display screen, 76...title box, 77...image area, 78 ...Main text box, 80...Terminal device, 81...Processor, 82...Memory, 84...Program, 85...Input device, 86...Display, 100...Operation screen, 101...Navigation bar, 102...Sidebar, 103...Main column, 104...Tab, 105...Notification management button, 106...Notification generation button, 110...Notification information area, 111...Category button, 112...Selection field, 113...Date and time input field, 120...Notification generation area, 121...Title input field, 122...Image specification field, 123...Image display field, 124...URL input field, 125...Main text input field, 126...Rephrasing button, 127...Draft display field, 128...Selection button, 130...Notification list, 131...Details screen, 132...Details field, 133...Achievements field.

Claims

1. a memory for storing a plurality of instructions; at least one processor; wherein the at least one processor executes the plurality of instructions to inputting a prompt into a machine learning generative model, the generative model being configured to output data generated in response to input of an instruction to generate data into the generative model, the prompt including a draft of a notification message to be sent to one or more user terminals and a correction instruction to correct the draft; Acquiring a draft from the generative model in which the generative model has modified the first original draft in accordance with the modification instruction; inputting the draft into a machine learning predictive model, the predictive model being configured to, upon input of the draft, output a predicted result of an indicator showing a user's reaction when receiving the draft; obtaining the prediction result for the draft output by the prediction model; An information processing system configured to perform the above.

2. the modification instructions include instructions for generating a plurality of drafts from the one original draft; the at least one processor: Obtaining from the generative model a plurality of drafts generated by the generative model in accordance with the modification instructions; inputting the plurality of drafts into the forecasting model; obtaining a plurality of prediction results for each of the plurality of drafts output by the predictive model; configured to: The information processing system according to claim 1 .

3. the at least one processor: The method is configured to transmit the plurality of drafts together with the plurality of prediction results for each of the plurality of drafts to a terminal device used by a person in charge of creating the notification text. The information processing system according to claim 2 .

4. the at least one processor is configured to send a notification to the one or more user terminals; The notification includes, as the notification text, one draft selected by the person in charge from among the plurality of drafts, or includes, as the notification text, a text obtained by amending the one draft by the person in charge. The information processing system according to claim 3 .

5. the at least one processor: and displaying one of the plurality of drafts with the best prediction result on a terminal device used by a person in charge of creating the notification. The information processing system according to claim 2 .

6. The notice includes an advertisement for one store, the at least one processor is configured to obtain store information for the one store; The correction instruction includes store information about the one store.

6. The information processing system according to claim 1.

7. The notice includes an advertisement for one store, The correction instruction includes supplemental information for generating the advertisement copy, The supplemental information includes information that the first store is a member store of an electronic payment service and that the user is a user of the electronic payment service.

6. The information processing system according to claim 1.

8. the at least one processor: configured to send a notification including the notification text to the one or more user terminals, the notification being formatted for display in an application installed on each of the one or more user terminals; The format includes a title and a body of the notification; The draft is at least a part of the title or the body of the document; 6. The information processing system according to claim 1.

9. the prediction model is configured to be trained using user data for a plurality of users, the user data including a plurality of attribute values ​​for each of the plurality of users, and thereby to output a prediction result according to the attributes of the users; the at least one processor: inputting attributes of users who are expected to receive the notification message together with the draft into the prediction model; acquiring the prediction result output by the prediction model according to the attributes of the user; configured to:

6. The information processing system according to claim 1.

10. The indicator is an indicator of user engagement.

6. The information processing system according to claim 1.

11. the at least one processor: configured to send a notification including the notification text to the one or more user terminals; The notice includes a website link; The metric is a click-through rate.

6. The information processing system according to claim 1.

12. The modification instructions include instructions for modifying the first draft so as to improve the value of the index.

6. The information processing system according to claim 1.

13. The correction instructions include instructions for correcting the first draft so that the value of the index approaches a specified target value or exceeds the target value.

6. The information processing system according to claim 1.

14. 1. An information processing method executed by at least one processor, comprising: inputting a prompt into a machine learning generative model, the generative model being configured to output data generated in response to input of an instruction to generate data into the generative model, the prompt including a draft of a notification message to be sent to one or more user terminals and a correction instruction to correct the draft; Acquiring a draft from the generative model in which the generative model has modified the first original draft in accordance with the modification instruction; inputting the draft into a machine learning predictive model, the predictive model being configured to, upon input of the draft, output a predicted result of an indicator showing a user's reaction when receiving the draft; An information processing method, including:

15. At least one processor inputting a prompt into a machine learning generative model, the generative model being configured to output data generated in response to input of an instruction to generate data into the generative model, the prompt including a draft of a notification message to be sent to one or more user terminals and a correction instruction to correct the draft; Acquiring a draft from the generative model in which the generative model has modified the first original draft in accordance with the modification instruction; inputting the draft into a machine learning predictive model, the predictive model being configured to, upon input of the draft, output a predicted result of an indicator showing a user's reaction when receiving the draft; A program to perform the following.

Images