Information processing system, information processing method, and computer program products

Abstract

[Problem] To provide an information processing system, an information processing method, and a computer program product. [Solution] An information processing method executed by at least one processor includes: inputting a prompt to a machine learning generative model, wherein the generative model is configured to output data generated in accordance with an instruction once an instruction is input to cause the generative model to generate data; the prompt includes: an original of a notification message to be sent to one or more user terminals, and a correction instruction to cause the original to be corrected; and obtaining a draft of the original modified by the generative model according to the correction instruction from the generative model; and inputting the draft to a machine learning predictive model, wherein the predictive model is configured to output a prediction result of an indicator representing the user's reaction when the draft is received once the draft is input; and obtaining the prediction result for the draft output by the predictive model.

Description

[Technical Field]

[0001] This disclosure relates to information processing systems, information processing methods, and computer program products. [Previous Technology]

[0002] In recent years, the use of machine learning models in the creation of advertising copy has been proposed. For example, Japanese Patent Application Publication No. 2023-182309 discloses a method for generating a machine learning model needed to generate advertising copy by fine-tuning an open-source pre-learning model. This machine learning model is trained to output a second advertising copy with a higher expected click-through rate by using a first advertising copy with a click-through rate (CTR) of 1 or higher as training data. [Summary of the Invention]

[0003] [Problem to be Solved by the Invention] Machine learning models are not limited to the creation of advertising copy; they are used in a variety of fields, but there is still much room for improvement in their application methods. The purpose of this disclosure is to provide an information processing system, information processing method, and computer program product that can utilize machine learning models in a more improved manner. [Means for Solving the Problem]

[0004] The information processing system described in one aspect of this disclosure includes: a memory for storing a plurality of commands; and at least one processor; wherein the at least one processor is configured to execute the plurality of commands to: input a prompt to a machine learning generative model, wherein the generative model is configured to output data generated in accordance with the input instructions to generate data; the prompt contains: communication required to send to one or more user terminals. The document contains one original draft of the document and the correction instructions required to correct the original draft; and a draft generated by the document generation model after correcting the original draft according to the correction instructions is obtained from the document generation model; the document draft is input into a machine learning prediction model, wherein the document prediction model is configured to output a prediction result of an indicator representing the user's reaction when the document draft is received, once the document draft is input; and the document prediction result of the document draft output by the document prediction model is obtained.

[0005] The information processing method described in one aspect of this disclosure is an information processing method executed by at least one processor, comprising: inputting a prompt to a machine learning generative model, wherein the prior generative model is configured to output data generated in accordance with the prior instruction once an instruction is input to cause the generative model to generate data; the prior prompt contains: an original of a notification message to be sent to one or more user terminals, and a correction instruction to cause it to correct the prior original; and obtaining from the prior generative model a draft obtained by correcting the prior original according to the prior correction instruction; inputting the prior draft to a machine learning predictive model, wherein the prior predictive model is configured to output a prediction result of an indicator representing the user's reaction when receiving the prior draft once the prior draft is input; and obtaining the prior prediction result for the prior draft output by the prior predictive model.

[0006] The computer program product described in one aspect of this disclosure causes at least one processor to: input a prompt to a machine learning generative model, wherein the prior generative model is configured to output data generated in accordance with the prior instructions once an instruction is input to cause the generative model to generate data; the prior prompt contains: an original of a notification message to be sent to one or more user terminals, and a correction instruction to cause it to correct the prior original; and obtains from the prior generative model a draft, which is a result of the prior original corrected according to the prior correction instruction; input the prior draft to a machine learning predictive model, wherein the prior predictive model is configured to output a prediction result of an indicator representing the user's reaction when the prior draft is received once the prior draft is input; and obtain the prior prediction result for the prior draft output by the prior predictive model. [Technical Effect]

[0007] An information processing system, information processing method, and computer program product that can utilize machine learning models in a more improved manner may be provided.

Implementation Method

[0009] Referring to Figures 1 to 7, examples of the information processing system 11, the information processing method, and the computer program product are described. This invention is not limited to these examples, but is intended to include all modifications within the scope and meaning of the claims disclosed herein.

[0010] [Information Processing System] The information processing system 11 shown in Figure 1 includes an information processing device 20 for sending various notifications to user terminals 60 used by users. The information processing device 20 is operated by the sender of the notification. Notifications sent from the information processing device 20 are received by one or more (usually multiple) user terminals 60. Each user views the notifications through an application (hereinafter sometimes simply referred to as "application") installed on their own user terminal 60.

[0011] A notification, as perceived by the user, is represented as a "message." A notification contains a notification message, which may include: promotional messages, information about new features and updates to the application, or "messages from merchants." Promotional messages contain information about promotional activities issued by the administrator of the information processing device 20, and "messages from merchants" contain advertisements commissioned by any advertiser. The administrator of the information processing device 20 may also be the provider of the application required to receive notifications.

[0012] The client (advertiser) of the advertisement is, for example, a seller who sells goods or services in a store. The store may be a physical store that conducts face-to-face sales, or an online store that operates on an EC (Electronic Commerce) website. The client operates the client terminal 50 to send the original advertisement to the information processing device 20.

[0013] The information processing device 20 performs information processing necessary for generating a notification based on the original advertisement received from the client terminal 50. More specifically, the information processing device 20 uses the generation model 14 to generate a draft notification text based on the original notification text (e.g., an advertisement) received from the client terminal 50. Subsequently, the information processing device 20 generates a notification to be sent to the user terminal 60 based on the draft notification text.

[0014] The generation model 14 is configured to output the data generated in accordance with the input instructions for generating certain data. The original of the notification message is not limited to the original of the advertisement received from the client terminal 50. For example, it may be the original of an advertisement made by a contractor who has contracted to make an advertisement, or the original of a notification made by a manager.

[0015] The generative model 14 can also be a language model that can edit or create new articles. The language model can also be a natural language processing model trained using a large amount of text data. The generative model 14 can also be a general-purpose, large-scale language model applicable to any natural language processing task, such as information extraction, article summarization, article generation, or question response. The language model is configured to generate an article corresponding to the prompt once input containing a prompt, and output it as the generated result.

[0016] This disclosure describes an example where the information processing device 20 uses a generative model 14 that is a general-purpose language model and is not included in the information processing system 11. Alternatively, the information processing device 20 may possess the generative model 14, or the information processing system 11 may possess a computer that holds the generative model 14.

[0017] A generalized language model can generate articles corresponding to instructions, but it does not know whether the article is appropriate. For example, if it is an advertisement, then if the user response from reading the advertisement is positive, it can be said that the advertisement is effective and appropriate. Metrics representing user response or advertisement effectiveness (also referred to as "advertising metrics") are, for example, the click-through rate (CTR) of links contained in the advertisement. Metrics can also be, but are not limited to, indicators representing the degree to which users participate in or engage with the advertisement (notification).

[0018] The information processing system 11 includes a prediction model 26 for predicting the value of an indicator involved in a notification message. For example, the prediction model 26 is configured to output the prediction result of an indicator representing the user's reaction when receiving a notification message once a notification message is input.

[0019] Once the information processing device 20 obtains the original notification text, it instructs the generation model 14 to create a draft version that modifies the original text. Furthermore, the information processing device 20 inputs the draft version into the prediction model 26 and obtains the predicted values ​​of the indicators from the prediction model 26 as the prediction result. The indicators are not limited to click-through rate; for example, they may also be the display rate (view rate) of the delivered notification, GMS (Gross Merchandise Sales), the number of installations or launches of the advertised application, or the number of views of the linked target video.

[0020] The information processing system 11 may also have at least one server 30. The at least one server 30 may also include one or more of the following: a web server providing an EC website, a processing server providing electronic checkout services, and a management server providing points programs. In this example, we are illustrating an example of the information processing system 11 having one web server providing an EC website.

[0021] The information processing system 11 may also include a learning model generation device 40. The learning model generation device 40 is configured to generate the prediction model 26. When the information processing system 11 (in this example, the information processing device 20) has a plurality of prediction models 26, the information processing system 11 may also include a plurality of learning model generation devices 40 required to generate the plurality of prediction models 26. This example illustrates an instance where the information processing system 11 has one learning model generation device 40 required to generate the prediction model 26 for predicting the prediction rate.

[0022] [Client Terminal] The client terminal 50 used by the client can also be implemented as a computer having at least one processor 51, at least one memory 52, and a communication interface (IF) 53. For convenience, the following example of a client terminal 50 having one processor 51 and one memory 52 will be described. The client terminal 50 can also be a portable terminal such as a smartphone or tablet.

[0023] Communication IF53 enables communication with other devices via a network. The client terminal 50 may also include or be externally connected to an input device 54 and an output device 55. The input device 54 may include, for example, a keyboard and a mouse, and the output device 55 may be, for example, a display. The client terminal 50 may also have a touch panel belonging to the input / output devices.

[0024] The client, for example, creates an original design for an advertisement required by a physical store or online store. The content and format of the advertisement can also be specified by the sender of the notification. For example, the format of an advertisement sent to a user via notification may include: a title as the main title, text, an image, and a link. The image disclosed herein is a main title image displayed together with the title, but is not limited thereto; it may also be an image contained in the text, or an advertisement image that replaces the text. The link is included in at least one of the title, text, or image.

[0025] [User Terminal] The user terminal 60 can also be implemented as a computer having at least one processor 61, at least one memory 62, and a communication IF 63. For convenience, the following example of a user terminal 60 having one processor 61 and one memory 62 will be described. The communication IF 63 enables communication with other devices via a network.

[0026] The user terminal 60 may also be a portable terminal such as a smartphone or tablet. The user terminal 60 may also have a display 64, which is an output device. The display 64 may also have a touch panel, which is an input device.

[0027] The memory 62 stores programs and various data that are executed by the processor 61. The programs include applications used to receive notifications sent from the information processing device 20. The processor 61 performs processing based on the programs to implement various functions.

[0028] The application used to receive the notification may be, for example, an electronic checkout application (hereinafter referred to as "payment application") required for electronic checkout such as code checkout, a points management application required for points management, or a shopping application required for using an EC website. In any case, the notification is made in a format required for display in any application installed on the user terminal 60. The following describes an example of receiving a notification through a payment application installed on the user terminal 60.

[0029] The payment application may also display a notification link button 70 in the header of the checkout screen 67, for example, displaying the checkout code 66, to inform the user of any pending notifications. The notification link button 70 may be a small icon or text. The user can learn from the display of the notification link button 70 that new notifications have been sent or that there are unread notifications. For example, in Figure 1, the value within the badge superimposed on the notification link button 70 (the bell icon) indicates the number of unread notifications. The user can view the content of the sent notifications by operating the notification link button 70.

[0030] [Web Server] The server 30 providing the EC website can also be implemented as a computer having at least one processor 31, at least one memory 32, and a communication IF 33. For convenience, the following example describes a server 30 having one processor 31 and one memory 32. The communication IF 33 enables communication with other devices via a network.

[0031] The memory 32 stores a program 34 and a database 35 that are executed by the processor 31. The program 34 includes an application program and an operating system. The processor 31 performs processing based on the program 34 to realize various functions.

[0032] In this example, server 30 is used to illustrate an example of an e-commerce website that allows multiple businesses or shops to operate. However, an e-commerce website can also be an online store operated for a single business or shop. In an e-commerce website, electronic checkout can also be enabled through the processing of server 30 or other servers, and points can be assigned to the points program according to the checkout amount.

[0033] Database 35 may also contain vendor information. Vendor information may also contain multiple vendor records for each of multiple vendors. Each vendor record may, for example, use vendor ID as an identifier and contain: vendor name, store name, store ID, store address, account on the EC website, email address, authentication information, payment receiving account, store terminal ID, electronic checkout history, sales history on the EC website, points assignment history, but is not limited to these. The store terminal may also be a cashless checkout terminal or a POS (Point of Sales) cash register integrated with the checkout terminal.

[0034] Seller information may also include: the goods (or services) sold, the type of goods (or services) sold, the brand, and the items. Types may include, for example: women's fashion, men's fashion, children's / baby products, daily necessities, cosmetics, weight loss, health, home appliances, sports, outdoor, home, pets, hobbies, but are not limited to these. The type may also be consistent with the sales type on the EC website.

[0035] Database 35 may also contain customer information about customers using the EC website. Customer information may, for example, use customer ID as an identifier and contain multiple customer records for each of the multiple customers. Each customer record may also contain, for example, name, address, account number, email address, and electronic checkout history, purchase history on the EC website, points acquisition history, and points usage history, but is not limited to these. Checkout history and purchase history may also contain, for example, the name of the store where the checkout was made, date and time, purchased items, type of purchased items, and purchase amount, but is not limited to these.

[0036] [Information Processing Device] The information processing device 20 can also be implemented as, for example, a computer having at least one processor 21, at least one memory 22, and a communication IF 23. Hereinafter, for convenience, an example of the information processing device 20 having one processor 21 and one memory 22 will be described. The communication IF 23 enables communication with other devices via a network.

[0037] The memory 22 stores a program 24 and a database 25 that are executed by the processor 21. The program 24 includes an application program and an operating system. The processor 21 performs processing based on the program 24 to implement various functions. The memory 22 may also store a predictive model 26 generated by the learning model generation device 40.

[0038] Database 25 may also contain: target information related to multiple users who are candidates for notification delivery, client information related to the client of the advertisement, and notification information related to the notification being delivered. Information processing device 20 may also obtain the data held by server 30 via the network. The data items of the target information may also be partially or entirely consistent with the customer information contained in database 35. The target information may also contain partially or entirely customer information. The data items of the client information may also be partially or entirely consistent with the vendor information contained in database 35. The client information may also contain partially or entirely vendor information.

[0039] The notification data may, for example, use the notification ID as the identifier of the record, and contain multiple notification records for each of the multiple notifications. Each notification record may also contain, for example: client name, category, title, launch start date and time, launch frequency, status, but is not limited to these. The category may also contain, for example: "Promotional message", "New features and update information of the application", and "Message from the store". Hereinafter, in "Promotional message" and "Message from the store", notifications containing advertisements will also be referred to as advertising notifications. The status may also contain, for example: launch in progress, draft, saved, closed, etc.

[0040] Notification data may also be a notification delivery log, containing: delivery target, delivery start date and time, and number of deliveries, and may also contain performance data regarding user engagement for delivered notifications. Performance data may also be user engagement data regarding advertising notifications, containing: at least one metric indicating advertising effectiveness, such as impressions (number of times the notification is displayed), clicks, and click-through rate. Performance data may also contain the content of each notification, such as the advertising content. Advertising content may also contain, for example: title, text, target URL linking to the EC website, and images.

[0041] [Learning Model Generation Device] The information processing system 11 may also include a learning model generation device 40 for generating the prediction model 26. The learning model generation device 40 may also be implemented as a computer having at least one processor 41, at least one memory 42, and a communication IF 43. For convenience, the following example of the learning model generation device 40 having one processor 41 and one memory 42 will be described. The communication IF 43 enables communication with other devices via a network.

[0042] The memory 42 stores a program 44 that is executed by the processor 41. The program 44 includes an application program and an operating system. The processor 41 performs processing based on the program 44 to realize various functions. The memory 42 may also store a learning model 45 and a learning dataset 46.

[0043] Data set 46 may also contain notification data (especially delivery logs and performance data) obtained from information processing device 20. Data set 46 is not limited to notifications sent through payment applications, but may also include delivery logs and performance data of notifications sent through other applications (such as points management applications or shopping applications) or notifications sent via email.

[0044] Data set 46 may also contain training data, validation data, and test data. Data set 46 may also contain vendor data and customer data obtained from server 30. The learning model generation device 40 may also edit the customer data to include user data for multiple user groups with multiple distinct attributes. Data set 46 may also contain such edited user data.

[0045] For example, when the user who is the target of the application is the same as the customer contained in the customer data, the learning model generation device 40 can also cluster multiple user groups based on one or more common attributes that each user possesses. The user attributes are, for example, residential area, age, gender, or annual income, but are not limited to these.

[0046] User groups can also be pre-clustered to make them the target of advertising notifications. For such clustering, one or more personas can be set as fictional user profiles. The personal profile of a persona can be, for example, "female in her 20s to 30s", and multiple attributes can be combined to set it.

[0047] The learning model 45 can also utilize algorithms such as logistic regression or gradient boosting. The learning model 45 can also be a pre-learned model. In this case, the pre-learned model can also be transferred or fine-tuned using the data set 46 to generate a learned model.

[0048] The learning model generation device 40 can also use advertising performance data as teacher data (training data) to train the learning model 45, so that when an advertising copy (at least one of the title or text) is input, it will output the click-through rate as a predicted value. The prediction model 26 trained in this way is configured to, for example, output the click-through rate predicted based on the original advertising copy once the original advertising copy is input. The learning model 45 can also be retrained periodically or irregularly based on additional performance data.

[0049] The learning model 45 is trained using the data set 46 by the learning model generation device 40, and the learned model is generated. The learned model thus generated is called the prediction model 26. The prediction model 26 generated by the learning model generation device 40 is provided to the information processing device 20 and stored in the memory 22.

[0050] The prediction model 26 is trained using user data containing multiple attribute values ​​for multiple users (e.g., customers of an EC website) to output the prediction results corresponding to the user's attributes. Once the attributes of the user who is the target of the campaign are input along with a notification example (a draft of the notification), the prediction model 26 will output the prediction results corresponding to the user's attributes.

[0051] The learning model generation device 40 can also train the learning model 45 using user data from user groups that share one or more common attributes, and generate a prediction model 26 that outputs prediction results that are specialized for that user group. That is, the learning model generation device 40 can also generate multiple prediction models 26 corresponding to multiple user groups. In this case, by changing the prediction model 26 used according to the attributes of the users who are to be targeted, prediction results corresponding to user attributes can be obtained.

[0052] [Terminal Device] The person in charge of creating the notification message may use the terminal device 80 to create the draft notification message. The person in charge of using the terminal device 80 may be the client who commissioned the creation of the original advertisement, the contractor who contracted the creation of the advertisement, the distributor who distributed the notification, or the operator of the information processing device 20, but is not limited to these.

[0053] The terminal device 80 can also be implemented as, for example, a computer having at least one processor 81, at least one memory 82, and a communication IF 83. Hereinafter, for convenience, an example of the terminal device 80 having one processor 81 and one memory 82 will be described. The communication IF 83 enables communication with other devices via a network.

[0054] The memory 82 contains a program 84 that is executed by at least one processor 81. The program 84 includes one or more applications and operating systems. The processor 81 performs processing based on the program 84 to realize various functions.

[0055] The terminal device 80 may also include or be externally connected to an input device 85 and a display 86. The input device 85 may also include, for example, a keyboard and a mouse. The terminal device 80 may also have a touch panel belonging to the input / output device.

[0056] One or more applications stored in memory 82 include a notification generation application. Hereinafter, the notification generation application will be referred to as the "Drafting Tool". The person in charge can use the Drafting Tool to generate drafts of notifications. In addition to the notification generation functions required to generate drafts of notifications, the Drafting Tool may also have notification management functions required to manage the delivery of notifications.

[0057] [Display of Notifications on the User Terminal] Once the notification link button 70 displayed on the user terminal 60 shown in FIG1 is operated, the message screen 71 shown in FIG2 is displayed. The message screen 71 may also display a list of notifications, and may also display a category list 72 as illustrated in FIG2. The category list 72 contains a plurality of list elements 73 indicating the category of notification (in FIG2, these are "Promotional Messages", "New Features / Updates", and "Messages from Stores").

[0058] Category list 72 may also contain a display indicating whether new or unread notifications have been received. For example, in Figure 2, the number of unread notifications is displayed at the right end of each list element 73. Once any of the list elements 73 is selected, the messages (notifications) of the selected category are displayed.

[0059] Figure 3 illustrates a notification display screen 75 that displays a notification. The notification display screen 75 includes, for example: a title box 76 for displaying a title, an image area 77 for displaying an image, and a text box 78 for displaying the text. The image area 77 disclosed herein is displayed between the title and the text.

[0060] [Drafting Tool] Figure 4 illustrates the operation screen 100 displayed on the display 86 when the drafting tool is activated on the terminal device 80. The operation screen 100 is merely an example and its design can be modified arbitrarily. The operation screen 100 may include: a navigation bar 101, a sidebar 102, and a main bar 103. The navigation bar 101 includes tabs 104 used to switch between the notification generation function and the notification management function.

[0061] The sidebar 102 may also include, for example, a notification management button 105 that is operated when a notification is delivered, and a notification generation button 106 that is operated when a notification is generated. Figure 4 illustrates the operation screen 100 when the notification generation button 106 is operated.

[0062] Once the notification generation button 106 is activated, the main bar 103 will also display a notification information field 110 representing notification information and a notification generation field 120 required to generate a notification. The notification information field 110 may also contain one or more category buttons 111 (in this example, three option buttons) required to select the type of notification.

[0063] Once any category button 111 is selected, a selection bar 112 can also be displayed in the notification information area 110. For example, when the category button 111 for "Messages from Stores" is selected, operating the selection bar 112 will display a drop-down menu of multiple stores (shops or vendors that have commissioned the advertisement). The notification information area 110 may also include a date and time input field 113 for entering the start date and time of the notification. Once the date and time input field 113 is operated, a date picker for entering a date from a calendar and a time picker for setting the time can also be displayed.

[0064] The notification generation field 120 may include: a title input field 121 for entering a title, and an image specification field 122 for specifying an image (here, a header image). The notification generation field 120 may also include: an image display field 123 for displaying the specified image. The notification generation field 120 may also include: a URL input field 124 for entering the target URL of an external browser to be included in this document.

[0065] The notification generation area 120 includes a text input field 125 for inputting the text required for the text. Once the person in charge inputs text in the text input field 125, an overwrite button 126 may also be displayed in the text input field 125 or the notification generation area 120. The overwrite button 126 is activated when the text input in the text input field 125 is used as the original to generate one or more drafts by the generation model 14. The overwrite button 126 may also be displayed in advance in the text input field 125 or the notification generation area 120 regardless of whether text is input.

[0066] Once the rewrite button 126 is activated, the terminal device 80 sends the original text entered in the text input field 125 to the information processing device 20. Upon receiving the original text, the information processing device 20 sends the draft generated by the generation model 14 based on the original text to the terminal device 80. The information processing device 20 may also send the draft, along with the prediction results (e.g., the predicted click-through rate) of the prediction model 26 for that draft, to the terminal device 80.

[0067] The information processing device 20 can send one draft to the terminal device 80 for one original proposal, or it can send multiple drafts to the terminal device 80 for one original proposal. When the information processing device 20 sends multiple drafts to the terminal device 80, it can also send multiple prediction results for each of the multiple drafts to the terminal device 80.

[0068] The notification generation field 120 may also include a button (illustration omitted) for specifying the number of drafts generated or an input field (illustration omitted) for entering the number. In this case, the information processing device 20 sends the specified number of drafts, along with the corresponding prediction results, to the terminal device 80 for one original.

[0069] The terminal device 80, triggered by the operation of the rewrite button 126, causes the notification generation field 120 to display a draft display bar 127, and causes the draft display bar 127 to display one or more drafts received from the information processing device 20. The terminal device 80 can also display one or more drafts, along with the prediction results for each draft, in the draft display bar 127.

[0070] The draft display bar 127 can also be pre-displayed in the notification generation area 120 regardless of whether the rewrite button 126 is operated. When multiple drafts are displayed in the draft display bar 127, the selection button 128 corresponding to each of the multiple drafts can also be displayed in the draft display bar 127.

[0071] When the terminal device 80 receives a plurality of drafts and their corresponding plurality of prediction results, the processor 81 may also display the draft with the best prediction result in the draft display bar 127. Alternatively, the processor 81 may also display a portion of the plurality of drafts in the draft display bar 127. For example, the processor 81 may also display only the drafts whose predicted advertising metrics are above a specified target value (e.g., 60%, 70%, 80%, or 90%) in the draft display bar 127.

[0072] The prediction results for the draft may not be the predicted values ​​of advertising metrics themselves, but rather displayed in the draft display panel 127 as a distinction between defined numerical ranges (e.g., less than 40%, 40-70%, 70-100%). The distinction between numerical ranges may be set considering, for example, the prediction accuracy of prediction model 26. Furthermore, the prediction results may not be displayed as specific metrics (e.g., click-through rate), but rather as higher-level concepts such as "user engagement".

[0073] Once the person in charge selects one draft from more than one draft by operating the selection button 128, the selected draft will replace the previously entered original draft and be displayed in the text input field 125. Therefore, the selection button 128 can also be called a replacement button. The same applies when only one draft is displayed in the draft display field 127; once the person in charge operates the selection button 128, the draft will replace the original draft and be displayed in the text input field 125.

[0074] In this example, although the original text is replaced with a draft or the predicted result of the draft is displayed, etc., in other examples, it can also replace the original text or be added to the text to replace the original text of the advertisement title with a draft or display the predicted result of the draft, etc.

[0075] The person in charge may further edit the draft that has been replaced with the original document. For example, if the document contains contact information such as the start date of an advertising promotion or the conditions for obtaining a gift, there is no need to modify such contact information. Therefore, the person may simply enter the document to be rewritten into the document input field 125, and after replacing it with a draft as needed, complete the document by adding contact information. In this way, the notification text generated by the generation model 14 may also be part of the document (or title).

[0076] The operation screen 100, although omitted in the illustration, may include one or more operation buttons for selecting the target users to whom a notification is to be sent. For example, one or more selection buttons may be used to send a notification, and may also include buttons for selecting predefined user groups, or buttons for specifying the attributes of user groups. User groups may also be pre-clustered to ensure that the prediction objects of the prediction model 26 are consistent. The attributes of user groups may include, but are not limited to, the user's age group (e.g., teenagers, 20s, 30s, etc.), gender, and region (e.g., prefecture).

[0077] Although the illustration is omitted, the operation screen 100 may include: a save button for saving the notification generated as described above, and a cancel button for ending the process without saving the notification. Once the person in charge presses the save button, the content entered and operated by the person in charge through the operation screen 100 is sent from the terminal device 80 to the information processing device 20 via the network as notification data. The information processing device 20, upon receiving the sent notification data, may also assign a notification ID to the notification data and store it in the memory 22 as part of the database 25.

[0078] Operation screen 100, although the illustration is omitted, may also include: an approval button for confirming the display of the notification generated as described above. Once the person in charge operates the approval button, the overall appearance of the notification, including the title, image, and text, may be displayed in the main bar 103 or another window, as shown in Figure 3.

[0079] The draft tool, although not illustrated, may also include the function of sending a draft of the generated notice to the approver or agent of the notice delivery in order to obtain approval for delivery. The sending and approval of the draft can be achieved through the function of the draft tool, or through the function of other communication applications such as email.

[0080] Figure 5 illustrates the operation screen 100 when the notification management button 105 is operated. When the notification management button 105 is operated, the terminal device 80 requests the information processing device 20 for notification data to be displayed on the operation screen 100. In response to the request from the terminal device 80, the information processing device 20 sends the notification data stored in memory 22 to the terminal device 80. Once the processor 81 of the terminal device 80 receives the notification data, it causes it to be displayed as a notification list 130 in the main bar 103.

[0081] In the main column 103, the notification ID can also be used as the identifier of the record to display multiple notification records for each of the multiple generated notifications. Each notification record may also contain, for example: the source of the placement (e.g., the name of the store that commissioned the advertisement), category, title, placement date and time (the scheduled placement date and time if no placement has been made, and the placement start date and time if placement has been made), status, and last update date and time, but is not limited to these.

[0082] During the creation and approval process of the notification, the status is "Draft". If the approver or the principal requests changes to the content of the notification (in the case of rejection), the status changes to "Reserved". Once approved, the "Draft" status is changed to "Pending Release". The notification records displayed in the main column 103 can also be selected to display their detailed information in the main column 103 or another window.

[0083] Figure 6 illustrates a detailed screen 131 showing detailed information about a notification record. In addition to the notification title, text, linked target, and image, detailed screen 131 may also include a detailed column 132 displaying the delivery status. The detailed column 132 may also include, for example, status, notification generation date and time, delivery start date and time, last update date and time, and delivery target, but is not limited to these.

[0084] The detailed screen 131 for notification records with statuses of "in progress" and "completed" may also include a performance column 133. The performance column 133 may, for example, contain, in addition to the number of deliveries to date, user engagement performance data. Performance data may also include, for example, impressions (number of times the notification is displayed), clicks, and click-through rate, but is not limited to these. Regarding impressions, for example, during the stage where the message screen 71 in Figure 2 is displayed, the notification is not considered to have been displayed, but once the notification display screen 75 in Figure 3 is displayed, the notification is considered to have been displayed.

[0085] Detailed screen 131 may also include an editing button 134 and a delivery button 135. The delivery button 135 is activated when delivery is about to stop, accompanied by a "Delivery Stop" display. The delivery button 135 is activated when delivery is about to begin, accompanied by a "Delivery Start" display. Thus, once the delivery start operation is performed, delivery will begin at the set delivery start date and time for the user terminal 60 that has been set as the delivery target by the information processing device 20.

[0086] The edit button 134 is operated when the notification record needs to be edited. The person in charge can operate the edit button 134 to display the operation screen 100 shown in Figure 4 to edit the notification record. For example, for a notification with a status of "reserved", the person in charge can change the draft to another draft as an editing operation. Also, even for a notification that is being launched, if the expected user engagement is not obtained, the person in charge can revise the advertising text, etc., before relaunching the notification.

[0087] [Notification Generation and Delivery Method] Referring to FIG7, the information processing method disclosed herein will be described. In particular, FIG7 illustrates the method for generating a draft notification text in the information processing method disclosed herein. The processor 21 of the information processing device 20 and the processor 81 of the terminal device 80 cooperate to execute a plurality of commands, thereby implementing the information processing method. The plurality of commands required to implement the information processing method are contained in a program 24 stored in memory 22 and a program 84 stored in memory 82.

[0088] In step S11, the processor 81 obtains the original draft of the notification text (e.g., an advertisement text) entered by the person in charge in the text input field 125 of the draft tool. Then, in step S12, once the person in charge operates the rewrite button 126, the processor 81 sends the original draft and rewrite request entered in step S11 to the information processing device 20.

[0089] In step S13, the processor 21 obtains the original file by receiving the original file and rewriting request sent by the terminal device 80. In step S14, the processor 21 generates the prompts required to input into the generation model 14 based on the rewriting request.

[0090] At least a portion of the information used in generating the prompt may also be included in the rewriting request. For example, the rewriting request may also include: the data entered in the notification information field 110 (e.g., notification type, advertising commissioner, or launch start date and time).

[0091] A prompt contains, for example, a draft of a notification (in this example, the text of an advertisement) and correction instructions required to amend the draft. Alternatively, the correction instructions may contain instructions to generate multiple drafts for a single draft. The prompt may also be generated in advance before the processing in Figure 7 begins.

[0092] The correction instruction system includes instructions to modify an original design in order to change the value of an indicator (such as the click-through rate) in a way that improves user response; that is, to improve user engagement. For example, the correction instruction system may also include instructions to modify an original design in order to make the value of the indicator approach or exceed a specified target value (such as 60%, 70%, 80%, or 90%). The correction instruction system may also not contain a target value, but simply contain instructions to make corrections necessary to improve the indicator value.

[0093] The correction instruction may also contain a plurality of indicators as objectives. At least one of the plurality of indicators may also be the same as the prediction indicator output by the prediction model 26. For example, if the advertising indicator and the prediction indicator are click-through rate, the correction instruction may also contain a plurality of indicators such as: "The first objective is to achieve a click-through rate of 70% or higher, and the second objective is to increase the GMS (Gross Merchandise Sales) of the target store."

[0094] If the notification message being rewritten contains an advertisement for a store, the correction instruction may also include supplementary information generated at the time the advertisement was created. The supplementary information may also include store information about the advertiser or the store that is the target of the advertisement. The store may be a franchisee of the electronic checkout service or a store operating on an EC website. In this case, the processor 21 may also retrieve the store information about the store from the database 35 of the server 30. The store information may also include, but is not limited to, the type of goods (or services) sold, the brand, and the items.

[0095] The amendment indicates that, as supplementary information, it may also include at least one of the following: the content of the promotional activity related to the advertisement, the period of the promotional activity, the application on which the notification is displayed (e.g., a payment application), and the attributes of the users who are scheduled to receive the notification (e.g., age, gender, and place of residence). For example, the supplementary information may also include: a store is a franchisee of the electronic checkout service, and the user who will receive the notification is a user of the electronic checkout service. The processor 21 may also retrieve this type of supplementary information from the notification data stored in the memory 22.

[0096] In step S15, the processor 21 inputs the generated prompts into the generation model 14. Then, in step S16, the processor 21 obtains the generation result output by the generation model 14. The generation result includes one or more drafts obtained by the generation model 14 modifying an original draft according to the modification instructions. Therefore, in step S16, the processor 21 obtains one or more drafts.

[0097] In step S17, processor 21 inputs one or more drafts obtained in step S16 into prediction model 26. In subsequent step S18, processor 21 obtains the prediction results of the indicators for each of the one or more drafts output by prediction model 26. If processor 21 has obtained multiple drafts in step S16, processor 21 can also input the drafts one by one into prediction model 26 and obtain the corresponding prediction results one by one, thereby obtaining multiple prediction results for each of the multiple drafts.

[0098] When the prediction model 26 is configured to output the prediction results corresponding to the user's attributes, the processor 21 can also input the attributes of the user who is scheduled to receive the notification message, along with the draft, into the prediction model 26. In this case, in step S18, the processor 21 obtains the prediction results corresponding to the user's attributes.

[0099] In step S19, the processor 21 sends one or more drafts generated by the generation model 14, along with one or more corresponding prediction results output by the prediction model 26, to the terminal device 80. Once the drafts and prediction results sent by the information processing device 20 are received by the terminal device 80, the processor 81 displays the received drafts and prediction results in the draft display bar 127, that is, the display 86 of the terminal device 80.

[0100] [Function of this disclosure] The draft display panel 127 of the draft tool displays one or more drafts rewritten by the generation model 14 based on one original draft of the notification. The draft display panel 127 displays each draft, as well as the prediction results of indicators representing the user's reaction to receiving a notification containing that draft. Therefore, the person in charge can judge the quality of the rewritten draft based on the prediction results. As a result, the person in charge can select the more appropriate one from the multiple drafts, or can adopt the selected draft as the notification.

[0101] If the person in charge determines that the draft generated by the generation model 14 is appropriate based on the prediction results of the prediction model 26, and operates the selection button 128, the selected draft is adopted as a notification. The person in charge can also modify the selected draft as needed. Subsequently, once the person in charge operates the release button 135, the processor 21 sends the notification related to the corresponding notification record to one or more user terminals 60. In this way, the notification containing the selected draft as a notification is sent to the user terminal 60.

[0102] The generative model 14 can also be a general-purpose language model, so there is no need to generate a dedicated learning model for the purpose of generating notification text. Therefore, even if the category of the notification changes or the content of the advertisement changes, the notification for the purpose can still be generated simply by changing the prompt.

[0103] [Effects of this disclosure] According to this disclosure, the following effects can be achieved. (1) The original draft of the notification can be modified by the generation model 14, thereby generating a draft notification. Furthermore, the prediction model 26 outputs the prediction results of indicators representing the user's reaction to the draft, thereby allowing the person in charge to determine whether the draft generated by the generation model 14 is appropriate. Thus, according to the information processing system 11, information processing method and program disclosed herein, a more improved approach can be used to utilize machine learning models (especially the generation model 14).

[0104] (2) The generation model 14 can generate multiple drafts for one original proposal by simply changing the correction instructions contained in the prompt. Then, the prediction model 26 outputs multiple prediction results for each of the multiple drafts based on the same judgment criteria. Therefore, the person in charge can easily compare which of the multiple drafts is appropriate.

[0105] (3) The processor 21 of the information processing device 20 sends a plurality of drafts and a plurality of prediction results to the terminal device 80, thereby enabling the person in charge to use the terminal device 80 he / she uses to generate notification messages.

[0106] (4) The person in charge selects one notification message to be sent from a plurality of drafts, or modifies the selected draft, and the information processing device 20 sends a notification containing the notification message to one or more user terminals 60. By including the modified notification message in the notification, it is expected to improve user engagement with the notification. Furthermore, based on the performance data of the notifications that are sent in this way, the prediction model 26 is retrained, and the draft is evaluated by the retrained prediction model 26, thereby expecting continuous improvement in user engagement.

[0107] (5) Among the multiple drafts, the one with the best prediction result of the prediction model 26 is displayed in the draft display bar 127 of the terminal device 80, thereby saving the person in charge the trouble of selecting one from the multiple drafts.

[0108] (6) The correction instruction contains store information about the store that issued the advertisement, thereby enabling the generation model 14 to generate a draft of an advertisement suitable for that store. (7) The correction instruction contains supplementary information at the time of generating the advertisement, thereby enabling the generation model 14 to generate a more appropriate draft corresponding to the supplementary information. For example, if the supplementary information includes: the store that issued the advertisement is a franchisee of the electronic checkout service, and the user who will receive the notification is a user of the electronic checkout service, then the generation model 14 can generate a draft suitable for users of the electronic checkout service.

[0109] (8) By displaying the advertising text contained in the notification of the electronic checkout application (payment application) on the notification display screen 75, users of the electronic checkout service can effectively see the advertisements of the affiliated stores of the electronic checkout service.

[0110] (9) By using the prediction model 26, which outputs the prediction results corresponding to the user's attributes, the information processing device 20 can provide the person in charge with the appropriate prediction results for each user. As a result, the person in charge can select a draft of the user's attributes that is suitable for the target.

[0111] (10) Prediction model 26 can output predicted values ​​of advertising metrics representing user engagement for the input draft. In this way, the person in charge can select a draft that will increase user engagement as a notification.

[0112] (11) Prediction model 26 can output the predicted value of the click rate for the input draft. In this way, the person in charge can select the draft that improves the click rate as the notification.

[0113] (12) In the case where the correction instruction contains an instruction required to correct one original case in order to improve the value of the indicator, the generation model 14 can be made to generate a draft that will improve the prediction result of the prediction model 26.

[0114] (13) In cases where the modification instruction contains instructions required to modify the original draft so that the value of the indicator is close to or exceeds the specified target value, a more appropriate draft can be generated by generating model 14.

[0115] This embodiment can be modified as follows. This embodiment and the following modifications can be combined and implemented within the scope of not causing technical contradictions. [Example of Modification of Generation Model] Generation model 14 is not limited to articles, but can also be configured such that once a draft containing an image and a prompt containing correction instructions for modifying the original image are input, a draft image with the original image modified will be output according to the correction instructions. According to generation model 14, in addition to the title and text of the advertisement, the modified advertisement image can also be obtained. Alternatively, the information processing system 11 may also have: generation model 14 for the language model required to generate the draft article, and other generation models suitable for image generation.

[0116] In this case, the information processing device 20 may also have a prediction model configured to output a prediction result of an index representing the user's reaction upon receiving an image example, once an image example is input. The information processing device 20 can obtain the prediction result for the draft image output by the prediction model.

[0117] [Example 1 of Change in Prediction Model] The information processing system 11 may also have a plurality of prediction models 26, which are configured to output prediction results of a plurality of mutually exclusive indicators. For example, the information processing device 20 may also include: a first prediction model that outputs the predicted value of the click rate (first prediction result), and a second prediction model that outputs the predicted value of GMS (second prediction result). In this case, the draft display column 127 may also display at least one of the first prediction result and the second prediction result.

[0118] [Example 2 of the change in the prediction model] The prediction model 26 is used as the prediction object, which is not limited to advertising indicators, but rather indicators that represent the user's engagement with the notification. For example, if it is a message about a new feature of the application, it can also be the number of uses or utilization rate of the new feature; if it is a request for an update, it can also be the implementation rate of the update.

[0119] [Example of Modification Instruction] The modification instruction contained in the prompt may also contain a target that is different from the prediction metric of prediction model 26. For example, if the prediction metric is the click rate, the target contained in the modification instruction may also be "an increase in the number of visitors to the target store", "an increase in the number of installations of the payment application", or "an increase in traffic to the target".

[0120] [Example of Change in Notification Generation Method] The modification instruction may also not contain the original draft as the source of the modification, but may contain generation instructions required for generation model 14 to generate a new draft. That is, the draft notification may not be a modification of the original draft from the client, but may be newly generated by generation model 14. In this case, step S11 is omitted, and step S12 is changed to sending the draft generation request. When generating model 14 generates a new draft, the prompt may contain one or more examples of the notification text to be generated, or it may not contain any examples.

[0121] [Application Change Example 1] The draft tool can also be installed on the information processing device 20. In this case, the person in charge can operate the information processing device 20 to use the draft tool.

[0122] [Example 2 of application change] The draft tool can be a native application installed and used on the terminal device 80, or a web application that operates on a web browser.

[0123] [Application Modification Example 3] The draft tool may also have the function of displaying the prediction results of the prediction model 26 even for the original draft entered by the person in charge. In this case, once the information processing device 20 receives the original draft, it sends the prediction results for the draft, along with the prediction results for the original draft, to the terminal device 80. The terminal device 80 may also, for example, display the prediction results received from the information processing device 20 in the text input field 125. According to this modification example, the person in charge can compare the prediction results for the original draft and the prediction results for the draft to determine whether the original draft and the draft draft should be replaced.

[0124] [Example of Change to Notification] The notification message is not limited to those displayed through the application of the user terminal 60, but may also be included in any electronic media such as email, chat, or web advertisement. Furthermore, the notification message may also be printed on paper.

[0125] [Examples of Changes to Flowcharts and Structure Diagrams] The flowcharts and structure diagrams disclosed herein represent the architecture, functions, and actions of the apparatus, systems, methods, and programs involved in the embodiments of this disclosure. Each step in these flowcharts and each component in the structure diagrams can be equivalent to a part of a program containing one or more commands required to implement the logic function. In other embodiments, a portion of the illustrated steps may be omitted, other steps may be included, the order of the steps may be different, and several steps may be executed simultaneously. Furthermore, a flowchart describing a series of actions may be divided into several and executed, and multiple flowcharts may be executed consecutively or in a mutually related manner. Also, in other embodiments, a portion of the illustrated component elements may be omitted, other component elements may be included, and the configuration of the component elements may be changed. Furthermore, the functions implemented by these steps and component elements can also be implemented by hardware, software, or a combination of hardware and software.

[0126] [Example of Change to Information Processing System] The information processing system 11 can also be implemented as a single information processing device 20 (e.g., a computer), or it can be distributed among multiple devices (e.g., computers) or subsystems that work together to execute programs.

[0127] [Server Modification Example] In the case where the information processing system 11 contains a plurality of servers 30, each server 30 may also have a database related to the services provided by that server 30. These databases may also be interconnected based on one or more common data items or common identifiers (e.g., user ID if it is a user database). For example, if there are multiple user databases, they may also be interconnected based on one or more common data items (user's name and date of birth) or common user IDs. The plurality of servers 30 may also include, for example, a web server providing an EC website, a processing server providing electronic checkout services, or a management server providing points programs, but are not limited to these.

[0128] [Example of memory modification] The memory modules 22, 32, 42, 52, 62, and 82 contained in the device disclosed herein are computer-readable memory media, including non-transitory computer-readable media. The memory system may include, for example, ROM, hard disk, storage, removable media, flash memory, memory stick, optical media, optical magnetic media, CD-ROM, but is not limited to these.

[0129] [Example of Processor Modification] The one or more processors 21, 31, 41, 51, 61, 81 included in the device disclosed herein may also include CPU (central processing unit), GPU (graphics processing unit), APU (accelerated processing unit), NPU (Neural network Processing Unit), microprocessor, microcontroller, DSP (digital signal processor), FPGA (field programmable gate array), CPLD (Complex Programmable Logic Device), application-specific integrated circuit (ASIC), other processors containing general-purpose processors, or any combination of these designed to perform the functions described in this specification, but not limited to these.

[0130] [Example of Network Modification] Communication between multiple devices or systems may also be performed through one or more communication networks in accordance with well-known communication protocols. The communication network may also be, for example: an intranet, the Internet, a local area network, a wide area network, a wireless network, a wired network, a virtual network, a software-defined network, or any other type of network, or a combination of these networks, but is not limited to these.

[0131] [Example of Change to Communication IF (Interface)] Communication IFs 23, 33, 43, 53, 63, and 83 enable a device to communicate with other devices through a communication network. Communication IF 23 can also be, for example, LAN (Local Area Network), Wi-Fi (registered trademark), Bluetooth (registered trademark), NFC (Near Field Communication), or other wireless communication IFs, but is not limited to these.

[0132] The states that are known based on the above-mentioned implementation forms and changes are listed below. [1] An information processing system comprising: a memory for storing a plurality of commands; and at least one processor; wherein the aforementioned at least one processor is configured to execute the aforementioned plurality of commands to: input a prompt to a generative model of machine learning, wherein the aforementioned generative model is configured to output data generated in accordance with the aforementioned instructions once an instruction is input to cause the generative model to generate data; the aforementioned prompt contains: a draft of a notification message to be sent to one or more user terminals, and a correction instruction to cause the aforementioned draft to be corrected; and a draft obtained from the aforementioned generative model by the aforementioned generative model after the aforementioned draft has been corrected in accordance with the aforementioned correction instruction; and input the aforementioned draft to a predictive model of machine learning, wherein the aforementioned predictive model is configured to output a prediction result of an index representing the user's reaction when the aforementioned draft is received once the aforementioned draft is input; and a prediction result of the aforementioned draft output by the aforementioned predictive model is obtained.

[0133] [2] As described in the above [1], the information processing system includes: instructions for generating a plurality of drafts for a single original draft; at least one processor is configured to: obtain from the previous generation model a plurality of drafts generated by the previous generation model according to the previous correction instructions; input the previous plurality of drafts into the previous prediction model; and obtain a plurality of prediction results for each of the previous plurality of drafts output by the previous prediction model.

[0134] [3] As described in the above [2], in the information processing system, at least one processor is configured to send the plurality of drafts of the foregoing, together with the plurality of prediction results of the foregoing for each of the plurality of drafts of the foregoing, to the terminal device used by the person in charge of the creation of the foregoing notification.

[0135] [4] The information processing system described in [2] or [3] above, wherein at least one processor of the foregoing is configured to send a notification to one or more user terminals of the foregoing; the foregoing notification contains one draft selected by the person in charge of the foregoing from among a plurality of drafts of the foregoing, or contains an article in which the person in charge of the foregoing has revised one draft of the foregoing, as a foregoing notification.

[0136] [5] The information processing system described in any of the above [2]~[4], wherein at least one processor of the foregoing is configured to display the one with the best prediction result among the plurality of drafts of the foregoing on the terminal device used by the person in charge of creating the foregoing notification.

[0137] [6] The information processing system described in any of the above [1]~[5], wherein the aforementioned notification text contains an advertisement about a store; the aforementioned at least one processor is configured to obtain store information about the aforementioned store; the aforementioned correction instruction contains: store information about the aforementioned store.

[0138] [7] The information processing system described in any of the above [1]~[6], wherein the aforementioned notification message contains an advertisement for a store; the aforementioned correction instruction contains supplementary information at the time of the generation of the aforementioned advertisement; the aforementioned supplementary information includes: the aforementioned store is a franchisee of the electronic checkout service, and the aforementioned user is a user of the aforementioned electronic checkout service.

[0139] [8] The information processing system described in any of the above [1] to [7], wherein at least one processor is configured to send a notification containing the notification text to one or more user terminals; the notification is created in a format required for display in an application installed in each of the one or more user terminals; the format contains a title and a text of the notification; the draft is at least a part of the title or text of the notification.

[0140] [9] The information processing system described in any of the above [1]~[8], wherein the preceding prediction model is trained using user data of the preceding plurality of users containing multiple attribute values ​​for each of the plurality of users, and is configured to output the prediction results corresponding to the user attributes; the preceding at least one processor is configured to: input the attributes of the user who is scheduled to receive the preceding notification message, together with the preceding draft, into the preceding prediction model; and obtain the preceding prediction results corresponding to the attributes of the preceding user output by the preceding prediction model.

[0141]

[10] The information processing system recorded in any of the above [1]~[9], wherein the aforementioned indicator is an indicator representing the user's level of participation.

[0142]

[11] The information processing system described in any of the above [1]~

[10] , wherein at least one processor is configured to send a notification containing the above notification text to one or more user terminals; the above notification contains a link to a website; and the above metric is the click rate.

[0143]

[12] The information processing system described in any of the above [1]~

[11] includes the following: the instruction to modify the original case in order to improve the value of the previous index.

[0144]

[13] The information processing system described in any of the above [1]~

[12] includes the following: the instruction to modify the previous original in order to make the value of the previous index close to or exceed the specified target value.

[0145]

[14] An information processing method, which is an information processing method executed by at least one processor, includes: inputting a prompt to a machine learning generative model, wherein the preceding generative model is configured to output data generated in accordance with the preceding instruction once an instruction is input to cause the generative model to generate data; the preceding prompt includes: an original of a notification message to be sent to one or more user terminals, and a correction instruction to cause the preceding original to be corrected; and obtaining from the preceding generative model a draft of the preceding original modified according to the preceding correction instruction; and inputting the preceding draft to a machine learning prediction model, wherein the preceding prediction model is configured to output a prediction result of an indicator representing the user's reaction when the preceding draft is received once the preceding draft is input.

[0146]

[15] A program is used to instruct at least one processor to: input a prompt to a generative model of machine learning, wherein the aforementioned generative model is configured to output data generated in accordance with the aforementioned instructions once an instruction is input to instruct the generative model to generate data; the aforementioned prompt contains: an original of a notification message to be sent to one or more user terminals, and a correction instruction to instruct it to correct the aforementioned original; and obtain from the aforementioned generative model a draft of the aforementioned original modified according to the aforementioned correction instruction; and input the aforementioned draft to a predictive model of machine learning, wherein the aforementioned predictive model is configured to output a prediction result of an index representing the user's reaction when the aforementioned draft is received once the aforementioned draft is input. [Simplified Explanation of the Diagram]

[0008] [Figure 1] Figure 1 is a schematic diagram of the information processing system described in the embodiment. [Figure 2] Figure 2 is an example of a message screen displayed on a user terminal. [Figure 3] Figure 3 is an example of a notification display screen displayed on a user terminal. [Figure 4] Figure 4 is an example of an operation screen for notification generation in a notification generation application. [Figure 5] Figure 5 is an example of an operation screen for notification management in the notification generation application of Figure 4. [Figure 6] Figure 6 is an example of a detailed screen that has been migrated from the operation screen of Figure 5. [Figure 7] Figure 7 is an example of a method for generating a draft notification text.

Claims

1. An information processing system comprising: a memory for storing a plurality of commands; and at least one processor; wherein the at least one processor is configured to execute the plurality of commands to: input a prompt into a generative model for machine learning, wherein, The preface generation model is configured to output data generated in accordance with the preface instructions once input into the model. The preface prompt contains: a draft of a notification message to be sent to one or more user terminals, and a correction instruction to correct the draft; and a draft of the preface generated model after correcting the draft according to the correction instruction, obtained from the preface generation model; and the preface draft is input into a machine learning prediction model, wherein the preface prediction model is configured to output a prediction result of an indicator representing the user's reaction when receiving the preface draft once the preface draft is input; and the preface prediction result for the preface draft output by the preface prediction model is obtained.

2. The information processing system as described in claim 1, wherein, The aforementioned revision instruction contains: instructions required to generate a plurality of drafts for a given original draft; at least one processor is configured to: obtain from the aforementioned generation model a plurality of drafts generated by the aforementioned generation model according to the aforementioned revision instruction; input the aforementioned plurality of drafts into the aforementioned prediction model; and obtain a plurality of prediction results for each of the aforementioned plurality of drafts output by the aforementioned prediction model.

3. The information processing system as described in claim 2, wherein, At least one processor is configured to send the plurality of drafts of the foregoing, together with the plurality of prediction results for each of the plurality of drafts of the foregoing, to the terminal device used by the person in charge of creating the foregoing notification.

4. The information processing system as described in claim 3, wherein, At least one processor is configured to send a notification to one or more user terminals mentioned above. The notification may contain one draft selected by the person in charge of the preceding document from among a plurality of drafts of the preceding document, or may contain an article in which the person in charge of the preceding document has revised one draft of the preceding document.

5. The information processing system as described in claim 2, wherein, At least one processor is configured to display, among the plurality of drafts of the foregoing, the one with the best prediction result of the foregoing to the terminal device used by the person in charge of creating the foregoing notification.

6. The information processing system described in any of the requests 1 to 5, wherein the aforementioned notification message contains an advertisement for a store; at least one of the aforementioned processors is configured to acquire store information for the aforementioned store; and the aforementioned correction instruction contains: store information for the aforementioned store.

7. The information processing system described in any of the requests 1 to 5, wherein the aforementioned notification message contains an advertisement for a store; the aforementioned correction instruction contains supplementary information at the time of generation of the aforementioned advertisement; the aforementioned supplementary information includes: the aforementioned store is a franchisee of the electronic checkout service, and the aforementioned user is a user of the aforementioned electronic checkout service.

8. The information processing system described in any of claims 1 to 5, wherein at least one of the aforementioned processors is configured to send a notification containing the aforementioned notification text to one or more of the aforementioned user terminals; the aforementioned notification is created in a format required for display in an application installed on each of the aforementioned one or more user terminals; the aforementioned format contains a title and a text of the aforementioned notification; the aforementioned draft is at least a part of the aforementioned title or the text of the aforementioned.

9. The information processing system described in any of claims 1 to 5, wherein the prior prediction model is trained using user data of the prior plurality of users, which contains multiple attribute values ​​for each of the multiple users, and is configured to output prediction results corresponding to the user attributes; at least one processor is configured to: input the attributes of the users who are scheduled to receive the prior notification message, together with the prior draft, into the prior prediction model; and obtain the prior prediction results corresponding to the attributes of the prior users output by the prior prediction model.

10. For any of the information processing systems described in Request 1 to 5, the aforementioned indicators are indicators representing user participation.

11. The information processing system described in any of the requests 1 to 5, wherein at least one of the processors is configured to send a notification containing the aforementioned notification text to one or more of the aforementioned user terminals; the aforementioned notification contains a link to a website; and the aforementioned metric is a click-through rate.

12. The information processing system described in any of the requests 1 to 5, wherein the aforementioned correction instruction includes: an instruction required to correct the aforementioned original in order to improve the value of the aforementioned index.

13. The information processing system described in any of the requests 1 to 5, wherein the aforementioned correction instruction includes: an instruction required to correct the aforementioned original in order to make the value of the aforementioned indicator close to or exceed the aforementioned target value.

14. An information processing method, which is an information processing method executed by at least one processor, comprising: inputting a prompt into a machine learning generative model, wherein, The preface generation model is configured to output data generated in accordance with the preface instructions once input into the model. The preface prompt contains: a draft of a notification message to be sent to one or more user terminals, and a correction instruction to correct the draft; and a draft of the preface generated model after correcting the draft according to the correction instruction, obtained from the preface generation model; and the preface draft is input into a machine learning prediction model, wherein the preface prediction model is configured to output a prediction result of an indicator representing the user's reaction when the preface draft is received once the preface draft is input.

15. A computer program product for instructing at least one processor to: input word suggestions into a generative model of machine learning, wherein, The preface generation model is configured to output data generated in accordance with the preface instructions once input into the model. The preface prompt contains: a draft of a notification message to be sent to one or more user terminals, and a correction instruction to correct the draft; and a draft of the preface generated model after correcting the draft according to the correction instruction, obtained from the preface generation model; and the preface draft is input into a machine learning prediction model, wherein the preface prediction model is configured to output a prediction result of an indicator representing the user's reaction when the preface draft is received once the preface draft is input.

Images