Information processing method and information processing program

Abstract

To enable flexible determination as to a posted text is inappropriate or not according to the characteristics of a service to which the posted text is posted.SOLUTION: An information processing method according to the present application is an information processing method executed by an information processing device, the information processing method comprising: a reception step for receiving a posted text posted by a posting user; and a generation step for inputting a determination prompt that instructs the user to determine whether the posted text is inappropriate or not, and the posted text into a generation model, and causing the generation model to generate determination information indicating whether the posted text is inappropriate or not.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an information processing method and an information processing program. [Background technology]

[0002] Some of the texts (comments, etc.) posted by users on social networking services (SNS) or online bulletin boards contain inappropriate texts, such as slanderous or libelous comments against individuals. Techniques for reducing the number of such inappropriate texts are known. For example, a technique is known in which a comment received from a user is input into a neural network trained using training data including target information and information indicating whether the target information is an inappropriate comment, and the network determines whether the comment is an inappropriate comment. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-122491 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-mentioned conventional technology, the neural network is simply trained in advance to determine whether a comment is inappropriate or not, and therefore it is not necessarily possible to flexibly determine whether a posted text is inappropriate or not depending on the characteristics of the service to which the posted text is posted.

[0005] Therefore, an object of the present disclosure is to enable flexible determination of whether a posted text is inappropriate or not, depending on the characteristics of the service to which the posted text is posted. [Means for solving the problem]

[0006] The information processing method of the present invention is an information processing method executed by an information processing device, and includes a reception process for receiving a posted text posted by a posting user, a judgment prompt that instructs the user to judge whether the posted text is inappropriate text, and a generation process for inputting the posted text into a generative model and causing the generative model to generate judgment information indicating whether the posted text is inappropriate text.

[0007] The information processing program of the present invention causes a computer to execute a reception procedure for receiving a posted text posted by a posting user, a judgment prompt for instructing the computer to judge whether the posted text is inappropriate, and a generation procedure for inputting the posted text into a generative model and causing the generative model to generate judgment information indicating whether the posted text is inappropriate. [Effects of the Invention]

[0008] According to the present invention, it is possible to flexibly determine whether a posted text is inappropriate or not, depending on the characteristics of the service to which the posted text is posted. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of the information processing device according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a user information table stored in a user information storage unit according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a posted information table stored in the posted information storage unit according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of a determination prompt according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of destination content according to the embodiment. [Figure 7]FIG. 7 is a diagram illustrating an example of destination content according to the embodiment. [Figure 8] FIG. 8 is a flowchart showing the procedure of information processing by the information processing device according to the embodiment. [Figure 9] FIG. 9 is a flowchart showing the procedure of information processing by the information processing device according to the embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a relieving prompt according to the first modification. [Figure 11] FIG. 11 is a diagram showing an example of destination content according to the first modified example. [Figure 12] FIG. 12 is a flowchart showing the procedure of information processing by the information processing device according to the first modified example. [Figure 13] FIG. 13 is a flowchart showing the procedure of information processing by the information processing device according to the first modified example. [Figure 14] FIG. 14 is a diagram showing an example of confirmation content according to the second modified example. [Figure 15] FIG. 15 is a flowchart showing the procedure of information processing by the information processing device according to the second modified example. [Figure 16] FIG. 16 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, a detailed description will be given of an information processing method and an information processing program according to the present application (hereinafter referred to as an "embodiment") with reference to the drawings. Note that the information processing method and the information processing program according to the present application are not limited to the embodiment. Furthermore, the same components in the following embodiments are denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0011] (Embodiment) 1. Introduction In recent years, there are many services that allow an unspecified number of users to post comments and other text via social networking sites, internet bulletin boards, chat rooms, inquiry forms, etc. Among these posts, there are cases where inappropriate content is posted against users and related parties of the service. Specifically, inappropriate content includes meaningless trolling, slander, critical comments, and posts that damage the corporate brand. Furthermore, such inappropriate content may cause harm to individuals, damage the brand of the service in question, or have a negative impact on the quality of the service in question. Therefore, there is a need for technology to reduce the posting of such inappropriate content.

[0012] For example, a technique is known in which a comment received from a user is input into a neural network (an example of a machine learning model) trained using training data including target information and information indicating whether the target information is an inappropriate comment, and the comment is then determined to be inappropriate. However, such a technique requires a great deal of effort because it requires the preparation of a large amount of training data to allow the machine learning model to learn in advance whether a posted sentence is inappropriate. Furthermore, because such a technique merely trains the neural network in advance to learn whether a comment is inappropriate, it does not necessarily enable flexible determination of whether a posted sentence is inappropriate depending on the characteristics of the service to which the posted sentence is posted.

[0013] In response to this, the information processing device 100 according to the embodiment inputs a determination prompt instructing the generative model to determine whether a posted text posted by a user is inappropriate, and the posted text, and causes the generative model to generate determination information indicating whether the posted text is inappropriate. This allows the information processing device 100 to cause the generative model to determine whether the posted text is inappropriate, based on a determination prompt with flexible content according to the characteristics of the service to which the posted text is posted. Therefore, the information processing device 100 can flexibly determine whether the posted text is inappropriate, depending on the characteristics of the service to which the posted text is posted.

[0014] [2. Information Processing System Configuration] Fig. 1 is a diagram showing an example of the configuration of an information processing system according to an embodiment. As shown in Fig. 1, the information processing system 1 includes a terminal device 10 and an information processing device 100. The terminal device 10 and the information processing device 100 are connected to each other via a network N so as to be able to communicate with each other via a wired or wireless connection. The network N is, for example, a wide area network (WAN) such as the Internet. Note that the information processing system 1 shown in Fig. 1 may include a plurality of terminal devices 10 and a plurality of information processing devices 100.

[0015] The terminal device 10 is an information processing device used by a user. The terminal device 10 is realized by, for example, a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), etc. The terminal device 10 displays information received from the information processing device 100, etc., using a web browser or an application.

[0016] The information processing device 100 is an information processing device that performs information processing according to the embodiment. The information processing device 100 is realized by a server device, a cloud system, or the like. For example, the information processing device 100 may perform the information processing according to the embodiment in accordance with an information processing method realized by an information processing program according to the embodiment. The information processing device 100 also distributes content to which a post text is to be posted (hereinafter, may be referred to as "destination content") to the terminal device 10.

[0017] 3. Configuration of Information Processing Device 2 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. The information processing device 100 includes a communication unit 110, a storage unit 120, and a control unit .

[0018] (Communication unit 110) The communication unit 110 is realized by a NIC (Network Interface Card), an antenna, etc. The communication unit 110 is connected to various networks by wire or wirelessly, and transmits and receives information to and from the terminal device 10, for example.

[0019] (Storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. Specifically, the storage unit 120 stores various programs. For example, the storage unit 120 stores an information processing program according to the embodiment. The storage unit 120 also stores various data. The storage unit 120 also has a user information storage unit 121 and a posted information storage unit 122.

[0020] (User information storage unit 121) The user information storage unit 121 stores information about users. Fig. 3 is a diagram showing an example of a user information table stored in the user information storage unit 121 according to the embodiment. In the example shown in Fig. 3, the user information table stored in the user information storage unit 121 includes information on items such as "user ID" and "terminal ID."

[0021] The "user ID" is identification information for identifying a user. The "terminal ID" is identification information for identifying the terminal device 10 of the user identified by the user ID.

[0022] (Posted information storage unit 122) The posted information storage unit 122 stores various types of information posted for each content. Fig. 4 is a diagram showing an example of a posted information table stored in the posted information storage unit 122 according to the embodiment. In the example shown in Fig. 4, the posted information table stored in the posted information storage unit 122 includes information items such as "posting ID," "terminal ID," "destination content ID," "posted text," and "determination information."

[0023] "Posting ID" is identification information that identifies the posted text. "Terminal ID" is identification information that identifies the terminal device 10 that is the sender of the posted text. In other words, "terminal ID" is identification information that identifies the terminal device 10 of the user who posted the posted text (hereinafter, may be referred to as "posting user"). "Destination content ID" is identification information that identifies the content to which the posted text identified by the posting ID is to be posted. "Posted text" is text posted by a user. "Determination information" is information that indicates whether the posted text is inappropriate text.

[0024] (control unit 130) The control unit 130 is a controller, and is realized, for example, by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like, using RAM as a work area to execute various programs (for example, the information processing program according to the embodiment) stored in a storage device inside the information processing device 100. The control unit 130 is also a controller, and is realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0025] The control unit 130 has a receiving unit 131, a generating unit 132, and a distributing unit 133 as functional units, and may realize or execute the information processing actions described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 3, and may be any other configuration that performs the information processing described below. Furthermore, each functional unit indicates a function of the control unit 130, and does not necessarily have to be physically distinct.

[0026] (Reception Department 131) The reception unit 131 receives various information. Specifically, the reception unit 131 receives a posted text posted by a posting user. For example, the reception unit 131 receives the posted text from the terminal device 10 of the posting user, thereby receiving the posted text from the terminal device 10 of the posting user. When the reception unit 131 receives a posted text, the reception unit 131 stores the posted text in the posted information storage unit 122 in association with identification information that identifies the posted text, identification information that identifies the terminal device 10 of the posting user, identification information that identifies the destination content to which the posted text was posted (hereinafter, may be referred to as "destination content").

[0027] (Generation unit 132) The generation unit 132 generates various types of information. For example, the generation unit 132 generates various types of information by causing a generative model to generate the various types of information. Here, the generative model is a machine learning model that generates and outputs information according to input information. Specifically, the generative model is a large-scale language model (LLM). For example, the generative model is a GPT (Generative Pre-trained Transformer) model.

[0028] For example, the generation unit 132 inputs a judgment prompt, which is an instruction sentence instructing the reception unit 131 to determine whether a posted text satisfies a predetermined condition, into the generative model, causing the generative model to generate judgment information indicating whether the posted text satisfies the predetermined condition. For example, when the reception unit 131 receives a posted text, the generation unit 132 inputs the posted text and the judgment prompt received by the reception unit 131 into the generative model, causing the generative model to generate judgment information indicating whether the posted text received by the reception unit 131 satisfies the predetermined condition. The judgment information includes a sentence indicating that the posted text satisfies the predetermined condition or a sentence indicating that the posted text does not satisfy the predetermined condition. Here, the predetermined condition is, for example, a condition for determining that the text is inappropriate. For example, the predetermined condition may be slander, meaningless trolling, critical remarks, or content that leads to brand damage of a company, etc. Furthermore, the judgment prompt includes a sentence indicating the predetermined condition. For example, the judgment prompt includes a sentence indicating the condition for determining that the text is inappropriate.

[0029] The determination prompt may be a flexible sentence depending on the characteristics of the service to which the posted text is posted. For example, if the service to which the posted text is posted is a service that allows relatively free content to be posted, the determination prompt may include a sentence that sets looser conditions for determining that the text is inappropriate. If the service to which the posted text is posted is a service that allows relatively free content to be posted, the determination prompt may include a sentence instructing the system not to be too strict in determining whether the text is inappropriate. If the service to which the posted text is posted is a service that values ​​the reliability of the posted content, the determination prompt may include a sentence instructing the system to be more strict in determining whether the text is inappropriate.

[0030] For example, bulletin board services are known that allow users to post free advertisements for selling or picking up unwanted items. Examples of inappropriate posts on such bulletin board services include posts containing users' personal contact information, phone numbers, or social networking site IDs. Such bulletin board services require flexible determination of whether a post is inappropriate. For example, in conventional rule-based determinations, a user may manipulate their personal contact information, phone number, or ID (e.g., by modifying the display format) to avoid determining that a post is inappropriate. This can also occur when using a machine learning model that has previously trained to determine whether a post is inappropriate. In response to this, the information processing device 100 according to the embodiment may input a determination prompt to the generative model, including an instruction such as, "Personal contact information and social networking site IDs are inappropriate posts. Alphanumeric characters that appear to fit these criteria are inappropriate posts. Please be aware that users may attempt to evade filters by splitting their IDs or contact information or inserting escape characters." This allows the information processing device 100 to generate determination information. This allows flexible determination of whether a post is inappropriate, even for posts that would otherwise evade determination by conventional rule-based or trained machine learning models.

[0031] FIG. 5 is a diagram illustrating an example of a determination prompt according to the embodiment. The generation unit 132 inputs the posted text received by the reception unit 131 and the determination prompt P1 illustrated in FIG. 5 into a generative model, and causes the generative model to generate determination information indicating whether the posted text received by the reception unit 131 is inappropriate. For example, the generation unit 132 causes the generative model to generate, as the determination information, a sentence indicating that the posted text is inappropriate or a sentence indicating that the posted text is not inappropriate. The determination prompt P1 is an instruction statement that instructs the generative model to determine whether a posted comment (an example of a posted text) is inappropriate, along with conditions for determining that the text is inappropriate. For example, the determination prompt P1 includes, as conditions for determining that the text is inappropriate, a sentence indicating that an inappropriate comment (an example of an inappropriate text) is slander, meaningless trolling, critical remarks, or remarks that lead to brand damage to a company, etc. The determination prompt P1 also includes a sentence specifying an output form for outputting a determination result indicating whether the posted text is inappropriate. For example, the determination prompt P1 includes a sentence instructing the generation unit 132 to output determination information "{"is inappropriate": true}" if it determines that the posted sentence is inappropriate, and to output determination information "{"is inappropriate": false}" if it determines that the posted sentence is not inappropriate. Furthermore, if the generation unit 132 determines that the posted sentence is inappropriate, it generates determination information "{"is inappropriate": true}" as a sentence indicating that the posted sentence is inappropriate. Furthermore, if the generation unit 132 determines that the posted sentence is not inappropriate, it generates determination information "{"is inappropriate": false}" as a sentence indicating that the posted sentence is not inappropriate. Furthermore, if the generation unit 132 generates determination information using a generative model, the generation unit 132 associates the posted sentence corresponding to the generated determination information with the determination information and stores the association information in the posted information storage unit 122.

[0032] (Distribution Section 133) The distribution unit 133 distributes various contents to the user's terminal device 10. For example, the distribution unit 133 distributes destination content, which is content to which a posted text is posted, to the terminal device of the browsing user. For example, the reception unit 131 receives a delivery request for destination content from the terminal device 10 of the browsing user. In response to the delivery request received by the reception unit 131, the distribution unit 133 distributes destination content that hides the posted text if the posted text is inappropriate, and distributes destination content that displays the posted text if the posted text is not inappropriate.

[0033] FIG. 6 is a diagram illustrating an example of destination content according to an embodiment. The left side of FIG. 6 illustrates an example of a posting form for posted information displayed on the screen of the terminal device 10 of the posting user. In the left side of FIG. 6, the posting user inputs a posting text PT1, such as "I think it would be easier to understand if we posted information about XX on the homepage. What do you think?" into the posting form and presses the post button B1. In response to the operation of pressing the post button B1, the terminal device 10 of the posting user transmits the posted text PT1 to the information processing device 100. The reception unit 131 of the information processing device 100 receives the posted text PT1 posted by the posting user. Furthermore, the generation unit 132 inputs the posted text PT1 and the determination prompt P1 received by the reception unit 131 into a generative model, causing the generative model to generate determination information indicating whether the posted text PT1 is inappropriate text. The generative model determines that the posted text PT1 is not inappropriate text and outputs determination information J1 (e.g., the text "{'is inappropriate': false}") indicating that the posted text PT1 is not inappropriate text. The generation unit 132 also stores the posted text PT1 and the determination information J1 in the posted information storage unit 122 in association with each other.

[0034] The right side of FIG. 6 shows an example of destination content displayed on the screen of the terminal device 10 of the browsing user. The receiving unit 131 receives a delivery request for destination content, which is content to which the posted text PT1 is to be posted, from the terminal device 10 of the browsing user. In response to the delivery request received by the receiving unit 131, the delivery unit 133 refers to the posted information storage unit 122 and acquires determination information J1 associated with the posted text PT1. The delivery unit 133 determines, based on the determination information J1, that the posted text PT1 is not inappropriate text. If the delivery unit 133 determines that the posted text PT1 is not inappropriate text, it generates destination content C1 that displays the posted text PT1. If the delivery unit 133 generates destination content C1, it delivers the destination content C1 to the terminal device 10 of the browsing user.

[0035] FIG. 7 is a diagram illustrating an example of destination content according to an embodiment. On the left side of FIG. 7, a posting user inputs a posted text PT2, such as "The management is terrible. This site should be shut down," into a posting form and presses the post button. The posting user's terminal device 10 transmits the posted text PT2 to the information processing device 100 in response to the operation of pressing the post button. The reception unit 131 of the information processing device 100 receives the posted text PT2. The generation unit 132 inputs the posted text PT2 and the determination prompt P1 received by the reception unit 131 into a generative model, causing the generative model to generate determination information indicating whether the posted text PT2 is inappropriate. The generative model determines that the posted text PT2 is inappropriate and outputs determination information J2 (e.g., the sentence "{"is inappropriate": true}") indicating that the posted text PT2 is inappropriate. The generation unit 132 then associates the posted text PT2 with the determination information J2 and stores them in the posted information storage unit 122.

[0036] The right side of FIG. 7 shows an example of destination content displayed on the screen of the terminal device 10 of the browsing user. The receiving unit 131 receives a delivery request for destination content, which is content to which the posted text PT2 is to be posted, from the terminal device 10 of the browsing user. In response to the delivery request received by the receiving unit 131, the delivery unit 133 references the posted information storage unit 122 and acquires determination information J2 associated with the posted text PT2. Furthermore, the delivery unit 133 determines that the posted text PT2 is inappropriate text based on the determination information J2. If the delivery unit 133 determines that the posted text PT2 is inappropriate text, it generates destination content C2 that hides the posted text PT2. For example, the delivery unit 133 generates destination content C2 that includes text NP2 that reads, "This opinion has been hidden because it contains inappropriate language." Furthermore, the delivery unit 133 generates destination content C2 that includes a button B2 that, when pressed, unhides the posted text PT2 and displays the posted text PT2. Furthermore, when the distribution unit 133 generates the destination content C2, the distribution unit 133 distributes the destination content C2 to the terminal device 10 of the browsing user.

[0037] [4. Information Processing Procedures] 8 is a flowchart showing the procedure of information processing by the information processing device according to the embodiment. In FIG. 8, the receiving unit 131 receives a posted text posted by a posting user (step S101). When the receiving unit 131 receives the posted text, the generating unit 132 inputs a determination prompt, which instructs the receiving unit 131 to determine whether the posted text is inappropriate, and the posted text into the generative model, causing the generative model to generate determination information indicating whether the posted text is inappropriate (step S102). When the generating unit 132 generates the determination information, the generating unit 132 stores information associating the posted text with the determination information in the posted information storage unit 122 (step S103).

[0038] FIG. 9 is a flowchart showing the procedure of information processing by the information processing device according to the embodiment. In FIG. 9, the receiving unit 131 receives a destination content delivery request from the terminal device of the browsing user (step S201). When the receiving unit 131 receives the destination content delivery request, the delivery unit 133 refers to the posted information storage unit 122 to acquire information associating the posted text with the determination information (step S202). The delivery unit 133 determines whether the posted text is inappropriate based on the information associating the posted text with the determination information (step S203). When the delivery unit 133 determines that the posted text is inappropriate (step S203; Yes), the delivery unit 133 delivers destination content that hides the posted text to the terminal device of the browsing user (step S204). When the delivery unit 133 determines that the posted text is not inappropriate (step S203; No), the delivery unit 133 delivers destination content that displays the posted text to the terminal device of the browsing user (step S205).

[0039] [5. Modifications] The processing according to the above-described embodiment may be implemented in various different forms other than the above embodiment.

[0040] [5-1. First Modified Example] In the above-described embodiment, a case has been described in which the generation unit 132 inputs the posted sentence received by the reception unit 131 and the determination prompt P1 shown in Fig. 5 into the generation model, and causes the generation model to generate determination information indicating whether the posted sentence received by the reception unit 131 is inappropriate. In a first modified example, a case will be described in which, when the posted sentence is inappropriate, the generation unit 132 inputs the posted sentence and a mitigating prompt that instructs the generation model to generate a mitigating sentence that rephrases the posted sentence into the generation model, and causes the generation model to generate the mitigating sentence.

[0041] When a posted text is inappropriate, the generation unit 132 inputs a mitigation prompt that instructs the generation model to generate a mitigating sentence by rephrasing the posted text and the posted text, causing the generation model to generate the mitigating sentence. The mitigation prompt includes, in addition to the sentence included in the determination prompt, an instruction statement that instructs the generation model to generate a mitigating sentence by rephrasing the posted text. When a posted text is inappropriate, the mitigation prompt includes an instruction statement that instructs the generation model to generate a mitigating sentence by converting the posted text into a text with more mitigating expression. The mitigation prompt includes an instruction statement that indicates a direction for generating the mitigating sentence. For example, the mitigation prompt includes an instruction statement that instructs the generation model to rephrase inappropriate content included in the posted text using indirect and milder expression. The mitigation prompt also includes an instruction statement that instructs the generation model to rephrase inappropriate expressions included in the posted text using softer expression. The inappropriate expression is, for example, an aggressive tone.

[0042] FIG. 10 is a diagram illustrating an example of a mitigating prompt according to a first modification. The generation unit 132 inputs the posted sentence received by the reception unit 131 and the mitigating prompt P2 shown in FIG. 10 into a generative model, and causes the generative model to generate determination information indicating whether the posted sentence received by the reception unit 131 is inappropriate. Furthermore, the generation unit 132 inputs the posted sentence received by the reception unit 131 and the mitigating prompt P2 shown in FIG. 10 into the generative model, and causes the generative model to generate a mitigating sentence if the posted sentence received by the reception unit 131 is inappropriate. For example, the mitigating prompt P2 includes a sentence instructing the generative model to output determination information such as "{"is inappropriate": true, "converted_post": "converted"}" if the posted sentence is determined to be inappropriate. Furthermore, the mitigating prompt P2 includes a sentence instructing the generative model to "enter the converted posted comment in converted_post." Furthermore, if the generation unit 132 determines that the posted sentence is inappropriate, it generates a mitigating sentence along with determination information "{"is inappropriate": true}" indicating that the posted sentence is inappropriate. Furthermore, if the generative model generates determination information and a mitigating sentence, the generation unit 132 stores the posted sentence corresponding to the generated determination information, the determination information, and the mitigating sentence in the posted information storage unit 122 in association with each other.

[0043] Furthermore, the receiving unit 131 receives a delivery request for destination content from the terminal device of the browsing user. The delivery unit 133 delivers destination content that displays mitigating text if the posted text is inappropriate, in response to the delivery request received by the receiving unit 131, and delivers destination content that displays the posted text if the posted text is not inappropriate.

[0044] FIG. 11 is a diagram showing an example of destination content according to a first modified example. On the left side of FIG. 11, a posting user inputs a posting text PT3, such as "The management is completely useless. This site should be closed," into a posting form and presses the post button. In response to the operation of pressing the post button, the posting user's terminal device 10 transmits the posted text PT3 to the information processing device 100. The reception unit 131 of the information processing device 100 receives the posted text PT3. Furthermore, the generation unit 132 inputs the posted text PT3 and the mitigation prompt P2 received by the reception unit 131 into a generative model, causing the generative model to generate determination information indicating whether the posted text PT3 is inappropriate text. Furthermore, if the posted text PT3 is inappropriate text, the generation unit 132 causes the generative model to generate a mitigation text. 11, the generative model determines that the posted text PT3 is inappropriate, and generates determination information J3 (for example, the sentence "{"is inappropriate": true}") indicating that the posted text PT3 is inappropriate, and a mitigating sentence MP3 that reads, "There may be room for improvement in the management. Feedback may improve it." The generation unit 132 also stores the posted text PT3, the determination information J3, and the mitigating sentence MP3 in the posted information storage unit 122 in association with each other.

[0045] The right side of FIG. 11 shows an example of destination content displayed on the screen of the terminal device 10 of the browsing user. The receiving unit 131 receives a delivery request for destination content, which is the content to which the posted text PT3 is to be posted, from the terminal device 10 of the browsing user. In response to the delivery request received by the receiving unit 131, the delivery unit 133 refers to the posted information storage unit 122 and acquires determination information J3 associated with the posted text PT3. Furthermore, the delivery unit 133 determines that the posted text PT3 is inappropriate text based on the determination information J3. If the delivery unit 133 determines that the posted text PT3 is inappropriate text, the delivery unit 133 refers to the posted information storage unit 122 and acquires a mitigating sentence MP3 associated with the posted text PT3. Furthermore, if the delivery unit 133 acquires the mitigating sentence MP3, the delivery unit 133 generates destination content C3 that displays the mitigating sentence MP3. For example, the delivery unit 133 generates destination content C3 including a sentence T3 that reads, "The expression in this post has been mitigated." Furthermore, the distribution unit 133 generates destination content C3 including a button B3 that displays the original posted text PT3 when the button is pressed. Furthermore, when the distribution unit 133 generates destination content C3, it distributes the destination content C3 to the terminal device 10 of the browsing user. Note that, similar to FIG. 6, the distribution unit 133 distributes destination content that displays the posted text if the posted text is not inappropriate.

[0046] Fig. 12 is a flowchart showing the procedure of information processing by an information processing device according to a first modified example. In Fig. 12, the receiving unit 131 receives a posted text posted by a posting user (step S301). When the receiving unit 131 receives a posted text and, if the posted text is inappropriate, the generating unit 132 inputs the posted text and a mitigation prompt that instructs the generating unit 132 to generate a mitigating text that rephrases the posted text into a generative model, thereby causing the generating unit 132 to generate determination information and a mitigating text (step S302). When the generating unit 132 generates the determination information and the mitigating text, the generating unit 132 stores information that associates the posted text, the determination information, and the mitigating text in the posted information storage unit 122 (step S303).

[0047] FIG. 13 is a flowchart showing the procedure of information processing by the information processing device according to the first modified example. In FIG. 13, the receiving unit 131 receives a destination content delivery request from the terminal device of the browsing user (step S401). When the receiving unit 131 receives the destination content delivery request, the delivery unit 133 refers to the posted information storage unit 122 to acquire information associating the posted text with the determination information (step S402). The delivery unit 133 determines whether the posted text is inappropriate based on the information associating the posted text with the determination information (step S403). When the delivery unit 133 determines that the posted text is inappropriate (step S403; Yes), the delivery unit 133 delivers destination content displaying a mitigating text to the terminal device of the browsing user (step S404). When the delivery unit 133 determines that the posted text is not inappropriate (step S403; No), the delivery unit 133 delivers destination content displaying the posted text to the terminal device of the browsing user (step S405).

[0048] [5-2. Second Modified Example] In the first modified example described above, the distribution unit 133 distributes destination content that displays a mitigating sentence if the posted text is inappropriate, and distributes destination content that displays the posted text if the posted text is not inappropriate, in response to a distribution request received by the receiving unit 131. In the second modified example, the distribution unit 133 distributes confirmation content that displays either the posted text or a mitigating sentence to the terminal device of the posting user, in a selectable manner. Specifically, the distribution unit 133 distributes confirmation content that displays either the posted text or the mitigating sentence to the terminal device of the posting user. The receiving unit 131 receives selection information related to a selection operation for the posted text or the mitigating sentence.

[0049] FIG. 14 is a diagram showing an example of confirmation content related to the second modified example. On the left side of FIG. 14, a posting user inputs posted text PT4, "The management is completely useless. This site should be closed," into a posting form and presses the post button. In response to the operation of pressing the post button, the posting user's terminal device 10 transmits posted text PT4 to the information processing device 100. The reception unit 131 of the information processing device 100 receives posted text PT4. Furthermore, the generation unit 132 inputs the posted text PT4 and the mitigation prompt P2 received by the reception unit 131 into a generative model, causing the generative model to generate determination information indicating whether the posted text PT4 is inappropriate text. Furthermore, if the posted text PT4 is inappropriate text, the generation unit 132 causes the generative model to generate a mitigation text. In Figure 14, the generative model determines that the posted text PT4 is inappropriate, and generates judgment information J4 indicating that the posted text PT4 is inappropriate (e.g., the text "{"is inappropriate": true}") and a mitigation text MP4 stating, "There may be room for improvement in the management. Feedback may improve it."

[0050] Also, on the right side of FIG. 14 , the distribution unit 133 generates confirmation content C4 that displays either the posted text PT4 or the mitigating text MP4 in a selectable manner. The distribution unit 133 generates confirmation content C4 to allow the poster to select either the posted text PT4 or the mitigating text MP4. For example, the distribution unit 133 generates confirmation content C4 including text T4 that reads, "This text may contain inappropriate language. AI has created a text with the mitigated language. Please select and post either the mitigated text or the original text." The distribution unit 133 also generates confirmation content C4 including a button B4 that posts the text selected by the poster by pressing the button. The reception unit 131 receives selection information regarding the selection operation for the posted text PT4 or the mitigating text MP4 in response to the press of button B4. If the distribution unit 133 determines, based on the selection information received by the reception unit 131, that the mitigating text MP4 has been selected, the distribution unit 133 stores information associating the mitigating text MP4 with determination information J4 in the posted information storage unit 122. On the other hand, when the distribution unit 133 determines that the posted text PT4 has been selected based on the selection information received by the reception unit 131, the distribution unit 133 stores information associating the posted text PT4 with the determination information J4 in the posted information storage unit 122.

[0051] FIG. 15 is a flowchart showing the procedure of information processing by an information processing device according to a second modified example. In FIG. 15, the receiving unit 131 receives a posted text posted by a posting user (step S501). When the receiving unit 131 receives the posted text, if the posted text is inappropriate, the generating unit 132 inputs the posted text and a mitigation prompt that instructs the generating unit 132 to generate a mitigation text that paraphrases the posted text into a generative model, thereby causing the generating model to generate determination information and a mitigation text (step S502). The distributing unit 133 determines whether the posted text is inappropriate based on the determination information (step S503). When the distributing unit 133 determines that the posted text is inappropriate (step S503; Yes), the distributing unit 133 distributes confirmation content that displays either the posted text or a mitigation text so that the user can select it to the terminal device of the posting user (step S504).

[0052] The receiving unit 131 also receives selection information related to a selection operation on a posted text or a mitigating text. The distribution unit 133 determines whether a mitigating text has been selected based on the selection information received by the receiving unit 131 (step S505). If the distribution unit 133 determines that a mitigating text has been selected (step S505; Yes), the distribution unit 133 stores information associating the mitigating text with the determination information in the posted information storage unit 122 (step S506). On the other hand, if the distribution unit 133 determines that a mitigating text has not been selected (step S505; No), the distribution unit 133 stores information associating the posted text with the determination information in the posted information storage unit 122 (step S507). If the distribution unit 133 determines that the posted text is not inappropriate (step S503; No), the distribution unit 133 stores information associating the posted text with the determination information in the posted information storage unit 122 (step S507).

[0053] [6. Effects] As described above, the information processing device 100 according to the embodiment includes the receiving unit 131 and the generating unit 132. The receiving unit 131 receives a posted sentence posted by a posting user. The generating unit 132 inputs the posted sentence and a determination prompt that instructs the posting user to determine whether the posted sentence is inappropriate, into a generative model, and causes the generative model to generate determination information indicating whether the posted sentence is inappropriate.

[0054] This allows information processing device 100 to have the generative model determine whether or not a posted text is inappropriate, based on flexible content determination prompts according to the characteristics of the service to which the posted text is posted. Therefore, information processing device 100 can flexibly determine whether or not a posted text is inappropriate, according to the characteristics of the service to which the posted text is posted.

[0055] The information processing device 100 further includes a delivery unit 133 that delivers destination content, which is content to which a posted text is to be posted, to the terminal device of the browsing user. The reception unit 131 receives a delivery request for destination content from the terminal device of the browsing user. In response to the delivery request received by the reception unit 131, the delivery unit 133 delivers destination content that hides the posted text if the posted text is inappropriate, and delivers destination content that displays the posted text if the posted text is not inappropriate.

[0056] This allows the information processing device 100 to hide the posted text if it is inappropriate, thereby preventing specific individuals, businesses, etc. from suffering any disadvantages due to the posting of inappropriate text.

[0057] Furthermore, if the posted sentence is inappropriate, the generation unit 132 inputs a mitigating prompt that instructs the generation of a mitigating sentence that rephrases the posted sentence and the posted sentence into the generation model, and causes the generation model to generate the mitigating sentence.

[0058] As a result, when a posted text is inappropriate, the information processing device 100 can generate a mitigating text that paraphrases the posted text, and can therefore display the mitigating text instead of the inappropriate posted text, for example. Furthermore, since the information processing device 100 can display the mitigating text instead of the inappropriate posted text, it is possible to prevent a specific person, business, or the like from suffering any disadvantage due to the posting of an inappropriate posted text.

[0059] The information processing device 100 further includes a delivery unit 133 that delivers destination content, which is content to which a posted text is to be posted, to the terminal device of the browsing user. The reception unit 131 receives a delivery request for destination content from the terminal device of the browsing user. In response to the delivery request received by the reception unit 131, the delivery unit 133 delivers destination content that displays mitigating text if the posted text is inappropriate text, and delivers destination content that displays the posted text if the posted text is not inappropriate text.

[0060] This allows the information processing device 100 to display mitigating text if the posted text is inappropriate, thereby preventing specific individuals, businesses, etc. from suffering disadvantages due to the posting of inappropriate text.

[0061] The information processing device 100 further includes a distribution unit 133 that distributes confirmation content that displays either the posted text or the mitigating text to the terminal device of the posting user. The reception unit 131 receives selection information related to a selection operation on the posted text or the mitigating text.

[0062] As a result, if the posted text is inappropriate, the information processing device 100 can allow the posting user to select the posted text or the mitigating text, so that, for example, if the posting user selects the mitigating text, it is possible to prevent an inappropriate posted text from being posted.

[0063] [7. Hardware Configuration] The information processing device 100 according to the embodiment described above is realized by, for example, a computer 1000 configured as shown in Fig. 16. Fig. 16 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 100. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0064] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, programs that depend on the hardware of the computer 1000, and the like.

[0065] The HDD 1400 stores programs executed by the CPU 1100, data used by such programs, etc. The communication interface 1500 receives data from other devices via a predetermined communication network and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the predetermined communication network.

[0066] The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input / output interface 1600. The CPU 1100 acquires data from the input devices via the input / output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input / output interface 1600.

[0067] Media interface 1700 reads a program or data stored in recording medium 1800 and provides it to CPU 1100 via RAM 1200. CPU 1100 loads the program or data from recording medium 1800 onto RAM 1200 via media interface 1700 and executes the loaded program. Recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0068] For example, when the computer 1000 functions as the information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the control unit 130. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a predetermined communication network.

[0069] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be implemented in other forms that include the embodiments described in the Disclosure of the Invention section and that have been modified and improved in various ways based on the knowledge of those skilled in the art.

[0070] [8. Other] Furthermore, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0071] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0072] Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content. [Explanation of symbols]

[0073] 100 Information processing device 110 Communications Department 120 Storage section 121 User information storage unit 122 Posted information storage unit 130 control section 131 Reception 132 Generation part 133 Distribution Department

Claims

1. An information processing method executed by an information processing device, a receiving step of receiving a post posted by a posting user; a generation step of inputting a judgment prompt that instructs the user to judge whether the posted text is inappropriate, the judgment prompt instructing the user to change the criteria for judging whether the posted text is inappropriate depending on the characteristics of the service to which the posted text is posted, and the posted text into a generative model, and causing the generative model to generate judgment information indicating whether the posted text is inappropriate; An information processing method including:

2. The method further includes a delivery step of delivering destination content, which is content to which the posted text is to be posted, to a terminal device of a browsing user, The receiving step includes: receiving a delivery request for the destination content from the terminal device of the browsing user; The delivery step includes: In response to the delivery request received in the receiving step, the destination content is delivered so that the posted text is not displayed if the posted text is inappropriate, and the destination content is delivered so that the posted text is displayed if the posted text is not inappropriate. The information processing method according to claim 1 .

3. The generating step includes: If the posted sentence is inappropriate, a mitigation prompt instructing the generative model to generate a mitigating sentence that paraphrases the posted sentence and the posted sentence are input to the generative model, and the generative model generates the mitigating sentence. The information processing method according to claim 1 .

4. The method further includes a delivery step of delivering destination content, which is content to which the posted text is to be posted, to a terminal device of a browsing user, The receiving step includes: receiving a delivery request for the destination content from the terminal device of the browsing user; The delivery step includes: In response to the delivery request received in the receiving step, if the posted text is inappropriate, the destination content is delivered, displaying the mitigating text, and if the posted text is not inappropriate, the destination content is delivered, displaying the posted text. The information processing method according to claim 3 .

5. a delivery step of delivering confirmation content that displays the posted text or the alleviation text in a selectable manner to the terminal device of the posting user; The receiving step includes: receiving selection information regarding a selection operation on the posted sentence or the alleviation sentence; The information processing method according to claim 3 .

6. A procedure for accepting posts submitted by users; A determination prompt that instructs a determination prompt to determine whether the posted text is inappropriate, wherein the determination prompt instructs a determination prompt to change the criteria for determining whether the posted text is inappropriate according to the characteristics of the service to which the posted text is posted, and a generation procedure that inputs the posted text into a generation model and causes the generation model to generate determination information indicating whether the posted text is inappropriate, An information processing program that causes a computer to execute the above.

Images