System

Abstract

A system is provided.SOLUTION: A generative system for processing generated data, comprising: means for receiving data generated by a user from a client device; means for processing the generated data using a generative artificial intelligence model; and means for transmitting a result of the processing to the client device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] User-generated content is rapidly increasing on modern online platforms. This growth has led to significant challenges in scaling and implementing content moderation. Accurate, real-time content moderation is particularly difficult on Japanese digital platforms. Traditional methods rely on manual moderation, which requires a large amount of human labor and is often inefficient. Furthermore, manual moderation is prone to human subjectivity and delays, making it difficult to consistently comply with community guidelines and ensure user safety. Technology is needed to solve these problems and achieve efficient and accurate content moderation. [Means for solving the problem]

[0005] The present invention relates to a generating system for processing generated data. The system includes a means for receiving user-generated data from a client device, a means for processing the generated data using a generative artificial intelligence model, and a means for transmitting the processing results to the client device. This configuration enables the system to moderate user-generated content in real time and filter content in compliance with community guidelines. In particular, by utilizing a generative artificial intelligence model, the system can efficiently process large amounts of content and achieve consistent moderation. This results in a safe and healthy digital environment for users. Specifically, the system has a function for checking whether content violates community guidelines and providing feedback on the results to the client device. This technology enables faster and more accurate responses than manual moderation.

[0006] A "generation system" is a system for processing data generated by a user and providing the results of that processing.

[0007] "Client device" refers to a terminal device used by a user to transmit generated data to the system and receive moderation results.

[0008] "User-Generated Data" means content or information entered or created by a user using a client device.

[0009] A "generative artificial intelligence model" is an algorithm or program for analyzing, evaluating, filtering, and other processes for the content of data generated using artificial intelligence technology.

[0010] "Processing" refers to filtering, evaluating, analyzing, and other operations performed on user-generated data using a generative artificial intelligence model.

[0011] "Moderation results" refers to data and its evaluation results after it has been processed by a generative artificial intelligence model.

[0012] "Community guidelines" are standards of conduct and content restrictions that users must follow, established by the operators of online platforms.

[0013] "Filtering" is the process of automatically filtering out, modifying, or marking inappropriate content that may violate community guidelines.

[0014] "Real-time" refers to near-instant processing and response, meaning that user input is answered immediately. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention relates to a generation system for automating the moderation of user-generated data and maintaining the integrity of content on online platforms, comprising a set of means for receiving data posted from a client device, processing the data using a generative artificial intelligence model, and transmitting the processing results to the client device.

[0037] System configuration

[0038] server:

[0039] The server uses the Flask framework to build a web server and sets up an endpoint that accepts requests from client devices.

[0040] The server receives user-submitted data and evaluates and filters the content using a generative artificial intelligence model.

[0041] The server formats the evaluation and filtering results and sends them back to the client device as a response.

[0042] Device:

[0043] The terminal sends a request containing user-generated data to the server.

[0044] The terminal displays the moderation results received from the server to the user.

[0045] User:

[0046] Users use their terminals to access the online platform, input content to post, and send it.

[0047] Users receive the results of the moderation and can modify their posts as necessary.

[0048] Program processing flow

[0049] server:

[0050] 1. The server receives user-generated data sent as a POST request from a client device.

[0051] 2. The server takes the received data and sends it to the generative artificial intelligence model.

[0052] 3. The generative AI model determines whether the data complies with community guidelines and generates an evaluation result.

[0053] 4. The server formats the evaluation results from the generative artificial intelligence model and sends them back to the client device along with the original data.

[0054] Device:

[0055] 1. The terminal converts the data entered by the user into JSON format and sends a POST request to the server.

[0056] 2. The device receives the response sent back from the server and displays it to the user.

[0057] Specific examples

[0058] For example, if a user posts content like this:

[0059] "This game is so fun! But that player said some horrible things."

[0060] 1. The user enters this information into the terminal and presses the send button.

[0061] 2. The device sends the posted content to the server.

[0062] 3. The server receives the posted content and requests the generative AI model to evaluate the content.

[0063] 4. The generative AI model analyzes the content of the post and generates an evaluation result, such as "avoid making negative comments about other players."

[0064] 5. The server receives the evaluation results, formats them together with the original post content, and sends them back to the device.

[0065] 6. The device displays the moderation results to the user and prompts them to correct their post.

[0066] This allows users to recognize and correct when their posts do not comply with community guidelines, and provides results that help maintain a healthy digital environment across online platforms.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] A user enters content and presses the submit button. For example, they might enter, "This game is really fun! But that player said some horrible things."

[0070] Step 2:

[0071] The device converts the input content into JSON format, for example:

[0072] json

[0073] {

[0074] "content": "This game is really fun! But that player said some horrible things."

[0075] }

[0076] Step 3:

[0077] The device sends the converted JSON data to the server as an HTTP POST request. The request destination is the endpoint ( / filter) configured in the Flask framework.

[0078] Step 4:

[0079] The server processes the POST request received from the client device. At that time, it extracts the JSON data from the request body and extracts the user-submitted content. In this case, the extracted content is "This game is really fun! But that player said some horrible things."

[0080] Step 5:

[0081] The server passes the extracted content to the moderate_content function. This function uses a generative AI model to evaluate the content. Specifically, it calls the OpenAI API to perform content moderation. For example, it generates the following prompt:

[0082] Moderate the following content according to the community guidelines: "This game is really fun! But that player said some horrible things."

[0083] Step 6:

[0084] A generative AI model evaluates the content based on the prompts received and generates moderation results, such as "Avoid negative comments about other players."

[0085] Step 7:

[0086] The server receives the moderation results returned by the generative AI model and formats them along with the original content, creating a JSON response like this:

[0087] json

[0088] {

[0089] "original_content": "This game is really fun! But that player said some horrible things.",

[0090] "moderation_result": "Please avoid negative comments about other players."

[0091] }

[0092] Step 8:

[0093] The server sends a formatted JSON response to the client device.

[0094] Step 9:

[0095] The device receives the response from the server and displays the moderation results to the user, for example, feedback such as "Please avoid making negative comments about other players" is displayed on the screen.

[0096] Step 10:

[0097] The user then edits and corrects the content based on the feedback received and resubmits it if necessary.

[0098] Example 1

[0099] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0100] While the amount of user-generated content on modern online platforms is increasing, there is a need to ensure the quality and integrity of that content. However, manual moderation requires a great deal of effort and time, and it is difficult to respond in real time. This creates a risk that inappropriate content will be published, potentially damaging the integrity of the platform. To solve this problem, an automated moderation system is needed.

[0101] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0102] In this invention, the server includes means for receiving user-generated data from a client device, means for processing the generated data using a generative AI model, means for transmitting the processing results to the client device, communication means for accepting requests from the client device, data formatting means for transmitting data to the generative AI model, and result formatting means for formatting and returning evaluation results from the generative AI model, thereby enabling automatic evaluation and filtering of user-generated content in real time and immediate provision of moderation results.

[0103] A "client device" is an electronic device through which a user inputs information and transmits data to a server.

[0104] "Server" means a computer system that processes data received from a client device and performs evaluation and filtering using a generative AI model.

[0105] "User-Generated Data" is information generated and transmitted by a user through a client device.

[0106] A "generative AI model" is an artificial intelligence algorithm used to analyze user-generated data and generate evaluation results.

[0107] The "data formatting means for sending to the generative AI model" is a function within the server that converts the received data into a format that is easy for the generative AI model to process.

[0108] The "result formatting means for formatting and returning the evaluation results" is a function within the server that converts the evaluation results obtained from the generative AI model into a format that is easy for the client device to understand and transmits them.

[0109] "Communication means" refers to the interface and protocol for transmitting and receiving data between the client device and the server.

[0110] The system of the present invention aims to automate the moderation of user-generated data and maintain the integrity of content on online platforms. The system comprises a set of means including a client device, a server, and a generative AI model.

[0111] Server Configuration

[0112] The server has the functionality to receive user-generated data from the client device, send that data to the generative AI model, obtain the evaluation results, format them and send them back to the client device. The server is built using the following technologies:

[0113] Web framework: Flask

[0114] Generative AI model: OpenAI's GPT-4 API (specific name generalization)

[0115] The server uses the Flask framework to receive POST requests sent from the client device, then formats the received data to send to the generative AI model, which analyzes the data and generates an evaluation result. The server formats the evaluation result and sends a response back to the client device.

[0116] Device configuration

[0117] The device has the function of converting data generated by the user into JSON format and sending it to the server, and also receives the response sent back from the server and displays the result to the user.

[0118] Data transmission method: HTTP request library such as fetch API

[0119] User Interface: HTML, CSS, JavaScript

[0120] When a user enters data such as a comment or post on their device and presses the send button, the content is sent to the server, and when a response is returned from the server, the result is displayed on the screen.

[0121] User Actions

[0122] Users access the online platform using their devices, enter and submit their posts, receive moderation results, and amend their posts as necessary.

[0123] Input method: keyboard, touchscreen, etc.

[0124] Specific examples

[0125] For example, consider the case where a user posts content such as, "This game is really fun! But that player said some horrible things."

[0126] 1. User: Enter this information into the terminal and press the send button.

[0127] 2. Device: Send this post to the server.

[0128] 3. Server: Receives the posted content and requests the generative AI model to evaluate the content.

[0129] 4. Generative AI model: Analyzes the content of the post and generates an evaluation result such as "avoid making negative comments about other players."

[0130] 5. Server: Receives the evaluation results, formats them together with the original post, and sends them back to the device.

[0131] 6. On the device: Display the moderation results to the user and prompt them to correct their post.

[0132] Prompt Sentence Examples

[0133] The generative AI model evaluates posts by providing prompts like the following:

[0134] "This game is so fun! But that player said some horrible things."

[0135] Rate Us: Rate this content to see if it adheres to our Community Guidelines and let us know what changes need to be made.

[0136] Based on this prompt, the generative AI model generates appropriate feedback and provides the results to the user via the server.

[0137] This allows users to ensure that their posts comply with community guidelines, maintaining a healthy digital environment across online platforms.

[0138] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0139] Step 1:

[0140] The server receives user-generated data sent as a POST request from the client device. The input is user-generated data in JSON format, for example, {"content": "This game is really fun!"}. The output is to store the received data object for internal processing. Specifically, it uses the Flask framework to set up an endpoint / api / moderate to receive the request and extract the data from the request body.

[0141] Step 2:

[0142] The server formats the received data and prepares it to be sent to the generative AI model. The input is the data object received in the previous step, e.g., {"content": "This game is really fun!"}. The output is formatted data to be sent to the generative AI model, e.g., {"texts": ["This game is really fun!"]}. Specifically, it converts the data into formatted JSON format and prepares an API request to send to the generative AI model (GPT-4 API).

[0143] Step 3:

[0144] The generative AI model receives and analyzes formatted data. The input is formatted JSON data, and the output is a JSON response containing the evaluation results. For example, if the input is {"texts": ["This game is really fun!"]}, the output will be an evaluation result such as {"evaluation": "appropriate"}. In concrete terms, the generative AI model analyzes the data and evaluates each text based on community guidelines.

[0145] Step 4:

[0146] The server receives the evaluation results returned from the generative AI model, formats them together with the original data, and sends them back to the client device. The input is the evaluation result JSON from the generative AI model, for example, {"evaluation": "appropriate"}. The output is a formatted JSON response to be sent back to the client device, for example, {"original_content": "This game is really fun!", "evaluation": "appropriate"}. Specifically, the server formats the evaluation results, integrates them with the original data, and sends them back to the client device.

[0147] Step 5:

[0148] The device receives the response returned from the server and displays it to the user. The input is the JSON response returned from the server, for example, {"original_content": "This game is really fun!", "evaluation": "Appropriate"}. The output is the evaluation result message displayed to the user, for example, "Evaluation result: Appropriate". Specifically, it uses JavaScript to parse the response and reflects the result in the HTML DOM.

[0149] Step 6:

[0150] The user checks the moderation results and modifies the post as necessary. The input is the evaluation result displayed on the device and the original post content, for example, "Evaluation result: appropriate" and "This game is really fun!". The output is the modified post content, for example, "This game is really fun!". Specifically, the user re-enters the post based on the presented evaluation results, reviews the content, and resubmits it from the device.

[0151] (Application example 1)

[0152] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0153] In recent years, online chat and messaging services have rapidly become popular, making maintaining the integrity of user-generated content a key challenge. Messages containing inappropriate language, suicidal thoughts, and violent language are particularly problematic. However, moderating this content in real time is extremely difficult, necessitating the development of automated and efficient solutions.

[0154] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0155] In this invention, the server includes means for receiving user-generated data from a client device, means for processing the generated data using a generative artificial intelligence model, means for transmitting the processing results to the client device, means for moderating user-generated messages in real time, and means for detecting inappropriate content and providing a warning or appropriate response information, thereby making it possible to maintain the integrity of content in chat and messaging services in real time and prevent the spread of inappropriate content.

[0156] A "generation system" is a system that processes and filters user-generated data and transmits the results to a client device.

[0157] A "client device" is a device (e.g., a smartphone or personal computer) that a user uses to input data and communicate with the production system.

[0158] "User-Generated Data" means text messages and other data that a user generates and sends through a client device.

[0159] A "generative artificial intelligence model" is an artificial intelligence-based software model for analyzing received data and performing appropriate processing and filtering.

[0160] "Moderation" is the process of determining whether the generated data complies with community guidelines, etc., and correcting the content or issuing a warning as necessary.

[0161] "Real-time" means that user data is processed and filtered almost instantly after it is generated.

[0162] "Filtering" is the process of detecting and removing or flagging inappropriate content from generated data.

[0163] A "warning" is a message that notifies the user when the data generated by the user is inappropriate and urges the user to correct it.

[0164] "Response information" refers to information about appropriate assistance and support provided to users when inappropriate content is detected.

[0165] "Integrity" refers to the content on online platforms being socially appropriate and being safe for users to use.

[0166] The present invention provides a system for real-time moderation of data generated by users in online chat and messaging services to prevent the spread of inappropriate content. The system comprises a client device, a server, and a generative artificial intelligence model.

[0167] System configuration

[0168] server:

[0169] The server uses the Flask framework to build a web server and set up endpoints to accept requests from client devices. The server receives user-generated data, processes and filters it using a generative artificial intelligence model, and formats the results and sends them back to the client device as a response.

[0170] Device:

[0171] The device converts the data generated by the user into JSON format and sends it to the server as a POST request. The device receives the moderation results returned by the server and displays them to the user.

[0172] User:

[0173] Users access online chat and messaging services using their devices, input and send messages, and receive moderation results, allowing them to modify the content of their messages as necessary.

[0174] Program processing flow

[0175] server:

[0176] The server uses the Flask framework to build a web server and set up an endpoint. It receives messages sent by users as POST requests from client devices. The server sends the data to a generative artificial intelligence model (e.g., OpenAI's GPT-3) to evaluate whether the message is appropriate. It then sends a response containing the evaluation results back to the client device.

[0177] Device:

[0178] The device converts the data generated by the user into JSON format and sends it to the server as a POST request, receives the response sent back from the server, and displays it to the user.

[0179] Specific examples

[0180] For example, if a user types the following message into chat:

[0181] "There's no point in living anymore."

[0182] 1. The user enters this information into the terminal and presses the send button.

[0183] 2. The terminal sends this message to the server.

[0184] 3. The server receives the message and requests an evaluation from the generative artificial intelligence model.

[0185] 4. The generative AI model analyzes the message and generates an evaluation result, such as, "This message may contain suicidal thoughts. Please consult a specialist."

[0186] 5. The server receives the evaluation result, formats it together with the original message, and sends it back to the terminal.

[0187] 6. The device displays the moderation results to the user and provides appropriate support information as needed.

[0188] Prompt Sentence Examples

[0189] "Does this message possibly contain suicidal thoughts? 'There's no point in living anymore.'\n\nPlease suggest an appropriate response to this message."

[0190] The present invention makes it possible to maintain the integrity of content in chat and messaging services in real time and prevent the spread of inappropriate content.

[0191] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0192] Step 1:

[0193] A user enters a message on the chat or messaging service and presses the send button, which causes the entered message to be received by the client device.

[0194] Input: The message entered by the user

[0195] Output: Message sent to the client device

[0196] Step 2:

[0197] The device converts the message entered by the user into JSON format and sends a POST request to the server, encoding the text as message data in JSON format.

[0198] Input: The message entered by the user

[0199] Output: POST request containing message data in JSON format

[0200] Step 3:

[0201] The server receives the POST request sent from the client device, analyzes the received data, and extracts the message content.

[0202] Input: A POST request containing message data in JSON format.

[0203] Output: Extracted text message

[0204] Step 4:

[0205] The server then sends the extracted messages to a generative artificial intelligence model for appropriate moderation evaluation, specifically using OpenAI's GPT-3 model to analyze the content of the messages and determine their appropriateness.

[0206] Input: Extracted text message

[0207] Output: Moderation evaluation results and warning messages

[0208] Step 5:

[0209] The generative AI model analyzes the content of received messages and evaluates their appropriateness based on the prompt. For example, it generates a response to a prompt such as, "'There's no point in living anymore.' Could this message contain suicidal thoughts?"

[0210] Input: Extracted text message and prompt

[0211] Output: Suitability assessment and warning messages

[0212] Step 6:

[0213] The server receives the evaluation results from the generative artificial intelligence model, formats them, and sends them back to the client device along with the original message.

[0214] Input: Moderation evaluation results and warning messages

[0215] Output: A response containing the formatted evaluation result and the original message.

[0216] Step 7:

[0217] The terminal receives the response from the server and displays the moderation results to the user, who can then modify the message content as necessary.

[0218] Input: Formatted evaluation results and original message

[0219] Output: Moderation results and prompts shown to the user

[0220] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0221] The present invention relates to a generation system for moderating user-generated data in real time to maintain the integrity of content on online platforms. The system includes a set of means for receiving data posted from a client device, processing the data using a generative artificial intelligence model, and transmitting the processing results to the client device. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system achieves more advanced moderation based on the user's emotional state.

[0222] System configuration

[0223] server:

[0224] The server uses the Flask framework to build a web server and sets up an endpoint that accepts requests from client devices.

[0225] The server receives user-submitted data and evaluates and filters the content using an emotion engine and generative artificial intelligence models.

[0226] The server formats the evaluation and filtering results and sends them back to the client device as a response.

[0227] Device:

[0228] The terminal sends a request containing user-generated data to the server.

[0229] The terminal displays the moderation results received from the server to the user.

[0230] User:

[0231] Users use their terminals to access the online platform, input content to post, and send it.

[0232] Users receive the results of the moderation and can modify their posts as necessary.

[0233] Details of system processing

[0234] Processing flow

[0235] server:

[0236] 1. The server receives user-generated data sent as a POST request from a client device.

[0237] 2. The server retrieves the received data and sends it to the emotion engine, which analyzes the emotions from the user-generated data and provides the results to the generation system.

[0238] 3. The server requests the generative artificial intelligence model to evaluate the content based on the emotion analysis results obtained from the emotion engine.

[0239] 4. The biointelligence model uses this data and sentiment analysis results to moderate the content, for example, easing the rating if it contains positive sentiment and stricter rating if it contains negative sentiment.

[0240] 5. The server formats the evaluation results from the biointelligence model and sends them back to the client device along with the original data.

[0241] Device:

[0242] 1. The terminal converts the data entered by the user into JSON format and sends a POST request to the server.

[0243] 2. The device receives the response sent back from the server and displays it to the user.

[0244] Specific examples

[0245] For example, if a user posts content like this:

[0246] "This game is so fun! But that player said some horrible things."

[0247] 1. The user enters this information into the terminal and presses the send button.

[0248] 2. The device sends the post to the server.

[0249] 3. The server receives the post and requests the emotion engine to analyze the sentiment of the content.

[0250] 4. The sentiment engine recognizes that the post contains negative sentiment (e.g., negative comments).

[0251] 5. The server requests strict moderation of the biointelligence model based on the results from the emotion engine.

[0252] 6. The biointelligence model analyzes the content of the posts and generates evaluation results such as "avoid making negative comments about other players."

[0253] 7. The server receives the evaluation results, formats them together with the original post content, and sends them back to the device.

[0254] 8. The device displays the moderation results to the user and prompts them to correct their post.

[0255] This allows users to recognize when content violates guidelines and correct it, and the emotional engine enables appropriate moderation based on the user's emotional state, helping to provide a healthy digital environment across online platforms.

[0256] The processing flow will be explained below.

[0257] Step 1:

[0258] The user enters content and presses the post button. For example, they might enter, "This game is really fun! But that player said some horrible things."

[0259] Step 2:

[0260] The terminal converts the input content into JSON format, which looks like this:

[0261] json

[0262] {

[0263] "content": "This game is really fun! But that player said some horrible things."

[0264] }

[0265] Step 3:

[0266] The device sends the converted JSON data to the server as an HTTP POST request. The request destination is the endpoint ( / filter) configured in the Flask framework.

[0267] Step 4:

[0268] The server receives a POST request from the client device. In this case, the received data is "This game is really fun! But that player said some horrible things."

[0269] Step 5:

[0270] The server sends the received data to the emotion engine, which analyzes the emotions from the user-generated data and determines, for example, "positive" or "negative" emotions.

[0271] Step 6:

[0272] The emotion engine analyzes the input content and recognizes that it contains negative emotions. For example, the phrase "he said horrible things" is identified as a negative emotion.

[0273] Step 7:

[0274] The server requests strict moderation evaluation from the generative artificial intelligence model based on the analysis results of negative emotions obtained from the emotion engine.

[0275] Step 8:

[0276] A generative AI model will critically evaluate your submission, generating prompts such as:

[0277] Moderate the following content according to the community guidelines, considering that it includes negative sentiment: "This game is really fun! But that player said some horrible things."

[0278] Step 9:

[0279] A generative AI model generates content evaluation results, such as a moderation result like "Please avoid making negative comments about other players."

[0280] Step 10:

[0281] The server receives the moderation results returned by the generative AI model and formats them along with the original content, creating a JSON response like this:

[0282] json

[0283] {

[0284] "original_content": "This game is really fun! But that player said some horrible things.",

[0285] "moderation_result": "Please avoid negative comments about other players."

[0286] }

[0287] Step 11:

[0288] The server sends a formatted JSON response to the client device.

[0289] Step 12:

[0290] The device receives the response from the server and displays the moderation results to the user, for example, feedback such as "Please avoid making negative comments about other players" is displayed on the screen.

[0291] Step 13:

[0292] The user then edits and corrects the content based on the feedback received and resubmits it if necessary.

[0293] Example 2

[0294] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0295] Modern online platforms require the maintenance of the integrity of content generated by users, as a large amount of content is posted in real time. However, manual moderation is time-consuming and costly, and it is difficult to cover all content. To solve this problem, an automated real-time moderation system is needed. Furthermore, advanced moderation that takes user sentiment into account is required, rather than simply moderating content.

[0296] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0297] In this invention, the server includes means for receiving user-generated data from a client device, means for transmitting the generated data to a sentiment analysis engine to perform sentiment analysis, means for requesting data moderation using a generative AI model based on the sentiment analysis result, and means for formatting and transmitting the moderation result to the client device, thereby automatically and efficiently maintaining content integrity in real time and enabling more advanced moderation based on emotional states.

[0298] A "client device" is a terminal device used by a user, and is a device for sending and receiving data to a server via the Internet.

[0299] "User-generated data" refers to content, such as text, images, and video, entered by a user and transmitted through a client device.

[0300] A "sentiment analysis engine" is a software or hardware component for extracting and analyzing emotional information from user-generated data.

[0301] A "generative AI model" is a program or system that uses artificial intelligence techniques to generate, process, and analyze data.

[0302] "Moderation" is the process of evaluating user-generated content and filtering and adjusting it to ensure compliance with community guidelines and other regulations.

[0303] "Formatting" is the process of organizing processing results and data into a specific format to make them easier to read.

[0304] "Community guidelines" refer to rules and standards that govern appropriate behavior and content posted on online platforms.

[0305] "Real-time" means that data processing and moderation occurs immediately, without delay.

[0306] A "POST request" is a type of HTTP request for sending data from a client device to a server.

[0307] An "API endpoint" is a defined server URL that other programs or services can access, providing access to specific functionality or data.

[0308] This invention relates to a system for real-time moderation of data generated by users on online platforms to maintain the integrity of the platform. The system collects data posted by users from their devices, analyzes, evaluates, and filters the data using a sentiment analysis engine and generative AI model, and provides feedback on the evaluation results to users to encourage them to maintain healthy content.

[0309] System configuration

[0310] 1. Server:

[0311] The server uses the Flask framework to build a web server. Flask is a lightweight web framework that allows for fast deployment and simple API design.

[0312] The server receives user-generated data from the client device as a POST request, with the data being transmitted in JSON format.

[0313] The server sends the received data to a sentiment analysis engine, which can be implemented using a commercial API such as IBM Watson Tone Analyzer.

[0314] The server receives the results of the sentiment analysis and requests moderation from a generative AI model based on the results. For example, OpenAI's GPT-4 is used as the generative AI model.

[0315] The server formats the evaluation results from the generative AI model and sends them back to the client device.

[0316] 2. Terminal:

[0317] The terminal converts the data entered by the user into JSON format and sends a POST request to the server.

[0318] The terminal receives the moderation results returned from the server and displays them to the user.

[0319] 3. User:

[0320] A user accesses the online platform using a terminal, inputs the content to be posted, and sends it.

[0321] The user reviews the moderation results and corrects the post if necessary.

[0322] Specific examples

[0323] For example, if a user posts something like this:

[0324] "This game is so fun! But that player said some horrible things."

[0325] 1. The user enters the content of this post into the device and presses the send button.

[0326] 2. The device converts the post content into JSON format and sends it to the server.

[0327] 3. The server receives the post and requests a sentiment analysis engine to analyze it, which analyzes both positive and negative sentiment.

[0328] 4. The server requests moderation from the generative AI model based on the results of the sentiment analysis engine. The generative AI model evaluates the content of the post while taking into account the results of the sentiment analysis.

[0329] 5. The server formats the evaluation results from the generated AI model and sends them back to the device.

[0330] 6. The device receives the moderation results from the server and displays them to the user, for example, feedback such as "Please avoid making negative comments about other players."

[0331] 7. Users review the feedback and modify their posts as needed. For example, they might post only "This game is really fun!"

[0332] Prompt Sentence Examples

[0333] A concrete example of a prompt for a generative AI model might be something like this:

[0334] User post: "This game is so fun! But that player said some horrible things."

[0335] Sentiment analysis results: "Positive sentiment: 0.7, Negative sentiment: 0.3"

[0336] This allows the generative AI model to perform appropriate moderation based on user comments.

[0337] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0338] Step 1:

[0339] The server receives a POST request from a client device.

[0340] Input: User-generated data (in text format) sent from the client device.

[0341] What it does: The server uses the Flask framework to set up an endpoint at / moderate to receive POST requests.

[0342] Output: The received user-generated data is persisted in the server.

[0343] Step 2:

[0344] The server sends the received data to the sentiment analysis engine.

[0345] Input: User-generated data received in step 1.

[0346] How it works: A server sends user-generated data to an API endpoint of a sentiment analysis engine (e.g., IBM Watson Tone Analyzer).

[0347] Output: The sentiment analysis result from the sentiment analysis engine (e.g., positive sentiment: 0.7, negative sentiment: 0.3) is returned.

[0348] Step 3:

[0349] The server requests moderation from the generative AI model based on the results of emotion analysis.

[0350] Input: Sentiment analysis results obtained in step 2 and the original user-generated data.

[0351] How it works: The server creates an input prompt for a generative AI model (e.g., OpenAI GPT-4) and sends it to the model.

[0352] Output: Moderation results from the generative AI model (e.g., "Please avoid negative comments to other players") are returned.

[0353] Step 4:

[0354] The server formats and transmits the moderation results from the generative AI model to the client device.

[0355] Input: Moderation results obtained in step 3.

[0356] How it works: The server formats the moderation results into a specific format, such as JSON.

[0357] Output: Formatted moderation results are sent to the client device.

[0358] Step 5:

[0359] The terminal receives the moderation results returned from the server and displays them to the user.

[0360] Input: Moderation results sent by the server in step 4.

[0361] Operation: The terminal receives the response from the server and displays the content to the user, for example, as a pop-up message or a dialog box.

[0362] Output: A display screen where users can see the moderation results.

[0363] Step 6:

[0364] The user reviews the moderation results and corrects the post if necessary.

[0365] Input: Moderation results displayed in Step 5.

[0366] Action: The user reviews the feedback message and follows the instructions to edit or modify the post. For example, "Delete negative comments about other players."

[0367] Output: A new, revised post is generated, ready to be sent again.

[0368] (Application example 2)

[0369] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0370] In conventional content distribution services, if content posted by users contains inappropriate content, manual moderation is required, which is inefficient and poses a risk to the integrity of the platform. In particular, real-time moderation is difficult, which poses the risk of inappropriate content spreading instantly.

[0371] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving data generated by a user from a client device, means for processing the generated data using a generative artificial intelligence model, means for performing sentiment analysis on the user-generated data, and means for transmitting the processing results to the client device. This enables sound moderation in real time in content distribution services, preventing the spread of inappropriate content and improving the user experience.

[0372] A "client device" is a device that transmits user-generated data to a server, and examples include smartphones and personal computers.

[0373] "Generated Data" means user-generated content, whether in the form of text, images, video, or other content.

[0374] A "generative artificial intelligence model" is an artificial intelligence algorithm used to process generated data and evaluate or filter its content.

[0375] "Means for processing" refers to the function of analyzing and evaluating the generated data using a generative artificial intelligence model.

[0376] "Sentiment analysis" is the process of recognizing the emotional state of a user from their generated data and determining whether it is positive or negative.

[0377] "Processing results" refers to information generated as a result of evaluation and filtering performed on generated data.

[0378] The "means for transmitting" refers to a function for returning the processing results to the client device.

[0379] "Community guidelines" refer to the rules and standards that users must follow on an online platform.

[0380] The present invention is a generative system for moderating user-generated data from client devices in real time to maintain the health of online platforms. The system combines a sentiment analysis engine and a generative artificial intelligence model to achieve advanced moderation based on the user's emotional state.

[0381] System configuration

[0382] server

[0383] The server first sets up an endpoint to accept requests from client devices. This uses the Flask framework. The server receives data posted by users and sends it to an emotion engine for sentiment analysis. This makes it possible to recognize the user's emotional state. Next, the server requests a content evaluation from a generative AI model based on the results of the sentiment analysis. The generative AI model then performs moderation based on the results of the sentiment analysis.

[0384] Terminal

[0385] The device converts user-generated data into JSON format and sends a POST request to the server. It receives the response from the server and displays it to the user, providing feedback on the moderation results. This allows users to check whether the content violates the platform's guidelines and make corrections if necessary.

[0386] User

[0387] Users access the online platform using their devices, input and submit content to be posted, and then check the moderation results from the server on their devices and modify the content according to the instructions.

[0388] Hardware and software used

[0389] Hardware:

[0390] Client devices such as smartphones and PCs

[0391] software:

[0392] On the server side, the Flask framework

[0393] Sentiment Engine API for sentiment analysis

[0394] Content rating system using live artificial intelligence models

[0395] React Native on the client side

[0396] Specific examples

[0397] For example, consider the case where a user attempts to post the following content:

[0398] "This video is great, but the comments section is awful."

[0399] 1. The user enters this information into the terminal and presses the send button.

[0400] 2. The device sends the post to the server.

[0401] 3. The server receives the post and runs it through a sentiment analysis engine.

[0402] 4. The sentiment analysis engine determines that the comment contains negative comments.

[0403] 5. The server requests moderation from the live AI model based on the results from the sentiment analysis engine.

[0404] 6. A raw AI model detects the phrase "The comments section is awful" and rates it as inappropriate.

[0405] 7. The server returns the evaluation results to the device.

[0406] 8. The device displays feedback to the user, prompting them to revise their statement, "The comments section is terrible."

[0407] Prompt Sentence Examples

[0408] "Please send the following text to the Sentiment Engine API for sentiment analysis:

[0409] A user wrote: "This video is great, but the comments section is awful."

[0410] Emotion Engine API endpoint: https: / / emotion-api.example.com / analyze

[0411] "

[0412] To generate smart moderation feedback, use prompts like these:

[0413] The post read: "The comments section is awful."

[0414] Generative AI model API endpoint: https: / / ai-moderation-api.example.com / moderate

[0415]

[0416] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0417] Step 1:

[0418] The user types content (e.g., "This video is great, but the comments section is terrible") into the device and hits send.

[0419] Input: User generated content text

[0420] Output: POST request sent from the terminal to the server

[0421] Specific behavior: The user enters content into the text input area on the device and clicks the send button, which sends a request in JSON format to the server.

[0422] Step 2:

[0423] The device converts the user-generated data and sends a POST request to the server.

[0424] Input: Content text entered by the user

[0425] Output: POST request sent to the server

[0426] Specific operation: The device detects that the send button has been pressed, converts the entered text into JSON format, and sends an HTTP POST request to the server endpoint.

[0427] Step 3:

[0428] The server receives the posted content and sends the text data to the sentiment analysis engine.

[0429] Input: Content text in JSON format

[0430] Output: Request to the sentiment analysis engine

[0431] Specific operation: The server receives the data sent from the client and sends a sentiment analysis request to the Emotion Engine API, including the content text in the body of the POST request and sending it to the API endpoint.

[0432] Step 4:

[0433] The sentiment analysis engine analyzes the text data and returns the sentiment analysis results.

[0434] Input: Content text

[0435] Output: Sentiment analysis result (e.g. positive, negative, neutral)

[0436] What it does: The sentiment analysis engine analyzes the received text data, evaluates the sentiment of each word or phrase, classifies the emotional state, and sends the results back to the server in JSON format.

[0437] Step 5:

[0438] The server receives the sentiment analysis results and requests the generative artificial intelligence model to evaluate the content.

[0439] Input: Sentiment analysis results

[0440] Output: A request to the generative AI model

[0441] What it does: The server receives the analysis results from the sentiment engine and sends a moderation request to the generative AI model API, which includes the sentiment analysis results and the original text.

[0442] Step 6:

[0443] The generative AI model performs moderation based on content and sentiment analysis results and generates evaluation results.

[0444] Input: Content text and sentiment analysis results

[0445] Output: Moderation evaluation result (e.g., classifying "The comments section is awful" as inappropriate)

[0446] Specific operation: The generative AI model analyzes the content text and sentiment analysis results, performs moderation based on the policy, filters and evaluates the content, and generates results, which are then sent back to the server.

[0447] Step 7:

[0448] The server receives the moderation evaluation results from the generative AI model, formats them, and sends them back to the device.

[0449] Input: Moderation evaluation results

[0450] Output: Response to the terminal

[0451] Specific operation: The server receives the evaluation results from the generative AI model, formats them to provide appropriate feedback to the user, and then sends the formatted results back to the device in JSON format.

[0452] Step 8:

[0453] The device will display the moderation results to the user and prompt them to correct the content if necessary.

[0454] Input: Moderation evaluation results

[0455] Output: Display feedback to the user

[0456] Specific operation: The device analyzes the JSON data returned from the server and displays the evaluation results in the user interface. The device prompts the user to make the necessary corrections by showing the specific areas and content of corrections.

[0457] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0458] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0459] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0460] [Second embodiment]

[0461] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0462] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0463] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0464] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0465] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0466] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0467] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0468] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0469] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0470] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0471] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0472] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0473] The present invention relates to a generation system for automating the moderation of user-generated data and maintaining the integrity of content on online platforms, comprising a set of means for receiving data posted from a client device, processing the data using a generative artificial intelligence model, and transmitting the processing results to the client device.

[0474] System configuration

[0475] server:

[0476] The server uses the Flask framework to build a web server and sets up an endpoint that accepts requests from client devices.

[0477] The server receives user-submitted data and evaluates and filters the content using a generative artificial intelligence model.

[0478] The server formats the evaluation and filtering results and sends them back to the client device as a response.

[0479] Device:

[0480] The terminal sends a request containing user-generated data to the server.

[0481] The terminal displays the moderation results received from the server to the user.

[0482] User:

[0483] Users use their terminals to access the online platform, input content to post, and send it.

[0484] Users receive the results of the moderation and can modify their posts as necessary.

[0485] Program processing flow

[0486] server:

[0487] 1. The server receives user-generated data sent as a POST request from a client device.

[0488] 2. The server takes the received data and sends it to the generative artificial intelligence model.

[0489] 3. The generative AI model determines whether the data complies with community guidelines and generates an evaluation result.

[0490] 4. The server formats the evaluation results from the generative artificial intelligence model and sends them back to the client device along with the original data.

[0491] Device:

[0492] 1. The terminal converts the data entered by the user into JSON format and sends a POST request to the server.

[0493] 2. The device receives the response sent back from the server and displays it to the user.

[0494] Specific examples

[0495] For example, if a user posts content like this:

[0496] "This game is so fun! But that player said some horrible things."

[0497] 1. The user enters this information into the terminal and presses the send button.

[0498] 2. The device sends the posted content to the server.

[0499] 3. The server receives the posted content and requests the generative AI model to evaluate the content.

[0500] 4. The generative AI model analyzes the content of the post and generates an evaluation result, such as "avoid making negative comments about other players."

[0501] 5. The server receives the evaluation results, formats them together with the original post content, and sends them back to the device.

[0502] 6. The device displays the moderation results to the user and prompts them to correct their post.

[0503] This allows users to recognize and correct when their posts do not comply with community guidelines, and provides results that help maintain a healthy digital environment across online platforms.

[0504] The processing flow will be explained below.

[0505] Step 1:

[0506] A user enters content and presses the submit button. For example, they might enter, "This game is really fun! But that player said some horrible things."

[0507] Step 2:

[0508] The device converts the input content into JSON format, for example:

[0509] json

[0510] {

[0511] "content": "This game is really fun! But that player said some horrible things."

[0512] }

[0513] Step 3:

[0514] The device sends the converted JSON data to the server as an HTTP POST request. The request destination is the endpoint ( / filter) configured in the Flask framework.

[0515] Step 4:

[0516] The server processes the POST request received from the client device. At that time, it extracts the JSON data from the request body and extracts the user-submitted content. In this case, the extracted content is "This game is really fun! But that player said some horrible things."

[0517] Step 5:

[0518] The server passes the extracted content to the moderate_content function. This function uses a generative AI model to evaluate the content. Specifically, it calls the OpenAI API to perform content moderation. For example, it generates the following prompt:

[0519] Moderate the following content according to the community guidelines: "This game is really fun! But that player said some horrible things."

[0520] Step 6:

[0521] A generative AI model evaluates the content based on the prompts received and generates moderation results, such as "Avoid negative comments about other players."

[0522] Step 7:

[0523] The server receives the moderation results returned by the generative AI model and formats them along with the original content, creating a JSON response like this:

[0524] json

[0525] {

[0526] "original_content": "This game is really fun! But that player said some horrible things.",

[0527] "moderation_result": "Please avoid negative comments about other players."

[0528] }

[0529] Step 8:

[0530] The server sends a formatted JSON response to the client device.

[0531] Step 9:

[0532] The device receives the response from the server and displays the moderation results to the user, for example, feedback such as "Please avoid making negative comments about other players" is displayed on the screen.

[0533] Step 10:

[0534] The user then edits and corrects the content based on the feedback received and resubmits it if necessary.

[0535] Example 1

[0536] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0537] While the amount of user-generated content on modern online platforms is increasing, there is a need to ensure the quality and integrity of that content. However, manual moderation requires a great deal of effort and time, and it is difficult to respond in real time. This creates a risk that inappropriate content will be published, potentially damaging the integrity of the platform. To solve this problem, an automated moderation system is needed.

[0538] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0539] In this invention, the server includes means for receiving user-generated data from a client device, means for processing the generated data using a generative AI model, means for transmitting the processing results to the client device, communication means for accepting requests from the client device, data formatting means for transmitting data to the generative AI model, and result formatting means for formatting and returning evaluation results from the generative AI model, thereby enabling automatic evaluation and filtering of user-generated content in real time and immediate provision of moderation results.

[0540] A "client device" is an electronic device through which a user inputs information and transmits data to a server.

[0541] "Server" means a computer system that processes data received from a client device and performs evaluation and filtering using a generative AI model.

[0542] "User-Generated Data" is information generated and transmitted by a user through a client device.

[0543] A "generative AI model" is an artificial intelligence algorithm used to analyze user-generated data and generate evaluation results.

[0544] The "data formatting means for sending to the generative AI model" is a function within the server that converts the received data into a format that is easy for the generative AI model to process.

[0545] The "result formatting means for formatting and returning the evaluation results" is a function within the server that converts the evaluation results obtained from the generative AI model into a format that is easy for the client device to understand and transmits them.

[0546] "Communication means" refers to the interface and protocol for transmitting and receiving data between the client device and the server.

[0547] The system of the present invention aims to automate the moderation of user-generated data and maintain the integrity of content on online platforms. The system comprises a set of means including a client device, a server, and a generative AI model.

[0548] Server Configuration

[0549] The server has the functionality to receive user-generated data from the client device, send that data to the generative AI model, obtain the evaluation results, format them and send them back to the client device. The server is built using the following technologies:

[0550] Web framework: Flask

[0551] Generative AI model: OpenAI's GPT-4 API (specific name generalization)

[0552] The server uses the Flask framework to receive POST requests sent from the client device, then formats the received data to send to the generative AI model, which analyzes the data and generates an evaluation result. The server formats the evaluation result and sends a response back to the client device.

[0553] Device configuration

[0554] The device has the function of converting data generated by the user into JSON format and sending it to the server, and also receives the response sent back from the server and displays the result to the user.

[0555] Data transmission method: HTTP request library such as fetch API

[0556] User Interface: HTML, CSS, JavaScript

[0557] When a user enters data such as a comment or post on their device and presses the send button, the content is sent to the server, and when a response is returned from the server, the result is displayed on the screen.

[0558] User Actions

[0559] Users access the online platform using their devices, enter and submit their posts, receive moderation results, and amend their posts as necessary.

[0560] Input method: keyboard, touchscreen, etc.

[0561] Specific examples

[0562] For example, consider the case where a user posts content such as, "This game is really fun! But that player said some horrible things."

[0563] 1. User: Enter this information into the terminal and press the send button.

[0564] 2. Device: Send this post to the server.

[0565] 3. Server: Receives the posted content and requests the generative AI model to evaluate the content.

[0566] 4. Generative AI model: Analyzes the content of the post and generates an evaluation result such as "avoid making negative comments about other players."

[0567] 5. Server: Receives the evaluation results, formats them together with the original post, and sends them back to the device.

[0568] 6. On the device: Display the moderation results to the user and prompt them to correct their post.

[0569] Prompt Sentence Examples

[0570] The generative AI model evaluates posts by providing prompts like the following:

[0571] "This game is so fun! But that player said some horrible things."

[0572] Rate Us: Rate this content to see if it adheres to our Community Guidelines and let us know what changes need to be made.

[0573] Based on this prompt, the generative AI model generates appropriate feedback and provides the results to the user via the server.

[0574] This allows users to ensure that their posts comply with community guidelines, maintaining a healthy digital environment across online platforms.

[0575] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0576] Step 1:

[0577] The server receives user-generated data sent as a POST request from the client device. The input is user-generated data in JSON format, for example, {"content": "This game is really fun!"}. The output is to store the received data object for internal processing. Specifically, it uses the Flask framework to set up an endpoint / api / moderate to receive the request and extract the data from the request body.

[0578] Step 2:

[0579] The server formats the received data and prepares it to be sent to the generative AI model. The input is the data object received in the previous step, e.g., {"content": "This game is really fun!"}. The output is formatted data to be sent to the generative AI model, e.g., {"texts": ["This game is really fun!"]}. Specifically, it converts the data into formatted JSON format and prepares an API request to send to the generative AI model (GPT-4 API).

[0580] Step 3:

[0581] The generative AI model receives and analyzes formatted data. The input is formatted JSON data, and the output is a JSON response containing the evaluation results. For example, if the input is {"texts": ["This game is really fun!"]}, the output will be an evaluation result such as {"evaluation": "appropriate"}. In concrete terms, the generative AI model analyzes the data and evaluates each text based on community guidelines.

[0582] Step 4:

[0583] The server receives the evaluation results returned from the generative AI model, formats them together with the original data, and sends them back to the client device. The input is the evaluation result JSON from the generative AI model, for example, {"evaluation": "appropriate"}. The output is a formatted JSON response to be sent back to the client device, for example, {"original_content": "This game is really fun!", "evaluation": "appropriate"}. Specifically, the server formats the evaluation results, integrates them with the original data, and sends them back to the client device.

[0584] Step 5:

[0585] The device receives the response returned from the server and displays it to the user. The input is the JSON response returned from the server, for example, {"original_content": "This game is really fun!", "evaluation": "Appropriate"}. The output is the evaluation result message displayed to the user, for example, "Evaluation result: Appropriate". Specifically, it uses JavaScript to parse the response and reflects the result in the HTML DOM.

[0586] Step 6:

[0587] The user checks the moderation results and modifies the post as necessary. The input is the evaluation result displayed on the device and the original post content, for example, "Evaluation result: appropriate" and "This game is really fun!". The output is the modified post content, for example, "This game is really fun!". Specifically, the user re-enters the post based on the presented evaluation results, reviews the content, and resubmits it from the device.

[0588] (Application example 1)

[0589] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0590] In recent years, online chat and messaging services have rapidly become popular, making maintaining the integrity of user-generated content a key challenge. Messages containing inappropriate language, suicidal thoughts, and violent language are particularly problematic. However, moderating this content in real time is extremely difficult, necessitating the development of automated and efficient solutions.

[0591] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0592] In this invention, the server includes means for receiving user-generated data from a client device, means for processing the generated data using a generative artificial intelligence model, means for transmitting the processing results to the client device, means for moderating user-generated messages in real time, and means for detecting inappropriate content and providing a warning or appropriate response information, thereby making it possible to maintain the integrity of content in chat and messaging services in real time and prevent the spread of inappropriate content.

[0593] A "generation system" is a system that processes and filters user-generated data and transmits the results to a client device.

[0594] A "client device" is a device (e.g., a smartphone or personal computer) that a user uses to input data and communicate with the production system.

[0595] "User-Generated Data" means text messages and other data that a user generates and sends through a client device.

[0596] A "generative artificial intelligence model" is an artificial intelligence-based software model for analyzing received data and performing appropriate processing and filtering.

[0597] "Moderation" is the process of determining whether the generated data complies with community guidelines, etc., and correcting the content or issuing a warning as necessary.

[0598] "Real-time" means that user data is processed and filtered almost instantly after it is generated.

[0599] "Filtering" is the process of detecting and removing or flagging inappropriate content from generated data.

[0600] A "warning" is a message that notifies the user when the data generated by the user is inappropriate and urges the user to correct it.

[0601] "Response information" refers to information about appropriate assistance and support provided to users when inappropriate content is detected.

[0602] "Integrity" refers to the content on online platforms being socially appropriate and being safe for users to use.

[0603] The present invention provides a system for real-time moderation of data generated by users in online chat and messaging services to prevent the spread of inappropriate content. The system comprises a client device, a server, and a generative artificial intelligence model.

[0604] System configuration

[0605] server:

[0606] The server uses the Flask framework to build a web server and set up endpoints to accept requests from client devices. The server receives user-generated data, processes and filters it using a generative artificial intelligence model, and formats the results and sends them back to the client device as a response.

[0607] Device:

[0608] The device converts the data generated by the user into JSON format and sends it to the server as a POST request. The device receives the moderation results returned by the server and displays them to the user.

[0609] User:

[0610] Users access online chat and messaging services using their devices, input and send messages, and receive moderation results, allowing them to modify the content of their messages as necessary.

[0611] Program processing flow

[0612] server:

[0613] The server uses the Flask framework to build a web server and set up an endpoint. It receives messages sent by users as POST requests from client devices. The server sends the data to a generative artificial intelligence model (e.g., OpenAI's GPT-3) to evaluate whether the message is appropriate. It then sends a response containing the evaluation results back to the client device.

[0614] Device:

[0615] The device converts the data generated by the user into JSON format and sends it to the server as a POST request, receives the response sent back from the server, and displays it to the user.

[0616] Specific examples

[0617] For example, if a user types the following message into chat:

[0618] "There's no point in living anymore."

[0619] 1. The user enters this information into the terminal and presses the send button.

[0620] 2. The terminal sends this message to the server.

[0621] 3. The server receives the message and requests an evaluation from the generative artificial intelligence model.

[0622] 4. The generative AI model analyzes the message and generates an evaluation result, such as, "This message may contain suicidal thoughts. Please consult a specialist."

[0623] 5. The server receives the evaluation result, formats it together with the original message, and sends it back to the terminal.

[0624] 6. The device displays the moderation results to the user and provides appropriate support information as needed.

[0625] Prompt Sentence Examples

[0626] "Does this message possibly contain suicidal thoughts? 'There's no point in living anymore.'\n\nPlease suggest an appropriate response to this message."

[0627] The present invention makes it possible to maintain the integrity of content in chat and messaging services in real time and prevent the spread of inappropriate content.

[0628] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0629] Step 1:

[0630] A user enters a message on the chat or messaging service and presses the send button, which causes the entered message to be received by the client device.

[0631] Input: The message entered by the user

[0632] Output: Message sent to the client device

[0633] Step 2:

[0634] The device converts the message entered by the user into JSON format and sends a POST request to the server, encoding the text as message data in JSON format.

[0635] Input: The message entered by the user

[0636] Output: POST request containing message data in JSON format

[0637] Step 3:

[0638] The server receives the POST request sent from the client device, analyzes the received data, and extracts the message content.

[0639] Input: A POST request containing message data in JSON format.

[0640] Output: Extracted text message

[0641] Step 4:

[0642] The server then sends the extracted messages to a generative artificial intelligence model for appropriate moderation evaluation, specifically using OpenAI's GPT-3 model to analyze the content of the messages and determine their appropriateness.

[0643] Input: Extracted text message

[0644] Output: Moderation evaluation results and warning messages

[0645] Step 5:

[0646] The generative AI model analyzes the content of received messages and evaluates their appropriateness based on the prompt. For example, it generates a response to a prompt such as, "'There's no point in living anymore.' Could this message contain suicidal thoughts?"

[0647] Input: Extracted text message and prompt

[0648] Output: Suitability assessment and warning messages

[0649] Step 6:

[0650] The server receives the evaluation results from the generative artificial intelligence model, formats them, and sends them back to the client device along with the original message.

[0651] Input: Moderation evaluation results and warning messages

[0652] Output: A response containing the formatted evaluation result and the original message.

[0653] Step 7:

[0654] The terminal receives the response from the server and displays the moderation results to the user, who can then modify the message content as necessary.

[0655] Input: Formatted evaluation results and original message

[0656] Output: Moderation results and prompts shown to the user

[0657] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0658] The present invention relates to a generation system for moderating user-generated data in real time to maintain the integrity of content on online platforms. The system includes a set of means for receiving data posted from a client device, processing the data using a generative artificial intelligence model, and transmitting the processing results to the client device. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system achieves more advanced moderation based on the user's emotional state.

[0659] System configuration

[0660] server:

[0661] The server uses the Flask framework to build a web server and sets up an endpoint that accepts requests from client devices.

[0662] The server receives user-submitted data and evaluates and filters the content using an emotion engine and generative artificial intelligence models.

[0663] The server formats the evaluation and filtering results and sends them back to the client device as a response.

[0664] Device:

[0665] The terminal sends a request containing user-generated data to the server.

[0666] The terminal displays the moderation results received from the server to the user.

[0667] User:

[0668] Users use their terminals to access the online platform, input content to post, and send it.

[0669] Users receive the results of the moderation and can modify their posts as necessary.

[0670] Details of system processing

[0671] Processing flow

[0672] server:

[0673] 1. The server receives user-generated data sent as a POST request from a client device.

[0674] 2. The server retrieves the received data and sends it to the emotion engine, which analyzes the emotions from the user-generated data and provides the results to the generation system.

[0675] 3. The server requests the generative artificial intelligence model to evaluate the content based on the emotion analysis results obtained from the emotion engine.

[0676] 4. The biointelligence model uses this data and sentiment analysis results to moderate the content, for example, easing the rating if it contains positive sentiment and stricter rating if it contains negative sentiment.

[0677] 5. The server formats the evaluation results from the biointelligence model and sends them back to the client device along with the original data.

[0678] Device:

[0679] 1. The terminal converts the data entered by the user into JSON format and sends a POST request to the server.

[0680] 2. The device receives the response sent back from the server and displays it to the user.

[0681] Specific examples

[0682] For example, if a user posts content like this:

[0683] "This game is so fun! But that player said some horrible things."

[0684] 1. The user enters this information into the terminal and presses the send button.

[0685] 2. The device sends the post to the server.

[0686] 3. The server receives the post and requests the emotion engine to analyze the sentiment of the content.

[0687] 4. The sentiment engine recognizes that the post contains negative sentiment (e.g., negative comments).

[0688] 5. The server requests strict moderation of the biointelligence model based on the results from the emotion engine.

[0689] 6. The biointelligence model analyzes the content of the posts and generates evaluation results such as "avoid making negative comments about other players."

[0690] 7. The server receives the evaluation results, formats them together with the original post content, and sends them back to the device.

[0691] 8. The device displays the moderation results to the user and prompts them to correct their post.

[0692] This allows users to recognize when content violates guidelines and correct it, and the emotional engine enables appropriate moderation based on the user's emotional state, helping to provide a healthy digital environment across online platforms.

[0693] The processing flow will be explained below.

[0694] Step 1:

[0695] The user enters content and presses the post button. For example, they might enter, "This game is really fun! But that player said some horrible things."

[0696] Step 2:

[0697] The terminal converts the input content into JSON format, which looks like this:

[0698] json

[0699] {

[0700] "content": "This game is really fun! But that player said some horrible things."

[0701] }

[0702] Step 3:

[0703] The device sends the converted JSON data to the server as an HTTP POST request. The request destination is the endpoint ( / filter) configured in the Flask framework.

[0704] Step 4:

[0705] The server receives a POST request from the client device. In this case, the received data is "This game is really fun! But that player said some horrible things."

[0706] Step 5:

[0707] The server sends the received data to the emotion engine, which analyzes the emotions from the user-generated data and determines, for example, "positive" or "negative" emotions.

[0708] Step 6:

[0709] The emotion engine analyzes the input content and recognizes that it contains negative emotions. For example, the phrase "he said horrible things" is identified as a negative emotion.

[0710] Step 7:

[0711] The server requests strict moderation evaluation from the generative artificial intelligence model based on the analysis results of negative emotions obtained from the emotion engine.

[0712] Step 8:

[0713] A generative AI model will critically evaluate your submission, generating prompts such as:

[0714] Moderate the following content according to the community guidelines, considering that it includes negative sentiment: "This game is really fun! But that player said some horrible things."

[0715] Step 9:

[0716] A generative AI model generates content evaluation results, such as a moderation result like "Please avoid making negative comments about other players."

[0717] Step 10:

[0718] The server receives the moderation results returned by the generative AI model and formats them along with the original content, creating a JSON response like this:

[0719] json

[0720] {

[0721] "original_content": "This game is really fun! But that player said some horrible things.",

[0722] "moderation_result": "Please avoid negative comments about other players."

[0723] }

[0724] Step 11:

[0725] The server sends a formatted JSON response to the client device.

[0726] Step 12:

[0727] The device receives the response from the server and displays the moderation results to the user, for example, feedback such as "Please avoid making negative comments about other players" is displayed on the screen.

[0728] Step 13:

[0729] The user then edits and corrects the content based on the feedback received and resubmits it if necessary.

[0730] Example 2

[0731] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0732] Modern online platforms require the maintenance of the integrity of content generated by users, as a large amount of content is posted in real time. However, manual moderation is time-consuming and costly, and it is difficult to cover all content. To solve this problem, an automated real-time moderation system is needed. Furthermore, advanced moderation that takes user sentiment into account is required, rather than simply moderating content.

[0733] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0734] In this invention, the server includes means for receiving user-generated data from a client device, means for transmitting the generated data to a sentiment analysis engine to perform sentiment analysis, means for requesting data moderation using a generative AI model based on the sentiment analysis result, and means for formatting and transmitting the moderation result to the client device, thereby automatically and efficiently maintaining content integrity in real time and enabling more advanced moderation based on emotional states.

[0735] A "client device" is a terminal device used by a user, and is a device for sending and receiving data to a server via the Internet.

[0736] "User-generated data" refers to content, such as text, images, and video, entered by a user and transmitted through a client device.

[0737] A "sentiment analysis engine" is a software or hardware component for extracting and analyzing emotional information from user-generated data.

[0738] A "generative AI model" is a program or system that uses artificial intelligence techniques to generate, process, and analyze data.

[0739] "Moderation" is the process of evaluating user-generated content and filtering and adjusting it to ensure compliance with community guidelines and other regulations.

[0740] "Formatting" is the process of organizing processing results and data into a specific format to make them easier to read.

[0741] "Community guidelines" refer to rules and standards that govern appropriate behavior and content posted on online platforms.

[0742] "Real-time" means that data processing and moderation occurs immediately, without delay.

[0743] A "POST request" is a type of HTTP request for sending data from a client device to a server.

[0744] An "API endpoint" is a defined server URL that other programs or services can access, providing access to specific functionality or data.

[0745] This invention relates to a system for real-time moderation of data generated by users on online platforms to maintain the integrity of the platform. The system collects data posted by users from their devices, analyzes, evaluates, and filters the data using a sentiment analysis engine and generative AI model, and provides feedback on the evaluation results to users to encourage them to maintain healthy content.

[0746] System configuration

[0747] 1. Server:

[0748] The server uses the Flask framework to build a web server. Flask is a lightweight web framework that allows for fast deployment and simple API design.

[0749] The server receives user-generated data from the client device as a POST request, with the data being transmitted in JSON format.

[0750] The server sends the received data to a sentiment analysis engine, which can be implemented using a commercial API such as IBM Watson Tone Analyzer.

[0751] The server receives the results of the sentiment analysis and requests moderation from a generative AI model based on the results. For example, OpenAI's GPT-4 is used as the generative AI model.

[0752] The server formats the evaluation results from the generative AI model and sends them back to the client device.

[0753] 2. Terminal:

[0754] The terminal converts the data entered by the user into JSON format and sends a POST request to the server.

[0755] The terminal receives the moderation results returned from the server and displays them to the user.

[0756] 3. User:

[0757] A user accesses the online platform using a terminal, inputs the content to be posted, and sends it.

[0758] The user reviews the moderation results and corrects the post if necessary.

[0759] Specific examples

[0760] For example, if a user posts something like this:

[0761] "This game is so fun! But that player said some horrible things."

[0762] 1. The user enters the content of this post into the device and presses the send button.

[0763] 2. The device converts the post content into JSON format and sends it to the server.

[0764] 3. The server receives the post and requests a sentiment analysis engine to analyze it, which analyzes both positive and negative sentiment.

[0765] 4. The server requests moderation from the generative AI model based on the results of the sentiment analysis engine. The generative AI model evaluates the content of the post while taking into account the results of the sentiment analysis.

[0766] 5. The server formats the evaluation results from the generated AI model and sends them back to the device.

[0767] 6. The device receives the moderation results from the server and displays them to the user, for example, feedback such as "Please avoid making negative comments about other players."

[0768] 7. Users review the feedback and modify their posts as needed. For example, they might post only "This game is really fun!"

[0769] Prompt Sentence Examples

[0770] A concrete example of a prompt for a generative AI model might be something like this:

[0771] User post: "This game is so fun! But that player said some horrible things."

[0772] Sentiment analysis results: "Positive sentiment: 0.7, Negative sentiment: 0.3"

[0773] This allows the generative AI model to perform appropriate moderation based on user comments.

[0774] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0775] Step 1:

[0776] The server receives a POST request from a client device.

[0777] Input: User-generated data (in text format) sent from the client device.

[0778] What it does: The server uses the Flask framework to set up an endpoint at / moderate to receive POST requests.

[0779] Output: The received user-generated data is persisted in the server.

[0780] Step 2:

[0781] The server sends the received data to the sentiment analysis engine.

[0782] Input: User-generated data received in step 1.

[0783] How it works: A server sends user-generated data to an API endpoint of a sentiment analysis engine (e.g., IBM Watson Tone Analyzer).

[0784] Output: The sentiment analysis result from the sentiment analysis engine (e.g., positive sentiment: 0.7, negative sentiment: 0.3) is returned.

[0785] Step 3:

[0786] The server requests moderation from the generative AI model based on the results of emotion analysis.

[0787] Input: Sentiment analysis results obtained in step 2 and the original user-generated data.

[0788] How it works: The server creates an input prompt for a generative AI model (e.g., OpenAI GPT-4) and sends it to the model.

[0789] Output: Moderation results from the generative AI model (e.g., "Please avoid negative comments to other players") are returned.

[0790] Step 4:

[0791] The server formats and transmits the moderation results from the generative AI model to the client device.

[0792] Input: Moderation results obtained in step 3.

[0793] How it works: The server formats the moderation results into a specific format, such as JSON.

[0794] Output: Formatted moderation results are sent to the client device.

[0795] Step 5:

[0796] The terminal receives the moderation results returned from the server and displays them to the user.

[0797] Input: Moderation results sent by the server in step 4.

[0798] Operation: The terminal receives the response from the server and displays the content to the user, for example, as a pop-up message or a dialog box.

[0799] Output: A display screen where users can see the moderation results.

[0800] Step 6:

[0801] The user reviews the moderation results and corrects the post if necessary.

[0802] Input: Moderation results displayed in Step 5.

[0803] Action: The user reviews the feedback message and follows the instructions to edit or modify the post. For example, "Delete negative comments about other players."

[0804] Output: A new, revised post is generated, ready to be sent again.

[0805] (Application example 2)

[0806] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0807] In conventional content distribution services, if content posted by users contains inappropriate content, manual moderation is required, which is inefficient and poses a risk to the integrity of the platform. In particular, real-time moderation is difficult, which poses the risk of inappropriate content spreading instantly.

[0808] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving data generated by a user from a client device, means for processing the generated data using a generative artificial intelligence model, means for performing sentiment analysis on the user-generated data, and means for transmitting the processing results to the client device. This enables sound moderation in real time in content distribution services, preventing the spread of inappropriate content and improving the user experience.

[0809] A "client device" is a device that transmits user-generated data to a server, and examples include smartphones and personal computers.

[0810] "Generated Data" means user-generated content, whether in the form of text, images, video, or other content.

[0811] A "generative artificial intelligence model" is an artificial intelligence algorithm used to process generated data and evaluate or filter its content.

[0812] "Means for processing" refers to the function of analyzing and evaluating the generated data using a generative artificial intelligence model.

[0813] "Sentiment analysis" is the process of recognizing the emotional state of a user from their generated data and determining whether it is positive or negative.

[0814] "Processing results" refers to information generated as a result of evaluation and filtering performed on generated data.

[0815] The "means for transmitting" refers to a function for returning the processing results to the client device.

[0816] "Community guidelines" refer to the rules and standards that users must follow on an online platform.

[0817] The present invention is a generative system for moderating user-generated data from client devices in real time to maintain the health of online platforms. The system combines a sentiment analysis engine and a generative artificial intelligence model to achieve advanced moderation based on the user's emotional state.

[0818] System configuration

[0819] server

[0820] The server first sets up an endpoint to accept requests from client devices. This uses the Flask framework. The server receives data posted by users and sends it to an emotion engine for sentiment analysis. This makes it possible to recognize the user's emotional state. Next, the server requests a content evaluation from a generative AI model based on the results of the sentiment analysis. The generative AI model then performs moderation based on the results of the sentiment analysis.

[0821] Terminal

[0822] The device converts user-generated data into JSON format and sends a POST request to the server. It receives the response from the server and displays it to the user, providing feedback on the moderation results. This allows users to check whether the content violates the platform's guidelines and make corrections if necessary.

[0823] User

[0824] Users access the online platform using their devices, input and submit content to be posted, and then check the moderation results from the server on their devices and modify the content according to the instructions.

[0825] Hardware and software used

[0826] Hardware:

[0827] Client devices such as smartphones and PCs

[0828] software:

[0829] On the server side, the Flask framework

[0830] Sentiment Engine API for sentiment analysis

[0831] Content rating system using live artificial intelligence models

[0832] React Native on the client side

[0833] Specific examples

[0834] For example, consider the case where a user attempts to post the following content:

[0835] "This video is great, but the comments section is awful."

[0836] 1. The user enters this information into the terminal and presses the send button.

[0837] 2. The device sends the post to the server.

[0838] 3. The server receives the post and runs it through a sentiment analysis engine.

[0839] 4. The sentiment analysis engine determines that the comment contains negative comments.

[0840] 5. The server requests moderation from the live AI model based on the results from the sentiment analysis engine.

[0841] 6. A raw AI model detects the phrase "The comments section is awful" and rates it as inappropriate.

[0842] 7. The server returns the evaluation results to the device.

[0843] 8. The device displays feedback to the user, prompting them to revise their statement, "The comments section is terrible."

[0844] Prompt Sentence Examples

[0845] "Please send the following text to the Sentiment Engine API for sentiment analysis:

[0846] A user wrote: "This video is great, but the comments section is awful."

[0847] Emotion Engine API endpoint: https: / / emotion-api.example.com / analyze

[0848] "

[0849] To generate smart moderation feedback, use prompts like these:

[0850] The post read: "The comments section is awful."

[0851] Generative AI model API endpoint: https: / / ai-moderation-api.example.com / moderate

[0852]

[0853] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0854] Step 1:

[0855] The user types content (e.g., "This video is great, but the comments section is terrible") into the device and hits send.

[0856] Input: User generated content text

[0857] Output: POST request sent from the terminal to the server

[0858] Specific behavior: The user enters content into the text input area on the device and clicks the send button, which sends a request in JSON format to the server.

[0859] Step 2:

[0860] The device converts the user-generated data and sends a POST request to the server.

[0861] Input: Content text entered by the user

[0862] Output: POST request sent to the server

[0863] Specific operation: The device detects that the send button has been pressed, converts the entered text into JSON format, and sends an HTTP POST request to the server endpoint.

[0864] Step 3:

[0865] The server receives the posted content and sends the text data to the sentiment analysis engine.

[0866] Input: Content text in JSON format

[0867] Output: Request to the sentiment analysis engine

[0868] Specific operation: The server receives the data sent from the client and sends a sentiment analysis request to the Emotion Engine API, including the content text in the body of the POST request and sending it to the API endpoint.

[0869] Step 4:

[0870] The sentiment analysis engine analyzes the text data and returns the sentiment analysis results.

[0871] Input: Content text

[0872] Output: Sentiment analysis result (e.g. positive, negative, neutral)

[0873] What it does: The sentiment analysis engine analyzes the received text data, evaluates the sentiment of each word or phrase, classifies the emotional state, and sends the results back to the server in JSON format.

[0874] Step 5:

[0875] The server receives the sentiment analysis results and requests the generative artificial intelligence model to evaluate the content.

[0876] Input: Sentiment analysis results

[0877] Output: A request to the generative AI model

[0878] What it does: The server receives the analysis results from the sentiment engine and sends a moderation request to the generative AI model API, which includes the sentiment analysis results and the original text.

[0879] Step 6:

[0880] The generative AI model performs moderation based on content and sentiment analysis results and generates evaluation results.

[0881] Input: Content text and sentiment analysis results

[0882] Output: Moderation evaluation result (e.g., classifying "The comments section is awful" as inappropriate)

[0883] Specific operation: The generative AI model analyzes the content text and sentiment analysis results, performs moderation based on the policy, filters and evaluates the content, and generates results, which are then sent back to the server.

[0884] Step 7:

[0885] The server receives the moderation evaluation results from the generative AI model, formats them, and sends them back to the device.

[0886] Input: Moderation evaluation results

[0887] Output: Response to the terminal

[0888] Specific operation: The server receives the evaluation results from the generative AI model, formats them to provide appropriate feedback to the user, and then sends the formatted results back to the device in JSON format.

[0889] Step 8:

[0890] The device will display the moderation results to the user and prompt them to correct the content if necessary.

[0891] Input: Moderation evaluation results

[0892] Output: Display feedback to the user

[0893] Specific operation: The device analyzes the JSON data returned from the server and displays the evaluation results in the user interface. The device prompts the user to make the necessary corrections by showing the specific areas and content of corrections.

[0894] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0895] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0896] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0897] [Third embodiment]

[0898] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0899] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0900] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0901] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0902] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0903] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0904] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0905] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0906] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0907] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0908] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0909] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0910] The present invention relates to a generation system for automating the moderation of user-generated data and maintaining the integrity of content on online platforms, comprising a set of means for receiving data posted from a client device, processing the data using a generative artificial intelligence model, and transmitting the processing results to the client device.

[0911] System configuration

[0912] server:

[0913] The server uses the Flask framework to build a web server and sets up an endpoint that accepts requests from client devices.

[0914] The server receives user-submitted data and evaluates and filters the content using a generative artificial intelligence model.

[0915] The server formats the evaluation and filtering results and sends them back to the client device as a response.

[0916] Device:

[0917] The terminal sends a request containing user-generated data to the server.

[0918] The terminal displays the moderation results received from the server to the user.

[0919] User:

[0920] Users use their terminals to access the online platform, input content to post, and send it.

[0921] Users receive the results of the moderation and can modify their posts as necessary.

[0922] Program processing flow

[0923] server:

[0924] 1. The server receives user-generated data sent as a POST request from a client device.

[0925] 2. The server takes the received data and sends it to the generative artificial intelligence model.

[0926] 3. The generative AI model determines whether the data complies with community guidelines and generates an evaluation result.

[0927] 4. The server formats the evaluation results from the generative artificial intelligence model and sends them back to the client device along with the original data.

[0928] Device:

[0929] 1. The terminal converts the data entered by the user into JSON format and sends a POST request to the server.

[0930] 2. The device receives the response sent back from the server and displays it to the user.

[0931] Specific examples

[0932] For example, if a user posts content like this:

[0933] "This game is so fun! But that player said some horrible things."

[0934] 1. The user enters this information into the terminal and presses the send button.

[0935] 2. The device sends the posted content to the server.

[0936] 3. The server receives the posted content and requests the generative AI model to evaluate the content.

[0937] 4. The generative AI model analyzes the content of the post and generates an evaluation result, such as "avoid making negative comments about other players."

[0938] 5. The server receives the evaluation results, formats them together with the original post content, and sends them back to the device.

[0939] 6. The device displays the moderation results to the user and prompts them to correct their post.

[0940] This allows users to recognize and correct when their posts do not comply with community guidelines, and provides results that help maintain a healthy digital environment across online platforms.

[0941] The processing flow will be explained below.

[0942] Step 1:

[0943] A user enters content and presses the submit button. For example, they might enter, "This game is really fun! But that player said some horrible things."

[0944] Step 2:

[0945] The device converts the input content into JSON format, for example:

[0946] json

[0947] {

[0948] "content": "This game is really fun! But that player said some horrible things."

[0949] }

[0950] Step 3:

[0951] The device sends the converted JSON data to the server as an HTTP POST request. The request destination is the endpoint ( / filter) configured in the Flask framework.

[0952] Step 4:

[0953] The server processes the POST request received from the client device. At that time, it extracts the JSON data from the request body and extracts the user-submitted content. In this case, the extracted content is "This game is really fun! But that player said some horrible things."

[0954] Step 5:

[0955] The server passes the extracted content to the moderate_content function. This function uses a generative AI model to evaluate the content. Specifically, it calls the OpenAI API to perform content moderation. For example, it generates the following prompt:

[0956] Moderate the following content according to the community guidelines: "This game is really fun! But that player said some horrible things."

[0957] Step 6:

[0958] A generative AI model evaluates the content based on the prompts received and generates moderation results, such as "Avoid negative comments about other players."

[0959] Step 7:

[0960] The server receives the moderation results returned by the generative AI model and formats them along with the original content, creating a JSON response like this:

[0961] json

[0962] {

[0963] "original_content": "This game is really fun! But that player said some horrible things.",

[0964] "moderation_result": "Please avoid negative comments about other players."

[0965] }

[0966] Step 8:

[0967] The server sends a formatted JSON response to the client device.

[0968] Step 9:

[0969] The device receives the response from the server and displays the moderation results to the user, for example, feedback such as "Please avoid making negative comments about other players" is displayed on the screen.

[0970] Step 10:

[0971] The user then edits and corrects the content based on the feedback received and resubmits it if necessary.

[0972] Example 1

[0973] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0974] While the amount of user-generated content on modern online platforms is increasing, there is a need to ensure the quality and integrity of that content. However, manual moderation requires a great deal of effort and time, and it is difficult to respond in real time. This creates a risk that inappropriate content will be published, potentially damaging the integrity of the platform. To solve this problem, an automated moderation system is needed.

[0975] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0976] In this invention, the server includes means for receiving user-generated data from a client device, means for processing the generated data using a generative AI model, means for transmitting the processing results to the client device, communication means for accepting requests from the client device, data formatting means for transmitting data to the generative AI model, and result formatting means for formatting and returning evaluation results from the generative AI model, thereby enabling automatic evaluation and filtering of user-generated content in real time and immediate provision of moderation results.

[0977] A "client device" is an electronic device through which a user inputs information and transmits data to a server.

[0978] "Server" means a computer system that processes data received from a client device and performs evaluation and filtering using a generative AI model.

[0979] "User-Generated Data" is information generated and transmitted by a user through a client device.

[0980] A "generative AI model" is an artificial intelligence algorithm used to analyze user-generated data and generate evaluation results.

[0981] The "data formatting means for sending to the generative AI model" is a function within the server that converts the received data into a format that is easy for the generative AI model to process.

[0982] The "result formatting means for formatting and returning the evaluation results" is a function within the server that converts the evaluation results obtained from the generative AI model into a format that is easy for the client device to understand and transmits them.

[0983] "Communication means" refers to the interface and protocol for transmitting and receiving data between the client device and the server.

[0984] The system of the present invention aims to automate the moderation of user-generated data and maintain the integrity of content on online platforms. The system comprises a set of means including a client device, a server, and a generative AI model.

[0985] Server Configuration

[0986] The server has the functionality to receive user-generated data from the client device, send that data to the generative AI model, obtain the evaluation results, format them and send them back to the client device. The server is built using the following technologies:

[0987] Web framework: Flask

[0988] Generative AI model: OpenAI's GPT-4 API (specific name generalization)

[0989] The server uses the Flask framework to receive POST requests sent from the client device, then formats the received data to send to the generative AI model, which analyzes the data and generates an evaluation result. The server formats the evaluation result and sends a response back to the client device.

[0990] Device configuration

[0991] The device has the function of converting data generated by the user into JSON format and sending it to the server, and also receives the response sent back from the server and displays the result to the user.

[0992] Data transmission method: HTTP request library such as fetch API

[0993] User Interface: HTML, CSS, JavaScript

[0994] When a user enters data such as a comment or post on their device and presses the send button, the content is sent to the server, and when a response is returned from the server, the result is displayed on the screen.

[0995] User Actions

[0996] Users access the online platform using their devices, enter and submit their posts, receive moderation results, and amend their posts as necessary.

[0997] Input method: keyboard, touchscreen, etc.

[0998] Specific examples

[0999] For example, consider the case where a user posts content such as, "This game is really fun! But that player said some horrible things."

[1000] 1. User: Enter this information into the terminal and press the send button.

[1001] 2. Device: Send this post to the server.

[1002] 3. Server: Receives the posted content and requests the generative AI model to evaluate the content.

[1003] 4. Generative AI model: Analyzes the content of the post and generates an evaluation result such as "avoid making negative comments about other players."

[1004] 5. Server: Receives the evaluation results, formats them together with the original post, and sends them back to the device.

[1005] 6. On the device: Display the moderation results to the user and prompt them to correct their post.

[1006] Prompt Sentence Examples

[1007] The generative AI model evaluates posts by providing prompts like the following:

[1008] "This game is so fun! But that player said some horrible things."

[1009] Rate Us: Rate this content to see if it adheres to our Community Guidelines and let us know what changes need to be made.

[1010] Based on this prompt, the generative AI model generates appropriate feedback and provides the results to the user via the server.

[1011] This allows users to ensure that their posts comply with community guidelines, maintaining a healthy digital environment across online platforms.

[1012] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1013] Step 1:

[1014] The server receives user-generated data sent as a POST request from the client device. The input is user-generated data in JSON format, for example, {"content": "This game is really fun!"}. The output is to store the received data object for internal processing. Specifically, it uses the Flask framework to set up an endpoint / api / moderate to receive the request and extract the data from the request body.

[1015] Step 2:

[1016] The server formats the received data and prepares it to be sent to the generative AI model. The input is the data object received in the previous step, e.g., {"content": "This game is really fun!"}. The output is formatted data to be sent to the generative AI model, e.g., {"texts": ["This game is really fun!"]}. Specifically, it converts the data into formatted JSON format and prepares an API request to send to the generative AI model (GPT-4 API).

[1017] Step 3:

[1018] The generative AI model receives and analyzes formatted data. The input is formatted JSON data, and the output is a JSON response containing the evaluation results. For example, if the input is {"texts": ["This game is really fun!"]}, the output will be an evaluation result such as {"evaluation": "appropriate"}. In concrete terms, the generative AI model analyzes the data and evaluates each text based on community guidelines.

[1019] Step 4:

[1020] The server receives the evaluation results returned from the generative AI model, formats them together with the original data, and sends them back to the client device. The input is the evaluation result JSON from the generative AI model, for example, {"evaluation": "appropriate"}. The output is a formatted JSON response to be sent back to the client device, for example, {"original_content": "This game is really fun!", "evaluation": "appropriate"}. Specifically, the server formats the evaluation results, integrates them with the original data, and sends them back to the client device.

[1021] Step 5:

[1022] The device receives the response returned from the server and displays it to the user. The input is the JSON response returned from the server, for example, {"original_content": "This game is really fun!", "evaluation": "Appropriate"}. The output is the evaluation result message displayed to the user, for example, "Evaluation result: Appropriate". Specifically, it uses JavaScript to parse the response and reflects the result in the HTML DOM.

[1023] Step 6:

[1024] The user checks the moderation results and modifies the post as necessary. The input is the evaluation result displayed on the device and the original post content, for example, "Evaluation result: appropriate" and "This game is really fun!". The output is the modified post content, for example, "This game is really fun!". Specifically, the user re-enters the post based on the presented evaluation results, reviews the content, and resubmits it from the device.

[1025] (Application example 1)

[1026] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1027] In recent years, online chat and messaging services have rapidly become popular, making maintaining the integrity of user-generated content a key challenge. Messages containing inappropriate language, suicidal thoughts, and violent language are particularly problematic. However, moderating this content in real time is extremely difficult, necessitating the development of automated and efficient solutions.

[1028] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1029] In this invention, the server includes means for receiving user-generated data from a client device, means for processing the generated data using a generative artificial intelligence model, means for transmitting the processing results to the client device, means for moderating user-generated messages in real time, and means for detecting inappropriate content and providing a warning or appropriate response information, thereby making it possible to maintain the integrity of content in chat and messaging services in real time and prevent the spread of inappropriate content.

[1030] A "generation system" is a system that processes and filters user-generated data and transmits the results to a client device.

[1031] A "client device" is a device (e.g., a smartphone or personal computer) that a user uses to input data and communicate with the production system.

[1032] "User-Generated Data" means text messages and other data that a user generates and sends through a client device.

[1033] A "generative artificial intelligence model" is an artificial intelligence-based software model for analyzing received data and performing appropriate processing and filtering.

[1034] "Moderation" is the process of determining whether the generated data complies with community guidelines, etc., and correcting the content or issuing a warning as necessary.

[1035] "Real-time" means that user data is processed and filtered almost instantly after it is generated.

[1036] "Filtering" is the process of detecting and removing or flagging inappropriate content from generated data.

[1037] A "warning" is a message that notifies the user when the data generated by the user is inappropriate and urges the user to correct it.

[1038] "Response information" refers to information about appropriate assistance and support provided to users when inappropriate content is detected.

[1039] "Integrity" refers to the content on online platforms being socially appropriate and being safe for users to use.

[1040] The present invention provides a system for real-time moderation of data generated by users in online chat and messaging services to prevent the spread of inappropriate content. The system comprises a client device, a server, and a generative artificial intelligence model.

[1041] System configuration

[1042] server:

[1043] The server uses the Flask framework to build a web server and set up endpoints to accept requests from client devices. The server receives user-generated data, processes and filters it using a generative artificial intelligence model, and formats the results and sends them back to the client device as a response.

[1044] Device:

[1045] The device converts the data generated by the user into JSON format and sends it to the server as a POST request. The device receives the moderation results returned by the server and displays them to the user.

[1046] User:

[1047] Users access online chat and messaging services using their devices, input and send messages, and receive moderation results, allowing them to modify the content of their messages as necessary.

[1048] Program processing flow

[1049] server:

[1050] The server uses the Flask framework to build a web server and set up an endpoint. It receives messages sent by users as POST requests from client devices. The server sends the data to a generative artificial intelligence model (e.g., OpenAI's GPT-3) to evaluate whether the message is appropriate. It then sends a response containing the evaluation results back to the client device.

[1051] Device:

[1052] The device converts the data generated by the user into JSON format and sends it to the server as a POST request, receives the response sent back from the server, and displays it to the user.

[1053] Specific examples

[1054] For example, if a user types the following message into chat:

[1055] "There's no point in living anymore."

[1056] 1. The user enters this information into the terminal and presses the send button.

[1057] 2. The terminal sends this message to the server.

[1058] 3. The server receives the message and requests an evaluation from the generative artificial intelligence model.

[1059] 4. The generative AI model analyzes the message and generates an evaluation result, such as, "This message may contain suicidal thoughts. Please consult a specialist."

[1060] 5. The server receives the evaluation result, formats it together with the original message, and sends it back to the terminal.

[1061] 6. The device displays the moderation results to the user and provides appropriate support information as needed.

[1062] Prompt Sentence Examples

[1063] "Does this message possibly contain suicidal thoughts? 'There's no point in living anymore.'\n\nPlease suggest an appropriate response to this message."

[1064] The present invention makes it possible to maintain the integrity of content in chat and messaging services in real time and prevent the spread of inappropriate content.

[1065] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1066] Step 1:

[1067] A user enters a message on the chat or messaging service and presses the send button, which causes the entered message to be received by the client device.

[1068] Input: The message entered by the user

[1069] Output: Message sent to the client device

[1070] Step 2:

[1071] The device converts the message entered by the user into JSON format and sends a POST request to the server, encoding the text as message data in JSON format.

[1072] Input: The message entered by the user

[1073] Output: POST request containing message data in JSON format

[1074] Step 3:

[1075] The server receives the POST request sent from the client device, analyzes the received data, and extracts the message content.

[1076] Input: A POST request containing message data in JSON format.

[1077] Output: Extracted text message

[1078] Step 4:

[1079] The server then sends the extracted messages to a generative artificial intelligence model for appropriate moderation evaluation, specifically using OpenAI's GPT-3 model to analyze the content of the messages and determine their appropriateness.

[1080] Input: Extracted text message

[1081] Output: Moderation evaluation results and warning messages

[1082] Step 5:

[1083] The generative AI model analyzes the content of received messages and evaluates their appropriateness based on the prompt. For example, it generates a response to a prompt such as, "'There's no point in living anymore.' Could this message contain suicidal thoughts?"

[1084] Input: Extracted text message and prompt

[1085] Output: Suitability assessment and warning messages

[1086] Step 6:

[1087] The server receives the evaluation results from the generative artificial intelligence model, formats them, and sends them back to the client device along with the original message.

[1088] Input: Moderation evaluation results and warning messages

[1089] Output: A response containing the formatted evaluation result and the original message.

[1090] Step 7:

[1091] The terminal receives the response from the server and displays the moderation results to the user, who can then modify the message content as necessary.

[1092] Input: Formatted evaluation results and original message

[1093] Output: Moderation results and prompts shown to the user

[1094] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1095] The present invention relates to a generation system for moderating user-generated data in real time to maintain the integrity of content on online platforms. The system includes a set of means for receiving data posted from a client device, processing the data using a generative artificial intelligence model, and transmitting the processing results to the client device. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system achieves more advanced moderation based on the user's emotional state.

[1096] System configuration

[1097] server:

[1098] The server uses the Flask framework to build a web server and sets up an endpoint that accepts requests from client devices.

[1099] The server receives user-submitted data and evaluates and filters the content using an emotion engine and generative artificial intelligence models.

[1100] The server formats the evaluation and filtering results and sends them back to the client device as a response.

[1101] Device:

[1102] The terminal sends a request containing user-generated data to the server.

[1103] The terminal displays the moderation results received from the server to the user.

[1104] User:

[1105] Users use their terminals to access the online platform, input content to post, and send it.

[1106] Users receive the results of the moderation and can modify their posts as necessary.

[1107] Details of system processing

[1108] Processing flow

[1109] server:

[1110] 1. The server receives user-generated data sent as a POST request from a client device.

[1111] 2. The server retrieves the received data and sends it to the emotion engine, which analyzes the emotions from the user-generated data and provides the results to the generation system.

[1112] 3. The server requests the generative artificial intelligence model to evaluate the content based on the emotion analysis results obtained from the emotion engine.

[1113] 4. The biointelligence model uses this data and sentiment analysis results to moderate the content, for example, easing the rating if it contains positive sentiment and stricter rating if it contains negative sentiment.

[1114] 5. The server formats the evaluation results from the biointelligence model and sends them back to the client device along with the original data.

[1115] Device:

[1116] 1. The terminal converts the data entered by the user into JSON format and sends a POST request to the server.

[1117] 2. The device receives the response sent back from the server and displays it to the user.

[1118] Specific examples

[1119] For example, if a user posts content like this:

[1120] "This game is so fun! But that player said some horrible things."

[1121] 1. The user enters this information into the terminal and presses the send button.

[1122] 2. The device sends the post to the server.

[1123] 3. The server receives the post and requests the emotion engine to analyze the sentiment of the content.

[1124] 4. The sentiment engine recognizes that the post contains negative sentiment (e.g., negative comments).

[1125] 5. The server requests strict moderation of the biointelligence model based on the results from the emotion engine.

[1126] 6. The biointelligence model analyzes the content of the posts and generates evaluation results such as "avoid making negative comments about other players."

[1127] 7. The server receives the evaluation results, formats them together with the original post content, and sends them back to the device.

[1128] 8. The device displays the moderation results to the user and prompts them to correct their post.

[1129] This allows users to recognize when content violates guidelines and correct it, and the emotional engine enables appropriate moderation based on the user's emotional state, helping to provide a healthy digital environment across online platforms.

[1130] The processing flow will be explained below.

[1131] Step 1:

[1132] The user enters content and presses the post button. For example, they might enter, "This game is really fun! But that player said some horrible things."

[1133] Step 2:

[1134] The terminal converts the input content into JSON format, which looks like this:

[1135] json

[1136] {

[1137] "content": "This game is really fun! But that player said some horrible things."

[1138] }

[1139] Step 3:

[1140] The device sends the converted JSON data to the server as an HTTP POST request. The request destination is the endpoint ( / filter) configured in the Flask framework.

[1141] Step 4:

[1142] The server receives a POST request from the client device. In this case, the received data is "This game is really fun! But that player said some horrible things."

[1143] Step 5:

[1144] The server sends the received data to the emotion engine, which analyzes the emotions from the user-generated data and determines, for example, "positive" or "negative" emotions.

[1145] Step 6:

[1146] The emotion engine analyzes the input content and recognizes that it contains negative emotions. For example, the phrase "he said horrible things" is identified as a negative emotion.

[1147] Step 7:

[1148] The server requests strict moderation evaluation from the generative artificial intelligence model based on the analysis results of negative emotions obtained from the emotion engine.

[1149] Step 8:

[1150] A generative AI model will critically evaluate your submission, generating prompts such as:

[1151] Moderate the following content according to the community guidelines, considering that it includes negative sentiment: "This game is really fun! But that player said some horrible things."

[1152] Step 9:

[1153] A generative AI model generates content evaluation results, such as a moderation result like "Please avoid making negative comments about other players."

[1154] Step 10:

[1155] The server receives the moderation results returned by the generative AI model and formats them along with the original content, creating a JSON response like this:

[1156] json

[1157] {

[1158] "original_content": "This game is really fun! But that player said some horrible things.",

[1159] "moderation_result": "Please avoid negative comments about other players."

[1160] }

[1161] Step 11:

[1162] The server sends a formatted JSON response to the client device.

[1163] Step 12:

[1164] The device receives the response from the server and displays the moderation results to the user, for example, feedback such as "Please avoid making negative comments about other players" is displayed on the screen.

[1165] Step 13:

[1166] The user then edits and corrects the content based on the feedback received and resubmits it if necessary.

[1167] Example 2

[1168] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1169] Modern online platforms require the maintenance of the integrity of content generated by users, as a large amount of content is posted in real time. However, manual moderation is time-consuming and costly, and it is difficult to cover all content. To solve this problem, an automated real-time moderation system is needed. Furthermore, advanced moderation that takes user sentiment into account is required, rather than simply moderating content.

[1170] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1171] In this invention, the server includes means for receiving user-generated data from a client device, means for transmitting the generated data to a sentiment analysis engine to perform sentiment analysis, means for requesting data moderation using a generative AI model based on the sentiment analysis result, and means for formatting and transmitting the moderation result to the client device, thereby automatically and efficiently maintaining content integrity in real time and enabling more advanced moderation based on emotional states.

[1172] A "client device" is a terminal device used by a user, and is a device for sending and receiving data to a server via the Internet.

[1173] "User-generated data" refers to content, such as text, images, and video, entered by a user and transmitted through a client device.

[1174] A "sentiment analysis engine" is a software or hardware component for extracting and analyzing emotional information from user-generated data.

[1175] A "generative AI model" is a program or system that uses artificial intelligence techniques to generate, process, and analyze data.

[1176] "Moderation" is the process of evaluating user-generated content and filtering and adjusting it to ensure compliance with community guidelines and other regulations.

[1177] "Formatting" is the process of organizing processing results and data into a specific format to make them easier to read.

[1178] "Community guidelines" refer to rules and standards that govern appropriate behavior and content posted on online platforms.

[1179] "Real-time" means that data processing and moderation occurs immediately, without delay.

[1180] A "POST request" is a type of HTTP request for sending data from a client device to a server.

[1181] An "API endpoint" is a defined server URL that other programs or services can access, providing access to specific functionality or data.

[1182] This invention relates to a system for real-time moderation of data generated by users on online platforms to maintain the integrity of the platform. The system collects data posted by users from their devices, analyzes, evaluates, and filters the data using a sentiment analysis engine and generative AI model, and provides feedback on the evaluation results to users to encourage them to maintain healthy content.

[1183] System configuration

[1184] 1. Server:

[1185] The server uses the Flask framework to build a web server. Flask is a lightweight web framework that allows for fast deployment and simple API design.

[1186] The server receives user-generated data from the client device as a POST request, with the data being transmitted in JSON format.

[1187] The server sends the received data to a sentiment analysis engine, which can be implemented using a commercial API such as IBM Watson Tone Analyzer.

[1188] The server receives the results of the sentiment analysis and requests moderation from a generative AI model based on the results. For example, OpenAI's GPT-4 is used as the generative AI model.

[1189] The server formats the evaluation results from the generative AI model and sends them back to the client device.

[1190] 2. Terminal:

[1191] The terminal converts the data entered by the user into JSON format and sends a POST request to the server.

[1192] The terminal receives the moderation results returned from the server and displays them to the user.

[1193] 3. User:

[1194] A user accesses the online platform using a terminal, inputs the content to be posted, and sends it.

[1195] The user reviews the moderation results and corrects the post if necessary.

[1196] Specific examples

[1197] For example, if a user posts something like this:

[1198] "This game is so fun! But that player said some horrible things."

[1199] 1. The user enters the content of this post into the device and presses the send button.

[1200] 2. The device converts the post content into JSON format and sends it to the server.

[1201] 3. The server receives the post and requests a sentiment analysis engine to analyze it, which analyzes both positive and negative sentiment.

[1202] 4. The server requests moderation from the generative AI model based on the results of the sentiment analysis engine. The generative AI model evaluates the content of the post while taking into account the results of the sentiment analysis.

[1203] 5. The server formats the evaluation results from the generated AI model and sends them back to the device.

[1204] 6. The device receives the moderation results from the server and displays them to the user, for example, feedback such as "Please avoid making negative comments about other players."

[1205] 7. Users review the feedback and modify their posts as needed. For example, they might post only "This game is really fun!"

[1206] Prompt Sentence Examples

[1207] A concrete example of a prompt for a generative AI model might be something like this:

[1208] User post: "This game is so fun! But that player said some horrible things."

[1209] Sentiment analysis results: "Positive sentiment: 0.7, Negative sentiment: 0.3"

[1210] This allows the generative AI model to perform appropriate moderation based on user comments.

[1211] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1212] Step 1:

[1213] The server receives a POST request from a client device.

[1214] Input: User-generated data (in text format) sent from the client device.

[1215] What it does: The server uses the Flask framework to set up an endpoint at / moderate to receive POST requests.

[1216] Output: The received user-generated data is persisted in the server.

[1217] Step 2:

[1218] The server sends the received data to the sentiment analysis engine.

[1219] Input: User-generated data received in step 1.

[1220] How it works: A server sends user-generated data to an API endpoint of a sentiment analysis engine (e.g., IBM Watson Tone Analyzer).

[1221] Output: The sentiment analysis result from the sentiment analysis engine (e.g., positive sentiment: 0.7, negative sentiment: 0.3) is returned.

[1222] Step 3:

[1223] The server requests moderation from the generative AI model based on the results of emotion analysis.

[1224] Input: Sentiment analysis results obtained in step 2 and the original user-generated data.

[1225] How it works: The server creates an input prompt for a generative AI model (e.g., OpenAI GPT-4) and sends it to the model.

[1226] Output: Moderation results from the generative AI model (e.g., "Please avoid negative comments to other players") are returned.

[1227] Step 4:

[1228] The server formats and transmits the moderation results from the generative AI model to the client device.

[1229] Input: Moderation results obtained in step 3.

[1230] How it works: The server formats the moderation results into a specific format, such as JSON.

[1231] Output: Formatted moderation results are sent to the client device.

[1232] Step 5:

[1233] The terminal receives the moderation results returned from the server and displays them to the user.

[1234] Input: Moderation results sent by the server in step 4.

[1235] Operation: The terminal receives the response from the server and displays the content to the user, for example, as a pop-up message or a dialog box.

[1236] Output: A display screen where users can see the moderation results.

[1237] Step 6:

[1238] The user reviews the moderation results and corrects the post if necessary.

[1239] Input: Moderation results displayed in Step 5.

[1240] Action: The user reviews the feedback message and follows the instructions to edit or modify the post. For example, "Delete negative comments about other players."

[1241] Output: A new, revised post is generated, ready to be sent again.

[1242] (Application example 2)

[1243] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1244] In conventional content distribution services, if content posted by users contains inappropriate content, manual moderation is required, which is inefficient and poses a risk to the integrity of the platform. In particular, real-time moderation is difficult, which poses the risk of inappropriate content spreading instantly.

[1245] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving data generated by a user from a client device, means for processing the generated data using a generative artificial intelligence model, means for performing sentiment analysis on the user-generated data, and means for transmitting the processing results to the client device. This enables sound moderation in real time in content distribution services, preventing the spread of inappropriate content and improving the user experience.

[1246] A "client device" is a device that transmits user-generated data to a server, and examples include smartphones and personal computers.

[1247] "Generated Data" means user-generated content, whether in the form of text, images, video, or other content.

[1248] A "generative artificial intelligence model" is an artificial intelligence algorithm used to process generated data and evaluate or filter its content.

[1249] "Means for processing" refers to the function of analyzing and evaluating the generated data using a generative artificial intelligence model.

[1250] "Sentiment analysis" is the process of recognizing the emotional state of a user from their generated data and determining whether it is positive or negative.

[1251] "Processing results" refers to information generated as a result of evaluation and filtering performed on generated data.

[1252] The "means for transmitting" refers to a function for returning the processing results to the client device.

[1253] "Community guidelines" refer to the rules and standards that users must follow on an online platform.

[1254] The present invention is a generative system for moderating user-generated data from client devices in real time to maintain the health of online platforms. The system combines a sentiment analysis engine and a generative artificial intelligence model to achieve advanced moderation based on the user's emotional state.

[1255] System configuration

[1256] server

[1257] The server first sets up an endpoint to accept requests from client devices. This uses the Flask framework. The server receives data posted by users and sends it to an emotion engine for sentiment analysis. This makes it possible to recognize the user's emotional state. Next, the server requests a content evaluation from a generative AI model based on the results of the sentiment analysis. The generative AI model then performs moderation based on the results of the sentiment analysis.

[1258] Terminal

[1259] The device converts user-generated data into JSON format and sends a POST request to the server. It receives the response from the server and displays it to the user, providing feedback on the moderation results. This allows users to check whether the content violates the platform's guidelines and make corrections if necessary.

[1260] User

[1261] Users access the online platform using their devices, input and submit content to be posted, and then check the moderation results from the server on their devices and modify the content according to the instructions.

[1262] Hardware and software used

[1263] Hardware:

[1264] Client devices such as smartphones and PCs

[1265] software:

[1266] On the server side, the Flask framework

[1267] Sentiment Engine API for sentiment analysis

[1268] Content rating system using live artificial intelligence models

[1269] React Native on the client side

[1270] Specific examples

[1271] For example, consider the case where a user attempts to post the following content:

[1272] "This video is great, but the comments section is awful."

[1273] 1. The user enters this information into the terminal and presses the send button.

[1274] 2. The device sends the post to the server.

[1275] 3. The server receives the post and runs it through a sentiment analysis engine.

[1276] 4. The sentiment analysis engine determines that the comment contains negative comments.

[1277] 5. The server requests moderation from the live AI model based on the results from the sentiment analysis engine.

[1278] 6. A raw AI model detects the phrase "The comments section is awful" and rates it as inappropriate.

[1279] 7. The server returns the evaluation results to the device.

[1280] 8. The device displays feedback to the user, prompting them to revise their statement, "The comments section is terrible."

[1281] Prompt Sentence Examples

[1282] "Please send the following text to the Sentiment Engine API for sentiment analysis:

[1283] A user wrote: "This video is great, but the comments section is awful."

[1284] Emotion Engine API endpoint: https: / / emotion-api.example.com / analyze

[1285] "

[1286] To generate smart moderation feedback, use prompts like these:

[1287] The post read: "The comments section is awful."

[1288] Generative AI model API endpoint: https: / / ai-moderation-api.example.com / moderate

[1289]

[1290] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1291] Step 1:

[1292] The user types content (e.g., "This video is great, but the comments section is terrible") into the device and hits send.

[1293] Input: User generated content text

[1294] Output: POST request sent from the terminal to the server

[1295] Specific behavior: The user enters content into the text input area on the device and clicks the send button, which sends a request in JSON format to the server.

[1296] Step 2:

[1297] The device converts the user-generated data and sends a POST request to the server.

[1298] Input: Content text entered by the user

[1299] Output: POST request sent to the server

[1300] Specific operation: The device detects that the send button has been pressed, converts the entered text into JSON format, and sends an HTTP POST request to the server endpoint.

[1301] Step 3:

[1302] The server receives the posted content and sends the text data to the sentiment analysis engine.

[1303] Input: Content text in JSON format

[1304] Output: Request to the sentiment analysis engine

[1305] Specific operation: The server receives the data sent from the client and sends a sentiment analysis request to the Emotion Engine API, including the content text in the body of the POST request and sending it to the API endpoint.

[1306] Step 4:

[1307] The sentiment analysis engine analyzes the text data and returns the sentiment analysis results.

[1308] Input: Content text

[1309] Output: Sentiment analysis result (e.g. positive, negative, neutral)

[1310] What it does: The sentiment analysis engine analyzes the received text data, evaluates the sentiment of each word or phrase, classifies the emotional state, and sends the results back to the server in JSON format.

[1311] Step 5:

[1312] The server receives the sentiment analysis results and requests the generative artificial intelligence model to evaluate the content.

[1313] Input: Sentiment analysis results

[1314] Output: A request to the generative AI model

[1315] What it does: The server receives the analysis results from the sentiment engine and sends a moderation request to the generative AI model API, which includes the sentiment analysis results and the original text.

[1316] Step 6:

[1317] The generative AI model performs moderation based on content and sentiment analysis results and generates evaluation results.

[1318] Input: Content text and sentiment analysis results

[1319] Output: Moderation evaluation result (e.g., classifying "The comments section is awful" as inappropriate)

[1320] Specific operation: The generative AI model analyzes the content text and sentiment analysis results, performs moderation based on the policy, filters and evaluates the content, and generates results, which are then sent back to the server.

[1321] Step 7:

[1322] The server receives the moderation evaluation results from the generative AI model, formats them, and sends them back to the device.

[1323] Input: Moderation evaluation results

[1324] Output: Response to the terminal

[1325] Specific operation: The server receives the evaluation results from the generative AI model, formats them to provide appropriate feedback to the user, and then sends the formatted results back to the device in JSON format.

[1326] Step 8:

[1327] The device will display the moderation results to the user and prompt them to correct the content if necessary.

[1328] Input: Moderation evaluation results

[1329] Output: Display feedback to the user

[1330] Specific operation: The device analyzes the JSON data returned from the server and displays the evaluation results in the user interface. The device prompts the user to make the necessary corrections by showing the specific areas and content of corrections.

[1331] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1332] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1333] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1334] [Fourth embodiment]

[1335] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1336] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1337] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1338] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1339] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1340] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1341] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1342] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1343] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1344] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1345] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1346] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1347] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1348] The present invention relates to a generation system for automating the moderation of user-generated data and maintaining the integrity of content on online platforms, comprising a set of means for receiving data posted from a client device, processing the data using a generative artificial intelligence model, and transmitting the processing results to the client device.

[1349] System configuration

[1350] server:

[1351] The server uses the Flask framework to build a web server and sets up an endpoint that accepts requests from client devices.

[1352] The server receives user-submitted data and evaluates and filters the content using a generative artificial intelligence model.

[1353] The server formats the evaluation and filtering results and sends them back to the client device as a response.

[1354] Device:

[1355] The terminal sends a request containing user-generated data to the server.

[1356] The terminal displays the moderation results received from the server to the user.

[1357] User:

[1358] Users use their terminals to access the online platform, input content to post, and send it.

[1359] Users receive the results of the moderation and can modify their posts as necessary.

[1360] Program processing flow

[1361] server:

[1362] 1. The server receives user-generated data sent as a POST request from a client device.

[1363] 2. The server takes the received data and sends it to the generative artificial intelligence model.

[1364] 3. The generative AI model determines whether the data complies with community guidelines and generates an evaluation result.

[1365] 4. The server formats the evaluation results from the generative artificial intelligence model and sends them back to the client device along with the original data.

[1366] Device:

[1367] 1. The terminal converts the data entered by the user into JSON format and sends a POST request to the server.

[1368] 2. The device receives the response sent back from the server and displays it to the user.

[1369] Specific examples

[1370] For example, if a user posts content like this:

[1371] "This game is so fun! But that player said some horrible things."

[1372] 1. The user enters this information into the terminal and presses the send button.

[1373] 2. The device sends the posted content to the server.

[1374] 3. The server receives the posted content and requests the generative AI model to evaluate the content.

[1375] 4. The generative AI model analyzes the content of the post and generates an evaluation result, such as "avoid making negative comments about other players."

[1376] 5. The server receives the evaluation results, formats them together with the original post content, and sends them back to the device.

[1377] 6. The device displays the moderation results to the user and prompts them to correct their post.

[1378] This allows users to recognize and correct when their posts do not comply with community guidelines, and provides results that help maintain a healthy digital environment across online platforms.

[1379] The processing flow will be explained below.

[1380] Step 1:

[1381] A user enters content and presses the submit button. For example, they might enter, "This game is really fun! But that player said some horrible things."

[1382] Step 2:

[1383] The device converts the input content into JSON format, for example:

[1384] json

[1385] {

[1386] "content": "This game is really fun! But that player said some horrible things."

[1387] }

[1388] Step 3:

[1389] The device sends the converted JSON data to the server as an HTTP POST request. The request destination is the endpoint ( / filter) configured in the Flask framework.

[1390] Step 4:

[1391] The server processes the POST request received from the client device. At that time, it extracts the JSON data from the request body and extracts the user-submitted content. In this case, the extracted content is "This game is really fun! But that player said some horrible things."

[1392] Step 5:

[1393] The server passes the extracted content to the moderate_content function. This function uses a generative AI model to evaluate the content. Specifically, it calls the OpenAI API to perform content moderation. For example, it generates the following prompt:

[1394] Moderate the following content according to the community guidelines: "This game is really fun! But that player said some horrible things."

[1395] Step 6:

[1396] A generative AI model evaluates the content based on the prompts received and generates moderation results, such as "Avoid negative comments about other players."

[1397] Step 7:

[1398] The server receives the moderation results returned by the generative AI model and formats them along with the original content, creating a JSON response like this:

[1399] json

[1400] {

[1401] "original_content": "This game is really fun! But that player said some horrible things.",

[1402] "moderation_result": "Please avoid negative comments about other players."

[1403] }

[1404] Step 8:

[1405] The server sends a formatted JSON response to the client device.

[1406] Step 9:

[1407] The device receives the response from the server and displays the moderation results to the user, for example, feedback such as "Please avoid making negative comments about other players" is displayed on the screen.

[1408] Step 10:

[1409] The user then edits and corrects the content based on the feedback received and resubmits it if necessary.

[1410] Example 1

[1411] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1412] While the amount of user-generated content on modern online platforms is increasing, there is a need to ensure the quality and integrity of that content. However, manual moderation requires a great deal of effort and time, and it is difficult to respond in real time. This creates a risk that inappropriate content will be published, potentially damaging the integrity of the platform. To solve this problem, an automated moderation system is needed.

[1413] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1414] In this invention, the server includes means for receiving user-generated data from a client device, means for processing the generated data using a generative AI model, means for transmitting the processing results to the client device, communication means for accepting requests from the client device, data formatting means for transmitting data to the generative AI model, and result formatting means for formatting and returning evaluation results from the generative AI model, thereby enabling automatic evaluation and filtering of user-generated content in real time and immediate provision of moderation results.

[1415] A "client device" is an electronic device through which a user inputs information and transmits data to a server.

[1416] "Server" means a computer system that processes data received from a client device and performs evaluation and filtering using a generative AI model.

[1417] "User-Generated Data" is information generated and transmitted by a user through a client device.

[1418] A "generative AI model" is an artificial intelligence algorithm used to analyze user-generated data and generate evaluation results.

[1419] The "data formatting means for sending to the generative AI model" is a function within the server that converts the received data into a format that is easy for the generative AI model to process.

[1420] The "result formatting means for formatting and returning the evaluation results" is a function within the server that converts the evaluation results obtained from the generative AI model into a format that is easy for the client device to understand and transmits them.

[1421] "Communication means" refers to the interface and protocol for transmitting and receiving data between the client device and the server.

[1422] The system of the present invention aims to automate the moderation of user-generated data and maintain the integrity of content on online platforms. The system comprises a set of means including a client device, a server, and a generative AI model.

[1423] Server Configuration

[1424] The server has the functionality to receive user-generated data from the client device, send that data to the generative AI model, obtain the evaluation results, format them and send them back to the client device. The server is built using the following technologies:

[1425] Web framework: Flask

[1426] Generative AI model: OpenAI's GPT-4 API (specific name generalization)

[1427] The server uses the Flask framework to receive POST requests sent from the client device, then formats the received data to send to the generative AI model, which analyzes the data and generates an evaluation result. The server formats the evaluation result and sends a response back to the client device.

[1428] Device configuration

[1429] The device has the function of converting data generated by the user into JSON format and sending it to the server, and also receives the response sent back from the server and displays the result to the user.

[1430] Data transmission method: HTTP request library such as fetch API

[1431] User Interface: HTML, CSS, JavaScript

[1432] When a user enters data such as a comment or post on their device and presses the send button, the content is sent to the server, and when a response is returned from the server, the result is displayed on the screen.

[1433] User Actions

[1434] Users access the online platform using their devices, enter and submit their posts, receive moderation results, and amend their posts as necessary.

[1435] Input method: keyboard, touchscreen, etc.

[1436] Specific examples

[1437] For example, consider the case where a user posts content such as, "This game is really fun! But that player said some horrible things."

[1438] 1. User: Enter this information into the terminal and press the send button.

[1439] 2. Device: Send this post to the server.

[1440] 3. Server: Receives the posted content and requests the generative AI model to evaluate the content.

[1441] 4. Generative AI model: Analyzes the content of the post and generates an evaluation result such as "avoid making negative comments about other players."

[1442] 5. Server: Receives the evaluation results, formats them together with the original post, and sends them back to the device.

[1443] 6. On the device: Display the moderation results to the user and prompt them to correct their post.

[1444] Prompt Sentence Examples

[1445] The generative AI model evaluates posts by providing prompts like the following:

[1446] "This game is so fun! But that player said some horrible things."

[1447] Rate Us: Rate this content to see if it adheres to our Community Guidelines and let us know what changes need to be made.

[1448] Based on this prompt, the generative AI model generates appropriate feedback and provides the results to the user via the server.

[1449] This allows users to ensure that their posts comply with community guidelines, maintaining a healthy digital environment across online platforms.

[1450] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1451] Step 1:

[1452] The server receives user-generated data sent as a POST request from the client device. The input is user-generated data in JSON format, for example, {"content": "This game is really fun!"}. The output is to store the received data object for internal processing. Specifically, it uses the Flask framework to set up an endpoint / api / moderate to receive the request and extract the data from the request body.

[1453] Step 2:

[1454] The server formats the received data and prepares it to be sent to the generative AI model. The input is the data object received in the previous step, e.g., {"content": "This game is really fun!"}. The output is formatted data to be sent to the generative AI model, e.g., {"texts": ["This game is really fun!"]}. Specifically, it converts the data into formatted JSON format and prepares an API request to send to the generative AI model (GPT-4 API).

[1455] Step 3:

[1456] The generative AI model receives and analyzes formatted data. The input is formatted JSON data, and the output is a JSON response containing the evaluation results. For example, if the input is {"texts": ["This game is really fun!"]}, the output will be an evaluation result such as {"evaluation": "appropriate"}. In concrete terms, the generative AI model analyzes the data and evaluates each text based on community guidelines.

[1457] Step 4:

[1458] The server receives the evaluation results returned from the generative AI model, formats them together with the original data, and sends them back to the client device. The input is the evaluation result JSON from the generative AI model, for example, {"evaluation": "appropriate"}. The output is a formatted JSON response to be sent back to the client device, for example, {"original_content": "This game is really fun!", "evaluation": "appropriate"}. Specifically, the server formats the evaluation results, integrates them with the original data, and sends them back to the client device.

[1459] Step 5:

[1460] The device receives the response returned from the server and displays it to the user. The input is the JSON response returned from the server, for example, {"original_content": "This game is really fun!", "evaluation": "Appropriate"}. The output is the evaluation result message displayed to the user, for example, "Evaluation result: Appropriate". Specifically, it uses JavaScript to parse the response and reflects the result in the HTML DOM.

[1461] Step 6:

[1462] The user checks the moderation results and modifies the post as necessary. The input is the evaluation result displayed on the device and the original post content, for example, "Evaluation result: appropriate" and "This game is really fun!". The output is the modified post content, for example, "This game is really fun!". Specifically, the user re-enters the post based on the presented evaluation results, reviews the content, and resubmits it from the device.

[1463] (Application example 1)

[1464] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1465] In recent years, online chat and messaging services have rapidly become popular, making maintaining the integrity of user-generated content a key challenge. Messages containing inappropriate language, suicidal thoughts, and violent language are particularly problematic. However, moderating this content in real time is extremely difficult, necessitating the development of automated and efficient solutions.

[1466] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1467] In this invention, the server includes means for receiving user-generated data from a client device, means for processing the generated data using a generative artificial intelligence model, means for transmitting the processing results to the client device, means for moderating user-generated messages in real time, and means for detecting inappropriate content and providing a warning or appropriate response information, thereby making it possible to maintain the integrity of content in chat and messaging services in real time and prevent the spread of inappropriate content.

[1468] A "generation system" is a system that processes and filters user-generated data and transmits the results to a client device.

[1469] A "client device" is a device (e.g., a smartphone or personal computer) that a user uses to input data and communicate with the production system.

[1470] "User-Generated Data" means text messages and other data that a user generates and sends through a client device.

[1471] A "generative artificial intelligence model" is an artificial intelligence-based software model for analyzing received data and performing appropriate processing and filtering.

[1472] "Moderation" is the process of determining whether the generated data complies with community guidelines, etc., and correcting the content or issuing a warning as necessary.

[1473] "Real-time" means that user data is processed and filtered almost instantly after it is generated.

[1474] "Filtering" is the process of detecting and removing or flagging inappropriate content from generated data.

[1475] A "warning" is a message that notifies the user when the data generated by the user is inappropriate and urges the user to correct it.

[1476] "Response information" refers to information about appropriate assistance and support provided to users when inappropriate content is detected.

[1477] "Integrity" refers to the content on online platforms being socially appropriate and being safe for users to use.

[1478] The present invention provides a system for real-time moderation of data generated by users in online chat and messaging services to prevent the spread of inappropriate content. The system comprises a client device, a server, and a generative artificial intelligence model.

[1479] System configuration

[1480] server:

[1481] The server uses the Flask framework to build a web server and set up endpoints to accept requests from client devices. The server receives user-generated data, processes and filters it using a generative artificial intelligence model, and formats the results and sends them back to the client device as a response.

[1482] Device:

[1483] The device converts the data generated by the user into JSON format and sends it to the server as a POST request. The device receives the moderation results returned by the server and displays them to the user.

[1484] User:

[1485] Users access online chat and messaging services using their devices, input and send messages, and receive moderation results, allowing them to modify the content of their messages as necessary.

[1486] Program processing flow

[1487] server:

[1488] The server uses the Flask framework to build a web server and set up an endpoint. It receives messages sent by users as POST requests from client devices. The server sends the data to a generative artificial intelligence model (e.g., OpenAI's GPT-3) to evaluate whether the message is appropriate. It then sends a response containing the evaluation results back to the client device.

[1489] Device:

[1490] The device converts the data generated by the user into JSON format and sends it to the server as a POST request, receives the response sent back from the server, and displays it to the user.

[1491] Specific examples

[1492] For example, if a user types the following message into chat:

[1493] "There's no point in living anymore."

[1494] 1. The user enters this information into the terminal and presses the send button.

[1495] 2. The terminal sends this message to the server.

[1496] 3. The server receives the message and requests an evaluation from the generative artificial intelligence model.

[1497] 4. The generative AI model analyzes the message and generates an evaluation result, such as, "This message may contain suicidal thoughts. Please consult a specialist."

[1498] 5. The server receives the evaluation result, formats it together with the original message, and sends it back to the terminal.

[1499] 6. The device displays the moderation results to the user and provides appropriate support information as needed.

[1500] Prompt Sentence Examples

[1501] "Does this message possibly contain suicidal thoughts? 'There's no point in living anymore.'\n\nPlease suggest an appropriate response to this message."

[1502] The present invention makes it possible to maintain the integrity of content in chat and messaging services in real time and prevent the spread of inappropriate content.

[1503] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1504] Step 1:

[1505] A user enters a message on the chat or messaging service and presses the send button, which causes the entered message to be received by the client device.

[1506] Input: The message entered by the user

[1507] Output: Message sent to the client device

[1508] Step 2:

[1509] The device converts the message entered by the user into JSON format and sends a POST request to the server, encoding the text as message data in JSON format.

[1510] Input: The message entered by the user

[1511] Output: POST request containing message data in JSON format

[1512] Step 3:

[1513] The server receives the POST request sent from the client device, analyzes the received data, and extracts the message content.

[1514] Input: A POST request containing message data in JSON format.

[1515] Output: Extracted text message

[1516] Step 4:

[1517] The server then sends the extracted messages to a generative artificial intelligence model for appropriate moderation evaluation, specifically using OpenAI's GPT-3 model to analyze the content of the messages and determine their appropriateness.

[1518] Input: Extracted text message

[1519] Output: Moderation evaluation results and warning messages

[1520] Step 5:

[1521] The generative AI model analyzes the content of received messages and evaluates their appropriateness based on the prompt. For example, it generates a response to a prompt such as, "'There's no point in living anymore.' Could this message contain suicidal thoughts?"

[1522] Input: Extracted text message and prompt

[1523] Output: Suitability assessment and warning messages

[1524] Step 6:

[1525] The server receives the evaluation results from the generative artificial intelligence model, formats them, and sends them back to the client device along with the original message.

[1526] Input: Moderation evaluation results and warning messages

[1527] Output: A response containing the formatted evaluation result and the original message.

[1528] Step 7:

[1529] The terminal receives the response from the server and displays the moderation results to the user, who can then modify the message content as necessary.

[1530] Input: Formatted evaluation results and original message

[1531] Output: Moderation results and prompts shown to the user

[1532] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1533] The present invention relates to a generation system for moderating user-generated data in real time to maintain the integrity of content on online platforms. The system includes a set of means for receiving data posted from a client device, processing the data using a generative artificial intelligence model, and transmitting the processing results to the client device. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system achieves more advanced moderation based on the user's emotional state.

[1534] System configuration

[1535] server:

[1536] The server uses the Flask framework to build a web server and sets up an endpoint that accepts requests from client devices.

[1537] The server receives user-submitted data and evaluates and filters the content using an emotion engine and generative artificial intelligence models.

[1538] The server formats the evaluation and filtering results and sends them back to the client device as a response.

[1539] Device:

[1540] The terminal sends a request containing user-generated data to the server.

[1541] The terminal displays the moderation results received from the server to the user.

[1542] User:

[1543] Users use their terminals to access the online platform, input content to post, and send it.

[1544] Users receive the results of the moderation and can modify their posts as necessary.

[1545] Details of system processing

[1546] Processing flow

[1547] server:

[1548] 1. The server receives user-generated data sent as a POST request from a client device.

[1549] 2. The server retrieves the received data and sends it to the emotion engine, which analyzes the emotions from the user-generated data and provides the results to the generation system.

[1550] 3. The server requests the generative artificial intelligence model to evaluate the content based on the emotion analysis results obtained from the emotion engine.

[1551] 4. The biointelligence model uses this data and sentiment analysis results to moderate the content, for example, easing the rating if it contains positive sentiment and stricter rating if it contains negative sentiment.

[1552] 5. The server formats the evaluation results from the biointelligence model and sends them back to the client device along with the original data.

[1553] Device:

[1554] 1. The terminal converts the data entered by the user into JSON format and sends a POST request to the server.

[1555] 2. The device receives the response sent back from the server and displays it to the user.

[1556] Specific examples

[1557] For example, if a user posts content like this:

[1558] "This game is so fun! But that player said some horrible things."

[1559] 1. The user enters this information into the terminal and presses the send button.

[1560] 2. The device sends the post to the server.

[1561] 3. The server receives the post and requests the emotion engine to analyze the sentiment of the content.

[1562] 4. The sentiment engine recognizes that the post contains negative sentiment (e.g., negative comments).

[1563] 5. The server requests strict moderation of the biointelligence model based on the results from the emotion engine.

[1564] 6. The biointelligence model analyzes the content of the posts and generates evaluation results such as "avoid making negative comments about other players."

[1565] 7. The server receives the evaluation results, formats them together with the original post content, and sends them back to the device.

[1566] 8. The device displays the moderation results to the user and prompts them to correct their post.

[1567] This allows users to recognize when content violates guidelines and correct it, and the emotional engine enables appropriate moderation based on the user's emotional state, helping to provide a healthy digital environment across online platforms.

[1568] The processing flow will be explained below.

[1569] Step 1:

[1570] The user enters content and presses the post button. For example, they might enter, "This game is really fun! But that player said some horrible things."

[1571] Step 2:

[1572] The terminal converts the input content into JSON format, which looks like this:

[1573] json

[1574] {

[1575] "content": "This game is really fun! But that player said some horrible things."

[1576] }

[1577] Step 3:

[1578] The device sends the converted JSON data to the server as an HTTP POST request. The request destination is the endpoint ( / filter) configured in the Flask framework.

[1579] Step 4:

[1580] The server receives a POST request from the client device. In this case, the received data is "This game is really fun! But that player said some horrible things."

[1581] Step 5:

[1582] The server sends the received data to the emotion engine, which analyzes the emotions from the user-generated data and determines, for example, "positive" or "negative" emotions.

[1583] Step 6:

[1584] The emotion engine analyzes the input content and recognizes that it contains negative emotions. For example, the phrase "he said horrible things" is identified as a negative emotion.

[1585] Step 7:

[1586] The server requests strict moderation evaluation from the generative artificial intelligence model based on the analysis results of negative emotions obtained from the emotion engine.

[1587] Step 8:

[1588] A generative AI model will critically evaluate your submission, generating prompts such as:

[1589] Moderate the following content according to the community guidelines, considering that it includes negative sentiment: "This game is really fun! But that player said some horrible things."

[1590] Step 9:

[1591] A generative AI model generates content evaluation results, such as a moderation result like "Please avoid making negative comments about other players."

[1592] Step 10:

[1593] The server receives the moderation results returned by the generative AI model and formats them along with the original content, creating a JSON response like this:

[1594] json

[1595] {

[1596] "original_content": "This game is really fun! But that player said some horrible things.",

[1597] "moderation_result": "Please avoid negative comments about other players."

[1598] }

[1599] Step 11:

[1600] The server sends a formatted JSON response to the client device.

[1601] Step 12:

[1602] The device receives the response from the server and displays the moderation results to the user, for example, feedback such as "Please avoid making negative comments about other players" is displayed on the screen.

[1603] Step 13:

[1604] The user then edits and corrects the content based on the feedback received and resubmits it if necessary.

[1605] Example 2

[1606] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1607] Modern online platforms require the maintenance of the integrity of content generated by users, as a large amount of content is posted in real time. However, manual moderation is time-consuming and costly, and it is difficult to cover all content. To solve this problem, an automated real-time moderation system is needed. Furthermore, advanced moderation that takes user sentiment into account is required, rather than simply moderating content.

[1608] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1609] In this invention, the server includes means for receiving user-generated data from a client device, means for transmitting the generated data to a sentiment analysis engine to perform sentiment analysis, means for requesting data moderation using a generative AI model based on the sentiment analysis result, and means for formatting and transmitting the moderation result to the client device, thereby automatically and efficiently maintaining content integrity in real time and enabling more advanced moderation based on emotional states.

[1610] A "client device" is a terminal device used by a user, and is a device for sending and receiving data to a server via the Internet.

[1611] "User-generated data" refers to content, such as text, images, and video, entered by a user and transmitted through a client device.

[1612] A "sentiment analysis engine" is a software or hardware component for extracting and analyzing emotional information from user-generated data.

[1613] A "generative AI model" is a program or system that uses artificial intelligence techniques to generate, process, and analyze data.

[1614] "Moderation" is the process of evaluating user-generated content and filtering and adjusting it to ensure compliance with community guidelines and other regulations.

[1615] "Formatting" is the process of organizing processing results and data into a specific format to make them easier to read.

[1616] "Community guidelines" refer to rules and standards that govern appropriate behavior and content posted on online platforms.

[1617] "Real-time" means that data processing and moderation occurs immediately, without delay.

[1618] A "POST request" is a type of HTTP request for sending data from a client device to a server.

[1619] An "API endpoint" is a defined server URL that other programs or services can access, providing access to specific functionality or data.

[1620] This invention relates to a system for real-time moderation of data generated by users on online platforms to maintain the integrity of the platform. The system collects data posted by users from their devices, analyzes, evaluates, and filters the data using a sentiment analysis engine and generative AI model, and provides feedback on the evaluation results to users to encourage them to maintain healthy content.

[1621] System configuration

[1622] 1. Server:

[1623] The server uses the Flask framework to build a web server. Flask is a lightweight web framework that allows for fast deployment and simple API design.

[1624] The server receives user-generated data from the client device as a POST request, with the data being transmitted in JSON format.

[1625] The server sends the received data to a sentiment analysis engine, which can be implemented using a commercial API such as IBM Watson Tone Analyzer.

[1626] The server receives the results of the sentiment analysis and requests moderation from a generative AI model based on the results. For example, OpenAI's GPT-4 is used as the generative AI model.

[1627] The server formats the evaluation results from the generative AI model and sends them back to the client device.

[1628] 2. Terminal:

[1629] The terminal converts the data entered by the user into JSON format and sends a POST request to the server.

[1630] The terminal receives the moderation results returned from the server and displays them to the user.

[1631] 3. User:

[1632] A user accesses the online platform using a terminal, inputs the content to be posted, and sends it.

[1633] The user reviews the moderation results and corrects the post if necessary.

[1634] Specific examples

[1635] For example, if a user posts something like this:

[1636] "This game is so fun! But that player said some horrible things."

[1637] 1. The user enters the content of this post into the device and presses the send button.

[1638] 2. The device converts the post content into JSON format and sends it to the server.

[1639] 3. The server receives the post and requests a sentiment analysis engine to analyze it, which analyzes both positive and negative sentiment.

[1640] 4. The server requests moderation from the generative AI model based on the results of the sentiment analysis engine. The generative AI model evaluates the content of the post while taking into account the results of the sentiment analysis.

[1641] 5. The server formats the evaluation results from the generated AI model and sends them back to the device.

[1642] 6. The device receives the moderation results from the server and displays them to the user, for example, feedback such as "Please avoid making negative comments about other players."

[1643] 7. Users review the feedback and modify their posts as needed. For example, they might post only "This game is really fun!"

[1644] Prompt Sentence Examples

[1645] A concrete example of a prompt for a generative AI model might be something like this:

[1646] User post: "This game is so fun! But that player said some horrible things."

[1647] Sentiment analysis results: "Positive sentiment: 0.7, Negative sentiment: 0.3"

[1648] This allows the generative AI model to perform appropriate moderation based on user comments.

[1649] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1650] Step 1:

[1651] The server receives a POST request from a client device.

[1652] Input: User-generated data (in text format) sent from the client device.

[1653] What it does: The server uses the Flask framework to set up an endpoint at / moderate to receive POST requests.

[1654] Output: The received user-generated data is persisted in the server.

[1655] Step 2:

[1656] The server sends the received data to the sentiment analysis engine.

[1657] Input: User-generated data received in step 1.

[1658] How it works: A server sends user-generated data to an API endpoint of a sentiment analysis engine (e.g., IBM Watson Tone Analyzer).

[1659] Output: The sentiment analysis result from the sentiment analysis engine (e.g., positive sentiment: 0.7, negative sentiment: 0.3) is returned.

[1660] Step 3:

[1661] The server requests moderation from the generative AI model based on the results of emotion analysis.

[1662] Input: Sentiment analysis results obtained in step 2 and the original user-generated data.

[1663] How it works: The server creates an input prompt for a generative AI model (e.g., OpenAI GPT-4) and sends it to the model.

[1664] Output: Moderation results from the generative AI model (e.g., "Please avoid negative comments to other players") are returned.

[1665] Step 4:

[1666] The server formats and transmits the moderation results from the generative AI model to the client device.

[1667] Input: Moderation results obtained in step 3.

[1668] How it works: The server formats the moderation results into a specific format, such as JSON.

[1669] Output: Formatted moderation results are sent to the client device.

[1670] Step 5:

[1671] The terminal receives the moderation results returned from the server and displays them to the user.

[1672] Input: Moderation results sent by the server in step 4.

[1673] Operation: The terminal receives the response from the server and displays the content to the user, for example, as a pop-up message or a dialog box.

[1674] Output: A display screen where users can see the moderation results.

[1675] Step 6:

[1676] The user reviews the moderation results and corrects the post if necessary.

[1677] Input: Moderation results displayed in Step 5.

[1678] Action: The user reviews the feedback message and follows the instructions to edit or modify the post. For example, "Delete negative comments about other players."

[1679] Output: A new, revised post is generated, ready to be sent again.

[1680] (Application example 2)

[1681] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1682] In conventional content distribution services, if content posted by users contains inappropriate content, manual moderation is required, which is inefficient and poses a risk to the integrity of the platform. In particular, real-time moderation is difficult, which poses the risk of inappropriate content spreading instantly.

[1683] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving data generated by a user from a client device, means for processing the generated data using a generative artificial intelligence model, means for performing sentiment analysis on the user-generated data, and means for transmitting the processing results to the client device. This enables sound moderation in real time in content distribution services, preventing the spread of inappropriate content and improving the user experience.

[1684] A "client device" is a device that transmits user-generated data to a server, and examples include smartphones and personal computers.

[1685] "Generated Data" means user-generated content, whether in the form of text, images, video, or other content.

[1686] A "generative artificial intelligence model" is an artificial intelligence algorithm used to process generated data and evaluate or filter its content.

[1687] "Means for processing" refers to the function of analyzing and evaluating the generated data using a generative artificial intelligence model.

[1688] "Sentiment analysis" is the process of recognizing the emotional state of a user from their generated data and determining whether it is positive or negative.

[1689] "Processing results" refers to information generated as a result of evaluation and filtering performed on generated data.

[1690] The "means for transmitting" refers to a function for returning the processing results to the client device.

[1691] "Community guidelines" refer to the rules and standards that users must follow on an online platform.

[1692] The present invention is a generative system for moderating user-generated data from client devices in real time to maintain the health of online platforms. The system combines a sentiment analysis engine and a generative artificial intelligence model to achieve advanced moderation based on the user's emotional state.

[1693] System configuration

[1694] server

[1695] The server first sets up an endpoint to accept requests from client devices. This uses the Flask framework. The server receives data posted by users and sends it to an emotion engine for sentiment analysis. This makes it possible to recognize the user's emotional state. Next, the server requests a content evaluation from a generative AI model based on the results of the sentiment analysis. The generative AI model then performs moderation based on the results of the sentiment analysis.

[1696] Terminal

[1697] The device converts user-generated data into JSON format and sends a POST request to the server. It receives the response from the server and displays it to the user, providing feedback on the moderation results. This allows users to check whether the content violates the platform's guidelines and make corrections if necessary.

[1698] User

[1699] Users access the online platform using their devices, input and submit content to be posted, and then check the moderation results from the server on their devices and modify the content according to the instructions.

[1700] Hardware and software used

[1701] Hardware:

[1702] Client devices such as smartphones and PCs

[1703] software:

[1704] On the server side, the Flask framework

[1705] Sentiment Engine API for sentiment analysis

[1706] Content rating system using live artificial intelligence models

[1707] React Native on the client side

[1708] Specific examples

[1709] For example, consider the case where a user attempts to post the following content:

[1710] "This video is great, but the comments section is awful."

[1711] 1. The user enters this information into the terminal and presses the send button.

[1712] 2. The device sends the post to the server.

[1713] 3. The server receives the post and runs it through a sentiment analysis engine.

[1714] 4. The sentiment analysis engine determines that the comment contains negative comments.

[1715] 5. The server requests moderation from the live AI model based on the results from the sentiment analysis engine.

[1716] 6. A raw AI model detects the phrase "The comments section is awful" and rates it as inappropriate.

[1717] 7. The server returns the evaluation results to the device.

[1718] 8. The device displays feedback to the user, prompting them to revise their statement, "The comments section is terrible."

[1719] Prompt Sentence Examples

[1720] "Please send the following text to the Sentiment Engine API for sentiment analysis:

[1721] A user wrote: "This video is great, but the comments section is awful."

[1722] Emotion Engine API endpoint: https: / / emotion-api.example.com / analyze

[1723] "

[1724] To generate smart moderation feedback, use prompts like these:

[1725] The post read: "The comments section is awful."

[1726] Generative AI model API endpoint: https: / / ai-moderation-api.example.com / moderate

[1727]

[1728] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1729] Step 1:

[1730] The user types content (e.g., "This video is great, but the comments section is terrible") into the device and hits send.

[1731] Input: User generated content text

[1732] Output: POST request sent from the terminal to the server

[1733] Specific behavior: The user enters content into the text input area on the device and clicks the send button, which sends a request in JSON format to the server.

[1734] Step 2:

[1735] The device converts the user-generated data and sends a POST request to the server.

[1736] Input: Content text entered by the user

[1737] Output: POST request sent to the server

[1738] Specific operation: The device detects that the send button has been pressed, converts the entered text into JSON format, and sends an HTTP POST request to the server endpoint.

[1739] Step 3:

[1740] The server receives the posted content and sends the text data to the sentiment analysis engine.

[1741] Input: Content text in JSON format

[1742] Output: Request to the sentiment analysis engine

[1743] Specific operation: The server receives the data sent from the client and sends a sentiment analysis request to the Emotion Engine API, including the content text in the body of the POST request and sending it to the API endpoint.

[1744] Step 4:

[1745] The sentiment analysis engine analyzes the text data and returns the sentiment analysis results.

[1746] Input: Content text

[1747] Output: Sentiment analysis result (e.g. positive, negative, neutral)

[1748] What it does: The sentiment analysis engine analyzes the received text data, evaluates the sentiment of each word or phrase, classifies the emotional state, and sends the results back to the server in JSON format.

[1749] Step 5:

[1750] The server receives the sentiment analysis results and requests the generative artificial intelligence model to evaluate the content.

[1751] Input: Sentiment analysis results

[1752] Output: A request to the generative AI model

[1753] What it does: The server receives the analysis results from the sentiment engine and sends a moderation request to the generative AI model API, which includes the sentiment analysis results and the original text.

[1754] Step 6:

[1755] The generative AI model performs moderation based on content and sentiment analysis results and generates evaluation results.

[1756] Input: Content text and sentiment analysis results

[1757] Output: Moderation evaluation result (e.g., classifying "The comments section is awful" as inappropriate)

[1758] Specific operation: The generative AI model analyzes the content text and sentiment analysis results, performs moderation based on the policy, filters and evaluates the content, and generates results, which are then sent back to the server.

[1759] Step 7:

[1760] The server receives the moderation evaluation results from the generative AI model, formats them, and sends them back to the device.

[1761] Input: Moderation evaluation results

[1762] Output: Response to the terminal

[1763] Specific operation: The server receives the evaluation results from the generative AI model, formats them to provide appropriate feedback to the user, and then sends the formatted results back to the device in JSON format.

[1764] Step 8:

[1765] The device will display the moderation results to the user and prompt them to correct the content if necessary.

[1766] Input: Moderation evaluation results

[1767] Output: Display feedback to the user

[1768] Specific operation: The device analyzes the JSON data returned from the server and displays the evaluation results in the user interface. The device prompts the user to make the necessary corrections by showing the specific areas and content of corrections.

[1769] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1770] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1771] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1772] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1773] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1774] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1775] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1776] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1777] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1778] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1779] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1780] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1781] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1782] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1783] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1784] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1785] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1786] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1787] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1788] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1789] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1790] The following is further disclosed regarding the above embodiment.

[1791] (Claim 1)

[1792] 1. A generation system for processing generated data, comprising:

[1793] means for receiving user-generated data from a client device;

[1794] means for processing the generated data using a generative artificial intelligence model;

[1795] means for transmitting the processing result to a client device;

[1796] A system including:

[1797] (Claim 2)

[1798] 2. The system of claim 1, wherein the generative artificial intelligence model filters the generated data to ensure compliance with community guidelines.

[1799] (Claim 3)

[1800] 10. The system of claim 1, wherein the generative artificial intelligence model performs processing on the generated data in real time.

[1801] "Example 1"

[1802] (Claim 1)

[1803] means for receiving user-generated data from a client device;

[1804] means for processing the generated data using a generative artificial intelligence model;

[1805] means for transmitting the processing result to a client device;

[1806] a communication means for receiving a request from a client device;

[1807] data formatting means for transmitting data to the generative artificial intelligence model;

[1808] a result formatting means for formatting and returning the evaluation results from the generative artificial intelligence model;

[1809] A system including:

[1810] (Claim 2)

[1811] 2. The system of claim 1, wherein the generative artificial intelligence model filters the generated data to ensure compliance with community guidelines.

[1812] (Claim 3)

[1813] 10. The system of claim 1, wherein the generative artificial intelligence model performs processing on the generated data in real time.

[1814] "Application Example 1"

[1815] (Claim 1)

[1816] 1. A generation system for processing generated data, comprising:

[1817] means for receiving user-generated data from a client device;

[1818] means for processing the generated data using a generative artificial intelligence model;

[1819] means for transmitting the processing result to a client device;

[1820] a means of moderating user-generated messages in real time; and

[1821] A means of detecting inappropriate content and providing warnings or appropriate response information;

[1822] A system including:

[1823] (Claim 2)

[1824] 10. The system of claim 1, wherein the generative artificial intelligence model filters the generated data to comply with community guidelines and issues a warning against inappropriate messages.

[1825] (Claim 3)

[1826] The system of claim 1, wherein the generative artificial intelligence model processes the generated data in real time and provides appropriate supporting information as needed.

[1827] "Example 2: Combining Emotion Engines"

[1828] (Claim 1)

[1829] means for receiving user-generated data from a client device;

[1830] means for transmitting the generated data to a sentiment analysis engine to perform sentiment analysis;

[1831] A means for requesting data moderation using a generative AI model based on the emotion analysis results;

[1832] means for formatting and transmitting the moderation results to a client device;

[1833] A system including:

[1834] (Claim 2)

[1835] 10. The system of claim 1, wherein the generative AI model filters the generated data to comply with community guidelines.

[1836] (Claim 3)

[1837] 10. The system of claim 1, wherein the generative AI model performs processing on the generated data in real time.

[1838] "Application example 2 when combining emotion engines"

[1839] (Claim 1)

[1840] 1. A generation system for processing generated data, comprising:

[1841] means for receiving user-generated data from a client device;

[1842] means for processing the generated data using a generative artificial intelligence model;

[1843] a means for performing sentiment analysis on user-generated data;

[1844] means for transmitting the processing result to a client device;

[1845] A system including:

[1846] (Claim 2)

[1847] 2. The system of claim 1, wherein the generative artificial intelligence model filters the generated data to ensure compliance with community guidelines.

[1848] (Claim 3)

[1849] 10. The system of claim 1, wherein the generative artificial intelligence model performs processing on the generated data in real time. [Explanation of symbols]

[1850] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. 1. A generation system for processing generated data, comprising: means for receiving user-generated data from a client device; means for processing the generated data using a generative artificial intelligence model; means for transmitting the processing result to a client device; A system including:

2. The system of claim 1 , wherein the generative artificial intelligence model filters the generated data to conform to community guidelines.

3. The system of claim 1 , wherein the generative artificial intelligence model performs real-time processing on the generated data.

Images