System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving posting data; means for analyzing the received posting data and comparing the posting data with past burning cases; means for evaluating a burning risk based on an analysis result; means for transmitting a warning to a user when the burning risk is high; means for generating substitute information for reducing the burning risk; and means for transmitting the warning and the substitute information to a user terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With the spread of the Internet and social networking services (SNS), the number of inappropriate posts by users has increased, resulting in numerous cases of online outrage. Such online outrage can lead to loss of social credibility and financial loss, with the risk being particularly pronounced among young people and businesses. Conventional methods often involve taking countermeasures after an online outrage has occurred following a post, which is insufficient as a fundamental solution. The present invention aims to prevent these risks by assessing the risk of online outrage before posting and preventing it in advance. [Means for solving the problem]

[0005] The system according to the present invention includes a means for receiving posted data, a means for analyzing the received posted data and comparing it with past cases of flaming, a means for assessing the risk of flaming based on the analysis results, a means for sending a warning to a user when the risk of flaming is high, a means for generating alternative information to reduce the risk of flaming, and a means for sending the warning and alternative information to a user terminal. This allows users to recognize the risk of flaming before posting and select a safe alternative, thereby preventing social and property risks before they occur.

[0006] "Posted data" refers to content such as text, images, and videos that a user intends to publish on an SNS.

[0007] "Analysis" refers to the act of analyzing the content of received posted data using algorithms such as natural language processing, image recognition, and voice recognition.

[0008] "Flame risk" is an indicator that shows the degree of possibility that posted data will be deemed socially inappropriate.

[0009] A "warning" is a message that notifies the user of the risk of a controversy if the risk is assessed to be high.

[0010] "Alternative information" is a modified version of the original post or recommended safe content that is generated to reduce the risk of flame wars.

[0011] The "receiving means" is a function that allows the server to receive posted data sent from a user terminal.

[0012] The "comparison method" is a function that compares the analyzed post data with a database of past flame war cases.

[0013] The "means of evaluation" is a function that scores the risk of posting data becoming a hot topic based on the analysis results and determines the degree of risk.

[0014] "Means of sending" is a function that notifies users of warnings and alternative information regarding the assessed risk of a social media outburst.

[0015] A "user terminal" is a device (e.g., smartphone, PC) that a user uses to use an SNS application. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a system for analyzing posted data on a social networking site in advance and assessing the risk of a social networking site becoming a hot topic. Specific embodiments of the system according to the present invention will be described in detail below.

[0038] Server-side processing

[0039] 1. Receiving submitted data

[0040] The server receives the posted data (content such as text, images, and videos) sent from the user's device. For example, it retrieves the data using a POST request using a REST API.

[0041] 2. Data Analysis

[0042] The AI ​​analysis module on the server analyzes the data in the following steps:

[0043] Sentence analysis:

[0044] Use a natural language processing (NLP) library to tokenize the text and perform morphological analysis.

[0045] Extract relevant keywords and emotional expressions from the text.

[0046] Image analysis:

[0047] Recognize objects in an image using an object detection model (e.g., YOLO or SSD).

[0048] The text in the image is extracted using OCR technology and then further analyzed.

[0049] Video Analysis:

[0050] The video is divided into frames, and each frame is processed in the same way as image analysis.

[0051] The audio portion of the video is converted into text using automatic speech recognition (ASR) technology, and text analysis is performed.

[0052] 3. Comparison with past cases of online outrage

[0053] The server compares the analysis results with a database of existing cases of online outrages and evaluates the risk of online outrage based on the degree of similarity. For example, it searches the database of past cases of online outrages and calculates a similarity score.

[0054] 4. Risk Assessment

[0055] The server scores the risk of a controversy based on the similarity score and the extracted sentiment and keywords. The server then determines the degree of risk based on the assessed risk score. For example, a risk score of 90 or above is high risk, 70-89 is medium risk, and anything below that is low risk.

[0056] 5. Risk Notification and Alternative Generation

[0057] The server sends a warning to users if the risk score is high. To support safe posting, the server uses a derived text generation model (e.g., GPT-3) to generate alternative text and image / video revision suggestions with less risk and sends them to the user's device.

[0058] Terminal side processing

[0059] 1. Create submission data

[0060] Users use SNS applications to create posts, including text they type and images and videos they upload.

[0061] 2. Submitting data

[0062] When the user presses the post button, the terminal sends the created post data to the server, for example, using a REST API POST request.

[0063] 3. Receive risk warnings

[0064] The device receives risk warnings sent from the server and notifies the user, including the risk assessment score and a specific warning message.

[0065] 4. Displaying alternatives

[0066] The terminal displays the alternatives sent from the server for reducing the risk on the user interface, and the user checks them and decides whether to modify the original post.

[0067] User operations

[0068] 1. Enter your post content

[0069] A user creates a post on a social networking app, for example, by typing a sentence such as, "Today is the worst day. Everything is going wrong."

[0070] 2. Risk confirmation

[0071] The user checks the risk assessment notification and warning message from the server on their device. For example, a warning message such as "This post contains negative sentiment and is at high risk of causing a backlash" may be displayed.

[0072] 3. Selection of alternatives

[0073] The user checks the alternatives provided by the server and decides whether to revise the post based on the alternatives or to post the original post as is. For example, an alternative suggestion might be "Today didn't go well, but I'll try my best tomorrow!"

[0074] 4. Confirmation of submission

[0075] The user can then modify the content of their post based on the displayed alternatives and finally confirm the post. The modified content is then sent back to the server and posted to the SNS.

[0076] In this way, the system of the present invention evaluates the risk of a controversy before posting, helping users to use SNS safely.

[0077] The processing flow will be explained below.

[0078] Server-side processing

[0079] Step 1:

[0080] The server receives post data (text, images, videos) from the user's device. When the user presses the post button, a POST request is sent from the device. The server receives this request using the REST API.

[0081] Step 2:

[0082] The server preprocesses the submitted data it receives, for example tokenizing text data and storing image and video data in specific directories.

[0083] Step 3:

[0084] The server's AI analysis module analyzes the posted data.

[0085] Text analysis: Using a natural language processing (NLP) library, posts are tokenized and semantic and sentiment analysis is performed.

[0086] Image analysis: Recognize objects in images using object detection models (e.g., YOLO and SSD), and extract text from images using hooked OCR technology.

[0087] Video analysis: Videos are divided into frames, and each frame is analyzed in the same way as image analysis. Audio in the video is converted into text using automatic speech recognition (ASR), and related sentence analysis is performed.

[0088] Step 4:

[0089] The server compares the analysis results with a database of past flaming cases, performs a similarity search, evaluates the degree to which the posted data is similar to past flaming cases, and calculates a similarity score.

[0090] Step 5:

[0091] The server evaluates the risk of a controversy based on the analysis results and the similarity score. For example, it calculates a risk score, determining that a score of 90 or higher is high risk, 70-89 is medium risk, and 70 or lower is low risk.

[0092] Step 6:

[0093] If the server determines that a post is at high risk of causing an uproar, it generates a risk warning and alternatives. It uses a derived sentence generation model (e.g., GPT-3) to generate alternative sentences with less risk. If necessary, it also generates suggested revisions for images and videos.

[0094] Step 7:

[0095] The server sends risk warnings and alternative solutions to the user's device, and keeps a log containing details of any errors that occur.

[0096] Terminal side processing

[0097] Step 1:

[0098] Users create posts through SNS applications, and can input text and upload images and videos.

[0099] Step 2:

[0100] When the user presses the submit button, the device sends the submission data to the server, for example, using a REST API POST request.

[0101] Step 3:

[0102] The device waits for a response from the server, and if the server returns a risk warning and alternatives, it receives them.

[0103] Step 4:

[0104] The device displays the risk warning and alternatives received from the server to the user, for example, by a pop-up message or a notification box.

[0105] Step 5:

[0106] Users review the risk warnings and alternative suggestions and decide whether to revise their original post. If revisions are necessary, they reconstruct the content using the alternative sentences and suggested revisions as a reference.

[0107] Step 6:

[0108] When the user decides on the final post and presses the post button again, the terminal resends the corrected post data to the server.

[0109] User Action

[0110] Step 1:

[0111] A user types or uploads a post to a social networking application, for example, typing the sentence "Today is the worst day. Everything is going wrong."

[0112] Step 2:

[0113] When the user presses the submit button, the data is sent to the server.

[0114] Step 3:

[0115] The user then checks the risk warning and alternatives from the server. For example, the user may be presented with a warning message such as "This post contains negative emotions and is at high risk of causing a backlash," along with an alternative sentence such as "Today didn't go well, but I'll try my best tomorrow!"

[0116] Step 4:

[0117] The user decides whether to revise the post based on the alternative suggestions, and if so, checks the new post.

[0118] Step 5:

[0119] Once the user has finalized the content of the post and pressed the post button again, the final post data will be posted on the SNS.

[0120] Example 1

[0121] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0122] Posting on social media is an important way to share information with many people, but if inappropriate content is included, it can lead to outrage and excessive criticism. In addition, since there are limited ways for users to know about such risks in advance, they may be cautious about using social media. Therefore, a system is needed that evaluates the risk of outrage before posting and suggests safe ways to post.

[0123] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0124] In this invention, the server includes means for receiving post data from a user terminal, means for analyzing the received post data using natural language processing, object recognition, and voice recognition, means for comparing the received post data with past flaming cases based on keywords and emotional expressions extracted by the analysis, means for scoring and assessing the risk of flaming based on the comparison results, means for sending a warning to the user if the risk of flaming is high, means for generating alternative information to reduce the risk of flaming, and means for sending the warning and alternative information to the user terminal. This allows users to assess the risk of flaming before posting and learn safe posting methods.

[0125] A "user terminal" is a device operated by a user, and includes a smartphone, tablet, PC, etc.

[0126] "Posted data" refers to content such as text, images, and videos that users share on social media.

[0127] "Server" refers to a high-performance computer system installed at a remote location that analyzes and processes data received from user devices.

[0128] "Natural language processing" is a technology for processing human language using computers, and includes text tokenization, morphological analysis, sentiment analysis, etc.

[0129] "Object recognition" refers to the technology of identifying objects in images and classifying them.

[0130] "Speech recognition" refers to the technology of converting voice data into text.

[0131] "Keywords" are words or phrases that are particularly important in a sentence or piece of content.

[0132] "Sentiment expression" refers to elements in text that indicate sentiment (positive, negative, neutral, etc.) analyzed by natural language processing.

[0133] "Flame war cases" include data that refer to large-scale negative reactions caused by inappropriate posts on social media in the past.

[0134] The "similarity score" is a numerical value that indicates the similarity between the target data and past flame war case data, and is used for risk assessment.

[0135] "Scoring" refers to a method of quantifying risk or performance based on specific criteria.

[0136] "Warning" refers to a caution message to inform the user of a high risk.

[0137] "Alternative information" refers to safe posts and suggested revisions to reduce the risk of outrage.

[0138] MODE FOR CARRYING OUT THE INVENTION

[0139] The present invention relates to a system for analyzing data posted to a social networking site in advance and assessing the risk of a social networking site becoming a hot topic. Specific embodiments of the present invention will be described in detail below.

[0140] Server Processing

[0141] The server receives the posted data (text, images, videos, and other content) sent from the user's device. Specifically, it uses a mechanism to receive data via a POST request via the REST API.

[0142] The server analyzes the received posted data using the following methods. First, it uses a natural language processing (NLP) library (e.g., spaCy) to tokenize the text data and extract emotional expressions and keywords. Next, it uses an object recognition model (e.g., YOLO) to recognize objects within the image. It also uses OCR technology (e.g., Tesseract OCR) to extract text from the image and further analyze it. It then divides the video data into frames, each of which is processed in the same way as image analysis. It then uses automatic speech recognition (ASR) technology to convert the audio portion of the video into text, which is then analyzed.

[0143] The server compares the analysis results with a database of past flaming cases. It calculates a similarity score from the database and evaluates the risk. For this purpose, it uses a scoring method; for example, a similarity score of 90 or higher with past cases is rated as high risk, 70-89 as medium risk, and anything below that is rated as low risk.

[0144] If the server determines that there is a high risk of outrage, it sends a warning to the user. It also uses a generative AI model (e.g., GPT-3) to generate alternative information (suggested revisions to text, images, and videos) with less risk and sends them to the user's device.

[0145] Terminal handling

[0146] Users use SNS applications to create posts, and the text they enter and the images and videos they upload become the post data.

[0147] When the user presses the submit button, the device sends the created post data to the server using a REST API POST request.

[0148] The device receives the risk warning sent from the server and notifies the user, displaying the risk assessment score and a specific warning message.

[0149] The terminal displays the alternatives for reducing the risk sent from the server to the user, who then checks the alternatives and decides whether to revise the content of the post and repost it.

[0150] User operations

[0151] A user inputs content to be posted on the SNS application to create post data. For example, the user inputs a sentence such as "Today is the worst day. Everything is going wrong."

[0152] The user checks the risk assessment notification and warning message from the server on their device. For example, a warning message such as "This post contains negative emotions and is at high risk of causing a backlash" may be displayed.

[0153] The user checks the alternatives provided by the server and modifies the content of the post based on them. For example, an alternative suggestion such as "Today didn't go well, but I'll try my best tomorrow!" may be presented.

[0154] Specific examples

[0155] Example prompt sentence:

[0156] "This sentence contains negative content. Please consider the following alternatives:

[0157] 1. Things didn't go well today, but I'll try my best tomorrow!

[0158] 2. Sometimes you have bad days, but that's life.

[0159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0160] Step 1: Receiving posted data

[0161] The server receives posted data from user devices. Input includes content such as text, images, and videos. Specifically, data is received via a POST request via the REST API. The output is the received posted data itself.

[0162] Specific behavior:

[0163] When a user presses the "Post" button on their device, the device sends a POST request to the server, and text, image, and video data are sent to the server.

[0164] Step 2: Text analysis

[0165] The server analyzes the received text data using a natural language processing (NLP) library. The input is the received text data, and the output is the analysis results such as keywords and emotional expressions.

[0166] Specific behavior:

[0167] The server uses the spaCy library to tokenize the text and extract important keywords and sentiments. For example, from the sentence "Today is the worst day. Everything is going wrong," it extracts keywords like "worst" and "going wrong."

[0168] Step 3: Image analysis

[0169] The server analyzes the received image data using an object recognition model (e.g., YOLO). The input is the received image data, and the output is the analysis results of objects and text in the image.

[0170] Specific behavior:

[0171] The server uses the YOLO model to identify people and objects in the image and extract their location information, and Tesseract OCR to extract text from the image and analyze important keywords such as "warning" and "caution."

[0172] Step 4: Video Analysis

[0173] The server divides the received video data into frames and processes each frame in the same way as image analysis. The input is the received video data, and the output is the image analysis results for each frame and the speech-to-text results.

[0174] Specific behavior:

[0175] The server uses OpenCV to split the video into frames and analyzes each frame with the YOLO model. At the same time, the audio portion is converted to text using the Google Speech-to-Text API and subjected to NLP processing.

[0176] Step 5: Compare with past flame wars

[0177] The server compares the analysis results with a database of past flaming cases. The input is the analyzed keywords and emotional expressions, and the output is a similarity score.

[0178] Specific behavior:

[0179] The server uses TfidfVectorizer to calculate the similarity score between the analysis results and past flaming cases. For example, if the keywords "worst" and "doesn't work" match many of the past flaming cases, a high similarity score will be calculated.

[0180] Step 6: Risk assessment

[0181] The server scores the flame risk based on the similarity score. The input is the similarity score, and the output is the estimated flame risk score.

[0182] Specific behavior:

[0183] The server applies a scoring algorithm to calculate a risk score. For example, a similarity score of 90 or higher is considered "high risk," and a score between 70 and 89 is considered "medium risk."

[0184] Step 7: Sending an alert

[0185] The server sends a warning to the user if the risk score is high. The input is the assessed risk score and the output is the warning message.

[0186] Specific behavior:

[0187] The server generates a warning message based on the risk score and sends it to the user's device. For example, a message might say, "This post contains negative emotions and is at high risk of causing a backlash."

[0188] Step 8: Generate alternative information

[0189] To reduce the risk of outrage, the server generates alternative information using a generative AI model (e.g., GPT-3). The input is the initial post data, and the output is the generated alternative sentences or suggested revisions.

[0190] Specific behavior:

[0191] If the initial post is "Today is the worst day. Everything is going wrong," GPT-3 will be used to generate an alternative post that says, "Some things didn't go well today, but I'll try my best tomorrow!"

[0192] Step 9: Notification of warnings and alternatives

[0193] The terminal receives the warning and alternatives sent from the server and notifies the user of them. The input is the warning message and alternatives sent from the server, and the output is the notified message.

[0194] Specific behavior:

[0195] The device will use notifications to display warnings and alternatives to the user, including a risk assessment score and specific warning messages.

[0196] Step 10: Edit and finalize your submission

[0197] The user modifies the posted content by referring to the alternatives and finally confirms it. The input is the alternatives and the initial posted data, and the output is the modified posted data.

[0198] Specific behavior:

[0199] The user then modifies the content of the post based on the alternative suggestions and clicks the "Post" button again to send the modified post to the server, where it is evaluated again and posted on the SNS.

[0200] (Application example 1)

[0201] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0202] Systems that can detect the risk of unexpected outrage or fraudulent use and enable users to take appropriate action are needed not only for data posted on social media but also for electronic payment services. However, current systems have limited scope for risk assessment, and do not adequately deter fraudulent use or prevent outrage. In particular, analysis and evaluation of transaction data and messages in electronic payment services is insufficient, potentially exposing users to the risk of fraud and fraudulent use. There is a need to solve these problems and provide a safe and secure user environment.

[0203] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0204] In this invention, the server includes means for receiving posted data, means for analyzing the received posted data and comparing it with past cases of online outrage, means for assessing the risk of online outrage based on the analysis results, means for sending a warning to the user if the risk of online outrage is high, means for generating alternative information to reduce the risk of online outrage, means for sending the warning and alternative information to the user terminal, means for receiving transaction data and messages and assessing the risk of fraud, means for sending a warning to the user if the risk of fraud is high, and means for generating an alternative transaction plan to reduce the risk of fraud. This makes it possible to comprehensively assess and address the risk of online outrage and fraud for SNS posted data and electronic payment transactions.

[0205] "Posted data" refers to various content such as text, images, and videos that users post on platforms such as social networking sites.

[0206] The "receiving means" refers to the technical means for obtaining data sent from a user terminal via the Internet.

[0207] The "analyzing means" is a means for analyzing the content and characteristics of received data using a specific algorithm or model.

[0208] "Flame cases" are records of past events that have caused a large-scale negative reaction on social media or online platforms.

[0209] "Comparative means" refers to a technical means of comparing the analysis results with past cases to evaluate similarities and commonalities.

[0210] "Means of evaluation" refers to technical means for determining the level of risk numerically or in categories based on the results of analysis and comparison.

[0211] "Means for sending warnings" refers to technical means for sending notifications to users to inform them of risks.

[0212] "Means for generating alternative information" refers to technical means that suggest alternative text, images, or videos to reduce risk.

[0213] A "user terminal" is a device such as a computer or smartphone that is directly used by a user.

[0214] "Transaction data" refers to specific transaction information such as buying and selling and remittances in electronic payment services.

[0215] "Fraud risk" is the possibility of loss due to illegal activities such as fraud or unauthorized access.

[0216] An "alternative transaction proposal" is an alternative proposed transaction content or method to reduce the risk of fraudulent use.

[0217] System configuration and program overview

[0218] The system for realizing the present invention comprises the following main hardware and software components.

[0219] Hardware:

[0220] Server: A computer system that performs data analysis and risk assessment.

[0221] User terminal: A user device such as a smartphone or personal computer.

[0222] software:

[0223] NLP library: A library for natural language processing. Example: SpaCy.

[0224] Image analysis models: Models that process and analyze image data. Examples: YOLO, SSD.

[0225] Video analysis technology: Technology that analyzes video data frame by frame. Example: OpenCV.

[0226] Generative AI model: A natural language generation model for generating alternatives. Example: GPT-3.

[0227] Program processing and behavior

[0228] In this system, the following processing is performed:

[0229] 1. Receiving posted data:

[0230] The server receives data sent from the user's device via a REST API, including text, images, and videos.

[0231] 2. Data Analysis:

[0232] Text analysis: The server uses NLP libraries to tokenize and morphologically analyze the text data, as well as sentiment analysis and keyword extraction.

[0233] Image Analysis: The server uses YOLO and SSD models to recognize objects in the image data and analyze it. It also uses OCR technology to extract text from the image.

[0234] Video analysis: The server divides the video data into frames and processes each frame in the same way as image analysis. The audio in the video is converted into text using ASR technology, and then the text is analyzed.

[0235] 3. Comparison with past scandals:

[0236] The server compares the analysis results with a database of past scandals and calculates a similarity score. It also compares with a database of fraud cases.

[0237] 4. Risk Assessment:

[0238] The server scores the risk of online outrage and abuse based on the similarity score, sentiment, and keywords. If the risk score is high, a warning is generated for the user.

[0239] 5. Risk Notification and Alternative Generation:

[0240] If the risk score is high, the server sends a risk warning to the user's device, and at the same time, it uses a generative AI model to generate alternatives to reduce the risk (such as safe trading suggestions or correction suggestions) and provides them to the user.

[0241] Specific examples

[0242] For example, consider the case where a user types "Today is the worst day. Everything is going wrong" in a social media post and then types "Is it okay to buy from this site?" as a transaction message.

[0243] Example prompt sentence:

[0244] The post read: "Today is the worst day. Everything is going wrong."

[0245] Transaction Comments: "Is it safe to buy from this site?"

[0246] The server analyzes this data, performs sentiment analysis and keyword extraction, and if it determines that a transaction is likely to be a scam or a controversy, it sends a warning to the user and offers alternatives such as "Today didn't go well, but I'll try again tomorrow!" or "This transaction is likely to be a scam, so we recommend purchasing from the official online store."

[0247] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0248] Step 1:

[0249] Input: Posted data and transaction data (text, images, videos) from user devices

[0250] Operation: The user device acquires the posting data and transaction data and sends them to the server. Specifically, the user enters text, images, or videos on the posting screen of an SNS or electronic payment app, and presses the send button to send the data.

[0251] Output: The server receives the data sent from the user terminal.

[0252] Step 2:

[0253] Input: Posting data and transaction data collected on the server

[0254] Operation: The server analyzes the received data. For text data, tokenization and morphological analysis are performed using an NLP library (e.g., SpaCy) to extract emotions and keywords. For image data, object detection is performed using YOLO or SSD models, and text is extracted using OCR technology. For video data, OpenCV is used to split the data into frames and perform the same processing as image analysis. The audio portion is also converted to text using ASR technology.

[0255] Output: Analysis results (emotion score, keywords, object recognition results, text extraction results)

[0256] Step 3:

[0257] Input: Analysis results

[0258] How it works: The server compares the analysis results with a database of past scandals and fraud cases. Specifically, it calculates a similarity score and evaluates how closely the results relate to past cases.

[0259] Output: Similarity score

[0260] Step 4:

[0261] Input: Similarity scores and analysis results

[0262] How it works: The server scores the risk of flame wars and abuse based on the similarity score, sentiment, and keywords. For example, if the similarity score exceeds a certain threshold, it is determined to be high risk.

[0263] Output: Risk score (flaming risk score, fraud risk score)

[0264] Step 5:

[0265] Input: Risk Score

[0266] How it works: If the risk score is high, the server generates a warning message for the user. It also uses a generative AI model (e.g., GPT-3) to generate alternatives to reduce the risk (such as suggestions for modifying the post or safe transactions). For example, if a post is overly negative, it suggests modifying it to a more positive one.

[0267] Output: Warning messages and alternatives

[0268] Step 6:

[0269] Input: Warning messages and alternatives

[0270] Operation: The server sends the generated warning message and alternatives to the user terminal, and the user terminal notifies the user of the received warning message and alternatives.

[0271] Output: A warning message and alternatives that are displayed to the user

[0272] Step 7:

[0273] Input: User confirms alternatives

[0274] How it works: The user reviews the warning message and alternatives they receive and decides whether to modify their posting or transaction. They can also submit the modifications to the server again.

[0275] Output: Modified submission and transaction data, or the original data

[0276] Through these steps, the system can assess in advance the risk of online outrage and fraudulent use of social media posting data and electronic transaction data, and provide users with appropriate warnings and alternatives.

[0277] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0278] The present invention relates to a system that analyzes posting data on social networking sites in advance and assesses the risk of a social networking site becoming a hot topic. By incorporating an emotion recognition engine, the present invention also analyzes the emotional aspects of posts, achieving a more comprehensive risk assessment. Specific embodiments of the system according to the present invention are described in detail below.

[0279] Server-side processing

[0280] 1. Receiving submitted data

[0281] The server receives post data (text, images, videos) from the user's device. When the user presses the post button, a POST request is sent from the device. The server receives this request using the REST API.

[0282] 2. Data Analysis

[0283] The server's AI analysis module analyzes the posted data, specifically as follows:

[0284] Sentence analysis:

[0285] A natural language processing (NLP) library is used to tokenize posts and perform contextual and sentiment analysis.

[0286] Image analysis:

[0287] Recognize objects in an image using an object detection model (e.g., YOLO or SSD).

[0288] It uses OCR technology to extract text from images, which can then be translated and analyzed.

[0289] Video Analysis:

[0290] The video is divided into frames, and each frame is processed in the same way as image analysis.

[0291] The audio portion of the video is converted into text using automatic speech recognition (ASR), and then sentiment analysis and related contextual analysis are performed.

[0292] 3. Emotion recognition

[0293] The server's emotion recognition engine detects the user's emotions from the received post data, as follows:

[0294] Text data: Natural language processing techniques are used to analyze emotions within the text. For example, emotions such as joy, anger, sadness, and happiness are identified, and an emotional assessment is made based on these.

[0295] Image data: Facial recognition and facial expression analysis techniques are used to identify the emotions of people in images. This is used in conjunction with image analysis techniques to assess the strength of emotions.

[0296] Video data: Emotions are recognized by analyzing the voice and facial expressions in the video. Image analysis and voice recognition technologies are combined to perform a comprehensive emotional evaluation.

[0297] 4. Comparison with past cases of online outrage

[0298] The server compares the analysis results and emotion recognition results with a database of past flaming cases, performs a similarity search, evaluates the degree of similarity of the posted data to past flaming cases, and calculates a similarity score.

[0299] 5. Risk Assessment

[0300] The server evaluates the risk of a controversy based on the analysis results, emotion recognition results, and similarity scores. For example, it calculates a risk score and determines that a score of 90 or higher is high risk, 70-89 is medium risk, and 70 or lower is low risk.

[0301] 6. Risk Notification and Alternative Generation

[0302] If the server determines that the risk of outrage is high, it generates a risk warning and alternatives. It uses a derived sentence generation model (e.g., GPT-3) to generate alternative sentences with less risk. If necessary, it also generates suggested revisions for images and videos.

[0303] 7. Sending warnings and alternatives

[0304] The server transmits the generated risk warning and alternatives to the user terminal.

[0305] Terminal side processing

[0306] 1. Create submission data

[0307] Users create posts through SNS applications, and can input text and upload images and videos.

[0308] 2. Submitting data

[0309] When the user presses the post button, the terminal sends the created post data to the server, for example, using a REST API POST request.

[0310] 3. Receive risk warnings

[0311] The device receives risk warnings sent from the server and notifies the user, including the risk assessment score and a specific warning message.

[0312] 4. Displaying alternatives

[0313] The terminal displays the alternatives sent from the server for reducing the risk on the user interface, and the user checks them and decides whether to modify the original post.

[0314] User operations

[0315] 1. Enter your post content

[0316] A user creates a post in a social networking application, for example, by typing, "Today is the worst day. Everything is going wrong."

[0317] 2. Risk confirmation

[0318] The user checks the risk assessment notification and warning message from the server on their device. For example, a warning message such as "This post contains negative sentiment and is at high risk of causing a backlash" may be displayed.

[0319] 3. Selection of alternatives

[0320] The user checks the alternatives provided by the server and decides whether to revise the post based on them. For example, an alternative suggestion might be "Today didn't go well, but I'll try my best tomorrow!"

[0321] 4. Confirmation of submission

[0322] The user can then modify the content of their post based on the displayed alternatives and finally confirm the post. The modified content is then sent back to the server and posted to the SNS.

[0323] In this way, the system of the present invention assesses the risk of a social media firestorm before a post is made, helping users to use social media safely. In addition, by combining it with an emotion recognition engine, it is possible to analyze the emotional aspects of posts, enabling more accurate risk assessment.

[0324] The processing flow will be explained below.

[0325] Server-side processing

[0326] Step 1:

[0327] The server receives post data (text, images, videos) from the user's device. When the user presses the post button, a POST request is sent from the device. The server receives this request via the REST API.

[0328] Step 2:

[0329] The server preprocesses the received submission data, tokenizing it for text data and storing image and video data in a specific directory.

[0330] Step 3:

[0331] The server's AI analysis module analyzes the posted data using the following steps:

[0332] Text analysis: Uses natural language processing (NLP) libraries to tokenize text and perform contextual and sentiment analysis.

[0333] Image analysis: Recognize objects and text in images using object detection models (e.g., YOLO or SSD).

[0334] Video analysis: Videos are divided into frames, and each frame is analyzed in the same way as image analysis. The audio portion of the video is converted into text using automatic speech recognition (ASR), and this is then analyzed.

[0335] Step 4:

[0336] The emotion recognition engine of the server recognizes the emotion of the user based on the received posting data.

[0337] Text data: Natural language processing techniques are used to analyze sentiment within the text, for example, identifying positive, negative, or neutral sentiment.

[0338] Image data: Using facial recognition and facial expression analysis techniques, we identify the emotions of people in images.

[0339] Video data: Analyzes the voice and facial expressions in the video, and integrates multiple emotional data to evaluate the user's overall emotions.

[0340] Step 5:

[0341] Based on the analysis results and emotion recognition results, the server compares the posted data with a database of past flaming cases, performs a similarity search, and scores the degree of similarity with past flaming cases.

[0342] Step 6:

[0343] The server evaluates the risk of a controversy based on the analysis results, emotion recognition results, and similarity scores. A score of 90 or above is considered high risk, 70-89 is considered medium risk, and 70 or below is considered low risk.

[0344] Step 7:

[0345] If the risk of a post being criticized is high, the server will notify the user of the risk and generate alternatives. For example, it will use a derived sentence generation model (e.g., GPT-3) to generate alternative sentences with less risk. In some cases, it will also generate suggested revisions for images and videos.

[0346] Step 8:

[0347] The server sends the generated risk warning and alternatives to the user's device, and if an error occurs, it records it in a log.

[0348] Terminal side processing

[0349] Step 1:

[0350] A user creates a post through a social networking application, for example, by inputting text and uploading images and videos.

[0351] Step 2:

[0352] When the user presses the post button, the terminal sends the created post data to the server, for example, using a POST request.

[0353] Step 3:

[0354] The device waits for a response from the server and receives a risk warning and alternatives.

[0355] Step 4:

[0356] The device will display risk warnings and alternatives to the user, for example as a pop-up message or notification.

[0357] User operations

[0358] Step 1:

[0359] A user inputs a post into a social networking application, for example, a sentence such as "Today is the worst day. Everything is going wrong."

[0360] Step 2:

[0361] When the user presses the post button, the post data is sent to the server.

[0362] Step 3:

[0363] The user will see a risk warning and alternatives from the server. For example, a warning message such as "This post contains negative sentiment and is at high risk of causing controversy" will be displayed.

[0364] Step 4:

[0365] The user checks the alternatives and decides whether to revise the post based on them. For example, an alternative suggestion might be "Today didn't go well, but I'll try my best tomorrow!"

[0366] Step 5:

[0367] Once the user has finalized the content of the post and pressed the post button again, the revised post data will be resent to the server and posted on the SNS.

[0368] Example 2

[0369] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0370] There is a need to provide an environment where users can safely use SNS by assessing the risk of data posted on SNS becoming a hot topic in advance. Conventional technologies simply filter the content of posts, without performing comprehensive analysis including emotional aspects and images and videos, making it difficult to accurately assess risk. The purpose of this invention is to solve these problems and provide a more comprehensive and accurate risk assessment system.

[0371] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving post data, means for analyzing the received post data using natural language processing and image analysis techniques, means for performing emotion recognition based on the analysis results, means for comparing the emotion recognition results and the analysis results with a database of past flame war cases, means for evaluating the risk of a flame war based on the comparison results, means for sending a warning to the user when the risk of a flame war is high, means for generating alternative information to reduce the risk of a flame war using a generative AI model, and means for sending the warning and alternative information to the user terminal. This makes it possible to provide a highly accurate evaluation of the risk of a flame war and appropriate alternatives before a user posts on an SNS.

[0372] "Posted data" refers to data in the form of text, images, and videos posted by users to online platforms such as social networking sites.

[0373] "Natural language processing" is a technology that enables computers to understand, interpret, and generate human language, typically involving tokenization, contextual analysis, and sentiment analysis.

[0374] "Image analysis" refers to the use of computer vision techniques to recognize and analyze objects and text in images. Examples include object detection and OCR.

[0375] "Emotion recognition" is a technology that uses natural language processing and image analysis technologies to detect emotions from text, images, and videos.

[0376] "Flame risk" is an evaluation index of the likelihood that posted data will receive a large amount of negative reactions or criticism on social media.

[0377] The "database of past social media outrage cases" is a database of past social media outrage cases, and is a data set used for risk assessment.

[0378] A "generative AI model" is an artificial intelligence model that generates new data from given input data. A specific example is GPT-3.

[0379] "Similarity search" is a technique for evaluating how similar certain data is to given reference data (in this case, past online scandals).

[0380] A "warning" is a message that notifies the user that there is a high risk of the posted data causing a firestorm.

[0381] "Alternative information" refers to modifications or alternatives suggested by generative AI models to reduce the risk of backlash.

[0382] The present invention is a system that analyzes social media posting data in advance and assesses the risk of a social media outcry. This system combines emotion recognition technology and generative AI models to provide a more comprehensive risk assessment and alternative solutions. Specific embodiments of the present invention are described in detail below.

[0383] Server-side processing

[0384] The server first receives the posted data (text, images, videos) sent from the user's device. This reception process uses a POST request using the REST API. For example, data is received through an API gateway with a set endpoint. The server uses a cloud platform to achieve scalable processing.

[0385] The server's AI analysis module then analyzes the received data using the following techniques:

[0386] Use a natural language processing (NLP) library (e.g., spaCy or BERT) to tokenize the text data and perform contextual and sentiment analysis.

[0387] It uses object detection models (e.g., YOLO and SSD) to identify objects in images, and OCR technology to extract text from images for translation and sentiment analysis.

[0388] In video analysis, the video is divided into frames and each frame is processed in the same way as image analysis. The audio portion of the video is converted into text using automatic speech recognition (ASR), and then emotion and context analysis are performed.

[0389] Emotion recognition is performed to detect user emotions from received posting data. This process uses natural language processing technology to detect emotions in text and identify emotions such as joy, anger, sadness, and happiness. It also uses facial recognition and facial expression analysis technology to analyze the emotions of people in images and videos.

[0390] Based on the analysis results and emotion recognition results, the server compares them with a database of past flaming cases. Using a similarity search technique, the server evaluates how similar the posted data is to past flaming cases and calculates a similarity score. Specifically, the comparison uses techniques such as Cosine similarity.

[0391] The server then assesses the risk of a controversy based on the analysis results, emotion recognition results, and similarity scores. A score of 90 or higher is considered high risk, 70-89 is considered medium risk, and 70 or lower is considered low risk.

[0392] If the risk of outrage is high, the server generates a risk warning and alternative suggestions. Using a generative AI model (e.g., GPT-3), it generates alternative text to reduce the risk. If necessary, it also generates suggested edits for images and videos. The generated alternative suggestions and risk warnings are converted into JSON format and sent to the user's device.

[0393] Terminal side processing

[0394] First, a user creates a post through a social networking application. They enter text and upload images and videos. When the user presses the post button, this post data is sent to the server via a REST API POST request.

[0395] When a risk warning is sent from the server, the device receives it and notifies the user. The risk assessment score and specific warning message are displayed on the interface, and the device also displays the alternatives sent from the server. The user can review the alternatives and decide whether to revise the original post.

[0396] User operations

[0397] A user creates a post in a social networking application. For example, the user enters the sentence, "Today is the worst day. Everything is going wrong." The server determines that this post contains strong negative emotions and sends a warning message to the device indicating a high risk of it starting a flame war. The warning message reads, "This post contains negative emotions and there is a high risk of it starting a flame war."

[0398] The user also checks alternative suggestions provided by the server. For example, an alternative sentence such as "Today didn't go well, but I'll try my best tomorrow!" is presented. The user can then revise the post based on this and finally confirm the post. The revised post is then sent back to the server and posted to the SNS.

[0399] This system allows users to use social media safely and assess the risk of a social media firestorm in advance. Furthermore, by using emotion recognition technology, it is possible to analyze the emotional aspects of posts and improve the accuracy of risk assessment.

[0400] Specific examples

[0401] For example, if a user tries to post "Today is the worst day. Everything is going wrong," the server analyzes it and determines that it is a post with a strong negative sentiment. If it is assessed as having a high risk of causing a flame war, the server generates an alternative suggestion, such as "Today didn't go well, but I'll try my best tomorrow!" and presents it to the user.

[0402] Prompt Sentence Examples

[0403] Input: Today is the worst day. Everything is going wrong.

[0404] Output: Things didn't go well today, but I'll try my best tomorrow!

[0405] This prompt sentence can be used to train a generative AI model (e.g., GPT-3) to generate a less risky alternative sentence.

[0406] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0407] Step 1: Receive post data

[0408] The server receives post data from the user's device. When a user presses the post button in the SNS application, a POST request is sent from the device. The input is the text, image, and video data created by the user, and the output is the post data received by the server. Specifically, the server receives this request using a REST API and retrieves the data through an API gateway with a configured endpoint.

[0409] Step 2: Data analysis

[0410] The AI ​​analysis module on the server analyzes the received post data. Specifically,

[0411] Text analysis: Using a natural language processing (NLP) library (e.g., spaCy or BERT), text data is tokenized and context and sentiment analysis is performed. The input is text data, and the output is the analysis results.

[0412] Image analysis: Identify objects in images using object detection models (such as YOLO or SSD). Also, extract text from images using OCR technology and perform sentiment analysis. The input is image data, and the output is the analysis results.

[0413] Video analysis: Videos are divided into frames, and each frame is processed in the same way as image analysis. The audio portion of the video is converted into text using automatic speech recognition (ASR), and emotion and context analysis is performed. The input is video data, and the output is the analysis results data.

[0414] Step 3: Emotion Recognition

[0415] The server's emotion recognition engine detects the user's emotions from the analyzed post data. Specifically,

[0416] Text data: Detect emotions in text using natural language processing techniques. The input is the result of text analysis, and the output is the emotion recognition result of the text.

[0417] Image data: Using facial recognition and facial expression analysis technology, we identify the emotions of people in an image. The input is the result of image analysis, and the output is the emotion recognition result of the image.

[0418] Video data: Analyzes the voice and facial expressions in the video to perform a comprehensive emotional evaluation. The input is the result of video analysis data, and the output is the emotion recognition results for the video.

[0419] Step 4: Compare with past flame war cases

[0420] The server compares the analysis results and emotion recognition results with a database of past flaming cases. Specifically, it uses a similarity search method to evaluate how similar the posted data is to past flaming cases. The input is the analysis results and emotion recognition results, and the output is a similarity score. Cosine similarity, for example, is used to evaluate the similarity.

[0421] Step 5: Risk assessment

[0422] The server evaluates the risk of a controversy based on the analysis results, emotion recognition results, and similarity score. It calculates a risk score and classifies a score of 90 or above as high risk, 70-89 as medium risk, and 70 or below as low risk. The inputs are the similarity score, emotion recognition results, and analysis results, and the output is the risk assessment score and its judgment result.

[0423] Step 6: Risk notification and alternative generation

[0424] If the server determines that there is a high risk of outrage, it generates a risk warning and alternatives. It uses a generative AI model (e.g., GPT-3) to generate alternative text with less risk. If necessary, it also generates suggested modifications for images and videos. The input is the risk assessment score and the judgment result, and the output is a risk warning message and alternatives.

[0425] Step 7: Sending warnings and alternatives

[0426] The server sends the generated risk warning and alternatives to the user's device. The input is the risk warning message and alternatives, and the output is the data to be sent to the user's device. The risk warning and alternatives are converted to JSON format and sent to the user's device via a REST API.

[0427] Step 8: Receive risk warnings

[0428] The terminal receives the risk warning sent from the server and notifies the user. The input is the risk warning message and alternatives sent from the server, and the output is the notification to the user interface.

[0429] Step 9: View alternatives

[0430] The terminal displays the alternatives sent from the server on the user interface. The user checks them and decides whether to modify the original post. The input is the alternatives, and the output is the display content presented to the user.

[0431] Step 10: User Edits to Posts

[0432] The user modifies the post based on the alternatives provided by the server. The input is the alternatives and the original post, and the output is the modified post.

[0433] Step 11: Finalize and submit your revised post

[0434] The user confirms the revised content of the post and sends it back to the server. The input is the revised content of the post, and the output is the final post data sent to the server. The server receives this revised post and posts it to the SNS.

[0435] (Application example 2)

[0436] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0437] In recent years, the risk of user posts on social media and online platforms causing online outrage has become a social issue. In particular, for brick-and-mortar stores, there is a growing risk that negative customer feedback spreading online could damage their reputation and negatively impact sales. It is also important to quickly analyze customer feedback and provide risk assessments and appropriate countermeasures. The present invention aims to solve these problems by providing a system that enables brick-and-mortar stores to analyze customer feedback and quickly take appropriate countermeasures.

[0438] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0439] In this invention, the server includes means for receiving posted data, means for analyzing the received posted data and comparing it with past cases of online flaming, means for assessing the risk of online flaming based on the analysis results, means for sending a warning to the user when the risk of online flaming is high, means for generating alternative information to reduce the risk of online flaming, means for sending the warning and alternative information to the user terminal, means for analyzing customer feedback at the physical store, means for performing sentiment analysis of the posted data and assessing the risk, and means for generating alternatives that arise when the risk is high. This enables quick and accurate analysis of customer feedback and risk assessment, and appropriate action to protect the reputation of the physical store.

[0440] "Posted data" refers to information such as text, images, and videos that users create and send on social media or online platforms.

[0441] "Means for receiving" refers to the functions and protocols that allow the server to receive posted data sent from the user terminal.

[0442] "Means of analysis" refers to the process of analyzing the content of received posted data using natural language processing and image analysis technology.

[0443] "Past cases of online outrage" refer to cases in the past where a user's post on a social media or online platform sparked criticism or negative reactions.

[0444] "Means of comparison" refers to the algorithms and methods used to evaluate the similarity of newly received posted data with past cases of online outrage.

[0445] "Flame risk" refers to the likelihood that a particular post will provoke criticism or negative reactions in the online community.

[0446] "Sentiment analysis" refers to the process of identifying emotional components in text or images using natural language processing and image recognition techniques.

[0447] "Risk assessment" refers to the process of evaluating the degree of risk of a post causing a controversy using numerical or categorized terms based on the analysis of received post data.

[0448] "Means for sending warnings" refers to the function of sending warning messages to users when there is a high risk of a social media outburst.

[0449] "Alternative information" refers to safer options, such as text or images that have been modified from the original post to reduce the risk of a backlash.

[0450] "Means for generating alternatives" refers to a process that uses generative AI models or other technologies to automatically create safe alternative information.

[0451] "User terminal" refers to devices such as smartphones, tablets, and PCs used to access platforms such as social networking sites.

[0452] "Brick and mortar store" refers to a store that offers goods or services in a physical location.

[0453] "Customer feedback" refers to customer experiences, opinions, and evaluations expressed in the form of posts in physical stores.

[0454] The present invention relates to a system that analyzes posted data in advance and evaluates the risk of a social media firestorm. This system is primarily intended to analyze customer feedback in brick-and-mortar stores, and is designed to prevent negative feedback from damaging the store's reputation. Specific embodiments are described below.

[0455] Server-side processing

[0456] 1. Receiving submitted data

[0457] The server receives the posted data (text, images, videos) sent from the user's device. When the user sends feedback, a POST request is sent from the user's device via the REST API. The server receives this request and prepares to analyze the posted data.

[0458] 2. Data Analysis

[0459] The server program analyzes the submitted data using the following hardware and software:

[0460] Natural Language Processing (NLP) library: Uses TextBlob to tokenize posts and perform contextual and sentiment analysis.

[0461] Image Analysis: Extract text in images using OpenCV and pytesseract and perform analysis on the same.

[0462] The analysis results are compared with a database of past online outrage cases and a similarity search is performed.

[0463] 3. Emotion recognition

[0464] The server analyzes the sentiment of the posted data through the following process:

[0465] Text data: Natural language processing techniques are used to identify sentiment within the text.

[0466] Image data: Use OCR technology to extract text from images and analyze their sentiment.

[0467] 4. Risk assessment and alternative generation

[0468] The server evaluates the risk of a controversy based on the analysis results and emotion recognition results. If the risk is determined to be high, it uses a generative AI model (e.g., GPT-3) to generate alternatives to reduce the risk.

[0469] Terminal side processing

[0470] The user terminal provides the following functions:

[0471] Feedback submission: Users can enter feedback in the form of text, images, or videos using their smartphones.

[0472] Display risk warning: Receives risk warnings sent from the server and notifies the user.

[0473] Displaying alternatives: The alternatives sent from the server are displayed in the user interface, allowing the user to choose whether to revise the post.

[0474] Specific examples

[0475] For example, if a customer posts feedback saying, "Today's customer service was terrible," this post is sent to the server. On the server side, negative sentiment is detected using TextBlob and evaluated as high risk. In response, a generative AI model is used to generate an alternative suggestion, such as, "Today was a bit disappointing, but I look forward to your next visit," and this is presented to the user.

[0476] Prompt Sentence Examples

[0477] "Generate positive alternatives for the user-submitted feedback, 'The customer service today was terrible.'"

[0478] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0479] Step 1:

[0480] The user uses their smartphone to enter feedback, which can be in the form of text, images, or videos. The entered feedback might be something like "Today is the worst day. Everything is going wrong." When the user presses the submit button, the device sends the feedback data to the server as a POST request.

[0481] Step 2:

[0482] The server receives the posted data sent from the user's device. The received data can be text, images, or videos, and is sent via the REST API. The server temporarily stores the received data.

[0483] Step 3:

[0484] The server analyzes the received post data. For text data, it uses TextBlob to tokenize the data, analyzes the context, and performs sentiment analysis. The input for analysis is the post text, and the output is the sentiment polarity (positive, negative, neutral). For example, "Today is the worst day. Everything is going wrong." is analyzed as negative.

[0485] Step 4:

[0486] The server analyzes the image data. It extracts the text from the image using OpenCV and performs OCR analysis using pytesseract. The input for the analysis is the posted image, and the extracted text is again subjected to sentiment analysis using TextBlob. The output is the text from the image and the results of the sentiment analysis.

[0487] Step 5:

[0488] The server analyzes the video data. It divides the video into frames and performs image analysis on each frame. It also converts the audio portion of the video into text using automatic speech recognition (ASR), and performs sentiment analysis on that text. The input for the analysis is the posted video, and the output is the text for each frame and the sentiment analysis results.

[0489] Step 6:

[0490] The server compares the analysis results with a database of past flaming cases. It uses a similarity algorithm to evaluate how similar the analysis data is to past flaming cases. The input is the analysis results and past flaming cases, and the output is a similarity score.

[0491] Step 7:

[0492] The server evaluates the risk of a controversy based on the analysis results and the similarity score. The emotional polarity and similarity scores are combined to calculate an overall risk score. The output is a risk score; for example, a risk score of 90 or higher is considered high risk.

[0493] Step 8:

[0494] If the risk of a firestorm is determined to be high, the server sends a risk warning to the user's device. The warning includes a risk score and a specific warning message. The input is the risk score, and the output is the warning message.

[0495] Step 9:

[0496] To generate alternatives, the server uses a generative AI model. For example, it uses a prompt such as "Generate a positive alternative to the user-posted feedback, 'Today is the worst day. Everything is going wrong.'" The input is the original post text and the prompt, and the output is the alternative text. For example, an alternative such as "Today didn't go well, but let's try our best tomorrow!" is generated.

[0497] Step 10:

[0498] The server sends the generated alternatives to the user's terminal. The user can check the alternatives and choose whether to correct the feedback. The input is the alternative text, and the output is a notification to the user's terminal.

[0499] Step 11:

[0500] The user then takes the suggested alternatives into consideration and finally submits the revised feedback, which is then sent back to the server and posted on the SNS platform.

[0501] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0502] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0503] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0504] [Second embodiment]

[0505] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0506] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0507] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0508] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0509] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0510] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0511] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0512] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0513] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0514] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0515] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0516] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0517] The present invention relates to a system for analyzing posted data on a social networking site in advance and assessing the risk of a social networking site becoming a hot topic. Specific embodiments of the system according to the present invention will be described in detail below.

[0518] Server-side processing

[0519] 1. Receiving submitted data

[0520] The server receives the posted data (content such as text, images, and videos) sent from the user's device. For example, it retrieves the data using a POST request using a REST API.

[0521] 2. Data Analysis

[0522] The AI ​​analysis module on the server analyzes the data in the following steps:

[0523] Sentence analysis:

[0524] Use a natural language processing (NLP) library to tokenize the text and perform morphological analysis.

[0525] Extract relevant keywords and emotional expressions from the text.

[0526] Image analysis:

[0527] Recognize objects in an image using an object detection model (e.g., YOLO or SSD).

[0528] The text in the image is extracted using OCR technology and then further analyzed.

[0529] Video Analysis:

[0530] The video is divided into frames, and each frame is processed in the same way as image analysis.

[0531] The audio portion of the video is converted into text using automatic speech recognition (ASR) technology, and text analysis is performed.

[0532] 3. Comparison with past cases of online outrage

[0533] The server compares the analysis results with a database of existing cases of online outrages and evaluates the risk of online outrage based on the degree of similarity. For example, it searches the database of past cases of online outrages and calculates a similarity score.

[0534] 4. Risk Assessment

[0535] The server scores the risk of a controversy based on the similarity score and the extracted sentiment and keywords. The server then determines the degree of risk based on the assessed risk score. For example, a risk score of 90 or above is high risk, 70-89 is medium risk, and anything below that is low risk.

[0536] 5. Risk Notification and Alternative Generation

[0537] The server sends a warning to users if the risk score is high. To support safe posting, the server uses a derived text generation model (e.g., GPT-3) to generate alternative text and image / video revision suggestions with less risk and sends them to the user's device.

[0538] Terminal side processing

[0539] 1. Create submission data

[0540] Users use SNS applications to create posts, including text they type and images and videos they upload.

[0541] 2. Submitting data

[0542] When the user presses the post button, the terminal sends the created post data to the server, for example, using a REST API POST request.

[0543] 3. Receive risk warnings

[0544] The device receives risk warnings sent from the server and notifies the user, including the risk assessment score and a specific warning message.

[0545] 4. Displaying alternatives

[0546] The terminal displays the alternatives sent from the server for reducing the risk on the user interface, and the user checks them and decides whether to modify the original post.

[0547] User operations

[0548] 1. Enter your post content

[0549] A user creates a post on a social networking app, for example, by typing a sentence such as, "Today is the worst day. Everything is going wrong."

[0550] 2. Risk confirmation

[0551] The user checks the risk assessment notification and warning message from the server on their device. For example, a warning message such as "This post contains negative sentiment and is at high risk of causing a backlash" may be displayed.

[0552] 3. Selection of alternatives

[0553] The user checks the alternatives provided by the server and decides whether to revise the post based on the alternatives or to post the original post as is. For example, an alternative suggestion might be "Today didn't go well, but I'll try my best tomorrow!"

[0554] 4. Confirmation of submission

[0555] The user can then modify the content of their post based on the displayed alternatives and finally confirm the post. The modified content is then sent back to the server and posted to the SNS.

[0556] In this way, the system of the present invention evaluates the risk of a controversy before posting, helping users to use SNS safely.

[0557] The processing flow will be explained below.

[0558] Server-side processing

[0559] Step 1:

[0560] The server receives post data (text, images, videos) from the user's device. When the user presses the post button, a POST request is sent from the device. The server receives this request using the REST API.

[0561] Step 2:

[0562] The server preprocesses the submitted data it receives, for example tokenizing text data and storing image and video data in specific directories.

[0563] Step 3:

[0564] The server's AI analysis module analyzes the posted data.

[0565] Text analysis: Using a natural language processing (NLP) library, posts are tokenized and semantic and sentiment analysis is performed.

[0566] Image analysis: Recognize objects in images using object detection models (e.g., YOLO and SSD), and extract text from images using hooked OCR technology.

[0567] Video analysis: Videos are divided into frames, and each frame is analyzed in the same way as image analysis. Audio in the video is converted into text using automatic speech recognition (ASR), and related sentence analysis is performed.

[0568] Step 4:

[0569] The server compares the analysis results with a database of past flaming cases, performs a similarity search, evaluates the degree to which the posted data is similar to past flaming cases, and calculates a similarity score.

[0570] Step 5:

[0571] The server evaluates the risk of a controversy based on the analysis results and the similarity score. For example, it calculates a risk score, determining that a score of 90 or higher is high risk, 70-89 is medium risk, and 70 or lower is low risk.

[0572] Step 6:

[0573] If the server determines that a post is at high risk of causing an uproar, it generates a risk warning and alternatives. It uses a derived sentence generation model (e.g., GPT-3) to generate alternative sentences with less risk. If necessary, it also generates suggested revisions for images and videos.

[0574] Step 7:

[0575] The server sends risk warnings and alternative solutions to the user's device, and keeps a log containing details of any errors that occur.

[0576] Terminal side processing

[0577] Step 1:

[0578] Users create posts through SNS applications, and can input text and upload images and videos.

[0579] Step 2:

[0580] When the user presses the submit button, the device sends the submission data to the server, for example, using a REST API POST request.

[0581] Step 3:

[0582] The device waits for a response from the server, and if the server returns a risk warning and alternatives, it receives them.

[0583] Step 4:

[0584] The device displays the risk warning and alternatives received from the server to the user, for example, by a pop-up message or a notification box.

[0585] Step 5:

[0586] Users review the risk warnings and alternative suggestions and decide whether to revise their original post. If revisions are necessary, they reconstruct the content using the alternative sentences and suggested revisions as a reference.

[0587] Step 6:

[0588] When the user decides on the final post and presses the post button again, the terminal resends the corrected post data to the server.

[0589] User Action

[0590] Step 1:

[0591] A user types or uploads a post to a social networking application, for example, typing the sentence "Today is the worst day. Everything is going wrong."

[0592] Step 2:

[0593] When the user presses the submit button, the data is sent to the server.

[0594] Step 3:

[0595] The user then checks the risk warning and alternatives from the server. For example, the user may be presented with a warning message such as "This post contains negative emotions and is at high risk of causing a backlash," along with an alternative sentence such as "Today didn't go well, but I'll try my best tomorrow!"

[0596] Step 4:

[0597] The user decides whether to revise the post based on the alternative suggestions, and if so, checks the new post.

[0598] Step 5:

[0599] Once the user has finalized the content of the post and pressed the post button again, the final post data will be posted on the SNS.

[0600] Example 1

[0601] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0602] Posting on social media is an important way to share information with many people, but if inappropriate content is included, it can lead to outrage and excessive criticism. In addition, since there are limited ways for users to know about such risks in advance, they may be cautious about using social media. Therefore, a system is needed that evaluates the risk of outrage before posting and suggests safe ways to post.

[0603] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0604] In this invention, the server includes means for receiving post data from a user terminal, means for analyzing the received post data using natural language processing, object recognition, and voice recognition, means for comparing the received post data with past flaming cases based on keywords and emotional expressions extracted by the analysis, means for scoring and assessing the risk of flaming based on the comparison results, means for sending a warning to the user if the risk of flaming is high, means for generating alternative information to reduce the risk of flaming, and means for sending the warning and alternative information to the user terminal. This allows users to assess the risk of flaming before posting and learn safe posting methods.

[0605] A "user terminal" is a device operated by a user, and includes a smartphone, tablet, PC, etc.

[0606] "Posted data" refers to content such as text, images, and videos that users share on social media.

[0607] "Server" refers to a high-performance computer system installed at a remote location that analyzes and processes data received from user devices.

[0608] "Natural language processing" is a technology for processing human language using computers, and includes text tokenization, morphological analysis, sentiment analysis, etc.

[0609] "Object recognition" refers to the technology of identifying objects in images and classifying them.

[0610] "Speech recognition" refers to the technology of converting voice data into text.

[0611] "Keywords" are words or phrases that are particularly important in a sentence or piece of content.

[0612] "Sentiment expression" refers to elements in text that indicate sentiment (positive, negative, neutral, etc.) analyzed by natural language processing.

[0613] "Flame war cases" include data that refer to large-scale negative reactions caused by inappropriate posts on social media in the past.

[0614] The "similarity score" is a numerical value that indicates the similarity between the target data and past flame war case data, and is used for risk assessment.

[0615] "Scoring" refers to a method of quantifying risk or performance based on specific criteria.

[0616] "Warning" refers to a caution message to inform the user of a high risk.

[0617] "Alternative information" refers to safe posts and suggested revisions to reduce the risk of outrage.

[0618] MODE FOR CARRYING OUT THE INVENTION

[0619] The present invention relates to a system for analyzing data posted to a social networking site in advance and assessing the risk of a social networking site becoming a hot topic. Specific embodiments of the present invention will be described in detail below.

[0620] Server Processing

[0621] The server receives the posted data (text, images, videos, and other content) sent from the user's device. Specifically, it uses a mechanism to receive data via a POST request via the REST API.

[0622] The server analyzes the received posted data using the following methods. First, it uses a natural language processing (NLP) library (e.g., spaCy) to tokenize the text data and extract emotional expressions and keywords. Next, it uses an object recognition model (e.g., YOLO) to recognize objects within the image. It also uses OCR technology (e.g., Tesseract OCR) to extract text from the image and further analyze it. It then divides the video data into frames, each of which is processed in the same way as image analysis. It then uses automatic speech recognition (ASR) technology to convert the audio portion of the video into text, which is then analyzed.

[0623] The server compares the analysis results with a database of past flaming cases. It calculates a similarity score from the database and evaluates the risk. For this purpose, it uses a scoring method; for example, a similarity score of 90 or higher with past cases is rated as high risk, 70-89 as medium risk, and anything below that is rated as low risk.

[0624] If the server determines that there is a high risk of outrage, it sends a warning to the user. It also uses a generative AI model (e.g., GPT-3) to generate alternative information (suggested revisions to text, images, and videos) with less risk and sends them to the user's device.

[0625] Terminal handling

[0626] Users use SNS applications to create posts, and the text they enter and the images and videos they upload become the post data.

[0627] When the user presses the submit button, the device sends the created post data to the server using a REST API POST request.

[0628] The device receives the risk warning sent from the server and notifies the user, displaying the risk assessment score and a specific warning message.

[0629] The terminal displays the alternatives for reducing the risk sent from the server to the user, who then checks the alternatives and decides whether to revise the content of the post and repost it.

[0630] User operations

[0631] A user inputs content to be posted on the SNS application to create post data. For example, the user inputs a sentence such as "Today is the worst day. Everything is going wrong."

[0632] The user checks the risk assessment notification and warning message from the server on their device. For example, a warning message such as "This post contains negative emotions and is at high risk of causing a backlash" may be displayed.

[0633] The user checks the alternatives provided by the server and modifies the content of the post based on them. For example, an alternative suggestion such as "Today didn't go well, but I'll try my best tomorrow!" may be presented.

[0634] Specific examples

[0635] Example prompt sentence:

[0636] "This sentence contains negative content. Please consider the following alternatives:

[0637] 1. Things didn't go well today, but I'll try my best tomorrow!

[0638] 2. Sometimes you have bad days, but that's life.

[0639] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0640] Step 1: Receiving posted data

[0641] The server receives posted data from user devices. Input includes content such as text, images, and videos. Specifically, data is received via a POST request via the REST API. The output is the received posted data itself.

[0642] Specific behavior:

[0643] When a user presses the "Post" button on their device, the device sends a POST request to the server, and text, image, and video data are sent to the server.

[0644] Step 2: Text analysis

[0645] The server analyzes the received text data using a natural language processing (NLP) library. The input is the received text data, and the output is the analysis results such as keywords and emotional expressions.

[0646] Specific behavior:

[0647] The server uses the spaCy library to tokenize the text and extract important keywords and sentiments. For example, from the sentence "Today is the worst day. Everything is going wrong," it extracts keywords like "worst" and "going wrong."

[0648] Step 3: Image analysis

[0649] The server analyzes the received image data using an object recognition model (e.g., YOLO). The input is the received image data, and the output is the analysis results of objects and text in the image.

[0650] Specific behavior:

[0651] The server uses the YOLO model to identify people and objects in the image and extract their location information, and Tesseract OCR to extract text from the image and analyze important keywords such as "warning" and "caution."

[0652] Step 4: Video Analysis

[0653] The server divides the received video data into frames and processes each frame in the same way as image analysis. The input is the received video data, and the output is the image analysis results for each frame and the speech-to-text results.

[0654] Specific behavior:

[0655] The server uses OpenCV to split the video into frames and analyzes each frame with the YOLO model. At the same time, the audio portion is converted to text using the Google Speech-to-Text API and subjected to NLP processing.

[0656] Step 5: Compare with past flame wars

[0657] The server compares the analysis results with a database of past flaming cases. The input is the analyzed keywords and emotional expressions, and the output is a similarity score.

[0658] Specific behavior:

[0659] The server uses TfidfVectorizer to calculate the similarity score between the analysis results and past flaming cases. For example, if the keywords "worst" and "doesn't work" match many of the past flaming cases, a high similarity score will be calculated.

[0660] Step 6: Risk assessment

[0661] The server scores the flame risk based on the similarity score. The input is the similarity score, and the output is the estimated flame risk score.

[0662] Specific behavior:

[0663] The server applies a scoring algorithm to calculate a risk score. For example, a similarity score of 90 or higher is considered "high risk," and a score between 70 and 89 is considered "medium risk."

[0664] Step 7: Sending an alert

[0665] The server sends a warning to the user if the risk score is high. The input is the assessed risk score and the output is the warning message.

[0666] Specific behavior:

[0667] The server generates a warning message based on the risk score and sends it to the user's device. For example, a message might say, "This post contains negative emotions and is at high risk of causing a backlash."

[0668] Step 8: Generate alternative information

[0669] To reduce the risk of outrage, the server generates alternative information using a generative AI model (e.g., GPT-3). The input is the initial post data, and the output is the generated alternative sentences or suggested revisions.

[0670] Specific behavior:

[0671] If the initial post is "Today is the worst day. Everything is going wrong," GPT-3 will be used to generate an alternative post that says, "Some things didn't go well today, but I'll try my best tomorrow!"

[0672] Step 9: Notification of warnings and alternatives

[0673] The terminal receives the warning and alternatives sent from the server and notifies the user of them. The input is the warning message and alternatives sent from the server, and the output is the notified message.

[0674] Specific behavior:

[0675] The device will use notifications to display warnings and alternatives to the user, including a risk assessment score and specific warning messages.

[0676] Step 10: Edit and finalize your submission

[0677] The user modifies the posted content by referring to the alternatives and finally confirms it. The input is the alternatives and the initial posted data, and the output is the modified posted data.

[0678] Specific behavior:

[0679] The user then modifies the content of the post based on the alternative suggestions and clicks the "Post" button again to send the modified post to the server, where it is evaluated again and posted on the SNS.

[0680] (Application example 1)

[0681] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0682] Systems that can detect the risk of unexpected outrage or fraudulent use and enable users to take appropriate action are needed not only for data posted on social media but also for electronic payment services. However, current systems have limited scope for risk assessment, and do not adequately deter fraudulent use or prevent outrage. In particular, analysis and evaluation of transaction data and messages in electronic payment services is insufficient, potentially exposing users to the risk of fraud and fraudulent use. There is a need to solve these problems and provide a safe and secure user environment.

[0683] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0684] In this invention, the server includes means for receiving posted data, means for analyzing the received posted data and comparing it with past cases of online outrage, means for assessing the risk of online outrage based on the analysis results, means for sending a warning to the user if the risk of online outrage is high, means for generating alternative information to reduce the risk of online outrage, means for sending the warning and alternative information to the user terminal, means for receiving transaction data and messages and assessing the risk of fraud, means for sending a warning to the user if the risk of fraud is high, and means for generating an alternative transaction plan to reduce the risk of fraud. This makes it possible to comprehensively assess and address the risk of online outrage and fraud for SNS posted data and electronic payment transactions.

[0685] "Posted data" refers to various content such as text, images, and videos that users post on platforms such as social networking sites.

[0686] The "receiving means" refers to the technical means for obtaining data sent from a user terminal via the Internet.

[0687] The "analyzing means" is a means for analyzing the content and characteristics of received data using a specific algorithm or model.

[0688] "Flame cases" are records of past events that have caused a large-scale negative reaction on social media or online platforms.

[0689] "Comparative means" refers to a technical means of comparing the analysis results with past cases to evaluate similarities and commonalities.

[0690] "Means of evaluation" refers to technical means for determining the level of risk numerically or in categories based on the results of analysis and comparison.

[0691] "Means for sending warnings" refers to technical means for sending notifications to users to inform them of risks.

[0692] "Means for generating alternative information" refers to technical means that suggest alternative text, images, or videos to reduce risk.

[0693] A "user terminal" is a device such as a computer or smartphone that is directly used by a user.

[0694] "Transaction data" refers to specific transaction information such as buying and selling and remittances in electronic payment services.

[0695] "Fraud risk" is the possibility of loss due to illegal activities such as fraud or unauthorized access.

[0696] An "alternative transaction proposal" is an alternative proposed transaction content or method to reduce the risk of fraudulent use.

[0697] System configuration and program overview

[0698] The system for realizing the present invention comprises the following main hardware and software components.

[0699] Hardware:

[0700] Server: A computer system that performs data analysis and risk assessment.

[0701] User terminal: A user device such as a smartphone or personal computer.

[0702] software:

[0703] NLP library: A library for natural language processing. Example: SpaCy.

[0704] Image analysis models: Models that process and analyze image data. Examples: YOLO, SSD.

[0705] Video analysis technology: Technology that analyzes video data frame by frame. Example: OpenCV.

[0706] Generative AI model: A natural language generation model for generating alternatives. Example: GPT-3.

[0707] Program processing and behavior

[0708] In this system, the following processing is performed:

[0709] 1. Receiving posted data:

[0710] The server receives data sent from the user's device via a REST API, including text, images, and videos.

[0711] 2. Data Analysis:

[0712] Text analysis: The server uses NLP libraries to tokenize and morphologically analyze the text data, as well as sentiment analysis and keyword extraction.

[0713] Image Analysis: The server uses YOLO and SSD models to recognize objects in the image data and analyze it. It also uses OCR technology to extract text from the image.

[0714] Video analysis: The server divides the video data into frames and processes each frame in the same way as image analysis. The audio in the video is converted into text using ASR technology, and then the text is analyzed.

[0715] 3. Comparison with past scandals:

[0716] The server compares the analysis results with a database of past scandals and calculates a similarity score. It also compares with a database of fraud cases.

[0717] 4. Risk Assessment:

[0718] The server scores the risk of online outrage and abuse based on the similarity score, sentiment, and keywords. If the risk score is high, a warning is generated for the user.

[0719] 5. Risk Notification and Alternative Generation:

[0720] If the risk score is high, the server sends a risk warning to the user's device, and at the same time, it uses a generative AI model to generate alternatives to reduce the risk (such as safe trading suggestions or correction suggestions) and provides them to the user.

[0721] Specific examples

[0722] For example, consider the case where a user types "Today is the worst day. Everything is going wrong" in a social media post and then types "Is it okay to buy from this site?" as a transaction message.

[0723] Example prompt sentence:

[0724] The post read: "Today is the worst day. Everything is going wrong."

[0725] Transaction Comments: "Is it safe to buy from this site?"

[0726] The server analyzes this data, performs sentiment analysis and keyword extraction, and if it determines that a transaction is likely to be a scam or a controversy, it sends a warning to the user and offers alternatives such as "Today didn't go well, but I'll try again tomorrow!" or "This transaction is likely to be a scam, so we recommend purchasing from the official online store."

[0727] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0728] Step 1:

[0729] Input: Posted data and transaction data (text, images, videos) from user devices

[0730] Operation: The user device acquires the posting data and transaction data and sends them to the server. Specifically, the user enters text, images, or videos on the posting screen of an SNS or electronic payment app, and presses the send button to send the data.

[0731] Output: The server receives the data sent from the user terminal.

[0732] Step 2:

[0733] Input: Posting data and transaction data collected on the server

[0734] Operation: The server analyzes the received data. For text data, tokenization and morphological analysis are performed using an NLP library (e.g., SpaCy) to extract emotions and keywords. For image data, object detection is performed using YOLO or SSD models, and text is extracted using OCR technology. For video data, OpenCV is used to split the data into frames and perform the same processing as image analysis. The audio portion is also converted to text using ASR technology.

[0735] Output: Analysis results (emotion score, keywords, object recognition results, text extraction results)

[0736] Step 3:

[0737] Input: Analysis results

[0738] How it works: The server compares the analysis results with a database of past scandals and fraud cases. Specifically, it calculates a similarity score and evaluates how closely the results relate to past cases.

[0739] Output: Similarity score

[0740] Step 4:

[0741] Input: Similarity scores and analysis results

[0742] How it works: The server scores the risk of flame wars and abuse based on the similarity score, sentiment, and keywords. For example, if the similarity score exceeds a certain threshold, it is determined to be high risk.

[0743] Output: Risk score (flaming risk score, fraud risk score)

[0744] Step 5:

[0745] Input: Risk Score

[0746] How it works: If the risk score is high, the server generates a warning message for the user. It also uses a generative AI model (e.g., GPT-3) to generate alternatives to reduce the risk (such as suggestions for modifying the post or safe transactions). For example, if a post is overly negative, it suggests modifying it to a more positive one.

[0747] Output: Warning messages and alternatives

[0748] Step 6:

[0749] Input: Warning messages and alternatives

[0750] Operation: The server sends the generated warning message and alternatives to the user terminal, and the user terminal notifies the user of the received warning message and alternatives.

[0751] Output: A warning message and alternatives that are displayed to the user

[0752] Step 7:

[0753] Input: User confirms alternatives

[0754] How it works: The user reviews the warning message and alternatives they receive and decides whether to modify their posting or transaction. They can also submit the modifications to the server again.

[0755] Output: Modified submission and transaction data, or the original data

[0756] Through these steps, the system can assess in advance the risk of online outrage and fraudulent use of social media posting data and electronic transaction data, and provide users with appropriate warnings and alternatives.

[0757] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0758] The present invention relates to a system that analyzes posting data on social networking sites in advance and assesses the risk of a social networking site becoming a hot topic. By incorporating an emotion recognition engine, the present invention also analyzes the emotional aspects of posts, achieving a more comprehensive risk assessment. Specific embodiments of the system according to the present invention are described in detail below.

[0759] Server-side processing

[0760] 1. Receiving submitted data

[0761] The server receives post data (text, images, videos) from the user's device. When the user presses the post button, a POST request is sent from the device. The server receives this request using the REST API.

[0762] 2. Data Analysis

[0763] The server's AI analysis module analyzes the posted data, specifically as follows:

[0764] Sentence analysis:

[0765] A natural language processing (NLP) library is used to tokenize posts and perform contextual and sentiment analysis.

[0766] Image analysis:

[0767] Recognize objects in an image using an object detection model (e.g., YOLO or SSD).

[0768] It uses OCR technology to extract text from images, which can then be translated and analyzed.

[0769] Video Analysis:

[0770] The video is divided into frames, and each frame is processed in the same way as image analysis.

[0771] The audio portion of the video is converted into text using automatic speech recognition (ASR), and then sentiment analysis and related contextual analysis are performed.

[0772] 3. Emotion recognition

[0773] The server's emotion recognition engine detects the user's emotions from the received post data, as follows:

[0774] Text data: Natural language processing techniques are used to analyze emotions within the text. For example, emotions such as joy, anger, sadness, and happiness are identified, and an emotional assessment is made based on these.

[0775] Image data: Facial recognition and facial expression analysis techniques are used to identify the emotions of people in images. This is used in conjunction with image analysis techniques to assess the strength of emotions.

[0776] Video data: Emotions are recognized by analyzing the voice and facial expressions in the video. Image analysis and voice recognition technologies are combined to perform a comprehensive emotional evaluation.

[0777] 4. Comparison with past cases of online outrage

[0778] The server compares the analysis results and emotion recognition results with a database of past flaming cases, performs a similarity search, evaluates the degree of similarity of the posted data to past flaming cases, and calculates a similarity score.

[0779] 5. Risk Assessment

[0780] The server evaluates the risk of a controversy based on the analysis results, emotion recognition results, and similarity scores. For example, it calculates a risk score and determines that a score of 90 or higher is high risk, 70-89 is medium risk, and 70 or lower is low risk.

[0781] 6. Risk Notification and Alternative Generation

[0782] If the server determines that the risk of outrage is high, it generates a risk warning and alternatives. It uses a derived sentence generation model (e.g., GPT-3) to generate alternative sentences with less risk. If necessary, it also generates suggested revisions for images and videos.

[0783] 7. Sending warnings and alternatives

[0784] The server transmits the generated risk warning and alternatives to the user terminal.

[0785] Terminal side processing

[0786] 1. Create submission data

[0787] Users create posts through SNS applications, and can input text and upload images and videos.

[0788] 2. Submitting data

[0789] When the user presses the post button, the terminal sends the created post data to the server, for example, using a REST API POST request.

[0790] 3. Receive risk warnings

[0791] The device receives risk warnings sent from the server and notifies the user, including the risk assessment score and a specific warning message.

[0792] 4. Displaying alternatives

[0793] The terminal displays the alternatives sent from the server for reducing the risk on the user interface, and the user checks them and decides whether to modify the original post.

[0794] User operations

[0795] 1. Enter your post content

[0796] A user creates a post in a social networking application, for example, by typing, "Today is the worst day. Everything is going wrong."

[0797] 2. Risk confirmation

[0798] The user checks the risk assessment notification and warning message from the server on their device. For example, a warning message such as "This post contains negative sentiment and is at high risk of causing a backlash" may be displayed.

[0799] 3. Selection of alternatives

[0800] The user checks the alternatives provided by the server and decides whether to revise the post based on them. For example, an alternative suggestion might be "Today didn't go well, but I'll try my best tomorrow!"

[0801] 4. Confirmation of submission

[0802] The user can then modify the content of their post based on the displayed alternatives and finally confirm the post. The modified content is then sent back to the server and posted to the SNS.

[0803] In this way, the system of the present invention assesses the risk of a social media firestorm before a post is made, helping users to use social media safely. In addition, by combining it with an emotion recognition engine, it is possible to analyze the emotional aspects of posts, enabling more accurate risk assessment.

[0804] The processing flow will be explained below.

[0805] Server-side processing

[0806] Step 1:

[0807] The server receives post data (text, images, videos) from the user's device. When the user presses the post button, a POST request is sent from the device. The server receives this request via the REST API.

[0808] Step 2:

[0809] The server preprocesses the received submission data, tokenizing it for text data and storing image and video data in a specific directory.

[0810] Step 3:

[0811] The server's AI analysis module analyzes the posted data using the following steps:

[0812] Text analysis: Uses natural language processing (NLP) libraries to tokenize text and perform contextual and sentiment analysis.

[0813] Image analysis: Recognize objects and text in images using object detection models (e.g., YOLO or SSD).

[0814] Video analysis: Videos are divided into frames, and each frame is analyzed in the same way as image analysis. The audio portion of the video is converted into text using automatic speech recognition (ASR), and this is then analyzed.

[0815] Step 4:

[0816] The emotion recognition engine of the server recognizes the emotion of the user based on the received posting data.

[0817] Text data: Natural language processing techniques are used to analyze sentiment within the text, for example, identifying positive, negative, or neutral sentiment.

[0818] Image data: Using facial recognition and facial expression analysis techniques, we identify the emotions of people in images.

[0819] Video data: Analyzes the voice and facial expressions in the video, and integrates multiple emotional data to evaluate the user's overall emotions.

[0820] Step 5:

[0821] Based on the analysis results and emotion recognition results, the server compares the posted data with a database of past flaming cases, performs a similarity search, and scores the degree of similarity with past flaming cases.

[0822] Step 6:

[0823] The server evaluates the risk of a controversy based on the analysis results, emotion recognition results, and similarity scores. A score of 90 or above is considered high risk, 70-89 is considered medium risk, and 70 or below is considered low risk.

[0824] Step 7:

[0825] If the risk of a post being criticized is high, the server will notify the user of the risk and generate alternatives. For example, it will use a derived sentence generation model (e.g., GPT-3) to generate alternative sentences with less risk. In some cases, it will also generate suggested revisions for images and videos.

[0826] Step 8:

[0827] The server sends the generated risk warning and alternatives to the user's device, and if an error occurs, it records it in a log.

[0828] Terminal side processing

[0829] Step 1:

[0830] A user creates a post through a social networking application, for example, by inputting text and uploading images and videos.

[0831] Step 2:

[0832] When the user presses the post button, the terminal sends the created post data to the server, for example, using a POST request.

[0833] Step 3:

[0834] The device waits for a response from the server and receives a risk warning and alternatives.

[0835] Step 4:

[0836] The device will display risk warnings and alternatives to the user, for example as a pop-up message or notification.

[0837] User operations

[0838] Step 1:

[0839] A user inputs a post into a social networking application, for example, a sentence such as "Today is the worst day. Everything is going wrong."

[0840] Step 2:

[0841] When the user presses the post button, the post data is sent to the server.

[0842] Step 3:

[0843] The user will see a risk warning and alternatives from the server. For example, a warning message such as "This post contains negative sentiment and is at high risk of causing controversy" will be displayed.

[0844] Step 4:

[0845] The user checks the alternatives and decides whether to revise the post based on them. For example, an alternative suggestion might be "Today didn't go well, but I'll try my best tomorrow!"

[0846] Step 5:

[0847] Once the user has finalized the content of the post and pressed the post button again, the revised post data will be resent to the server and posted on the SNS.

[0848] Example 2

[0849] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0850] There is a need to provide an environment where users can safely use SNS by assessing the risk of data posted on SNS becoming a hot topic in advance. Conventional technologies simply filter the content of posts, without performing comprehensive analysis including emotional aspects and images and videos, making it difficult to accurately assess risk. The purpose of this invention is to solve these problems and provide a more comprehensive and accurate risk assessment system.

[0851] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving post data, means for analyzing the received post data using natural language processing and image analysis techniques, means for performing emotion recognition based on the analysis results, means for comparing the emotion recognition results and the analysis results with a database of past flame war cases, means for evaluating the risk of a flame war based on the comparison results, means for sending a warning to the user when the risk of a flame war is high, means for generating alternative information to reduce the risk of a flame war using a generative AI model, and means for sending the warning and alternative information to the user terminal. This makes it possible to provide a highly accurate evaluation of the risk of a flame war and appropriate alternatives before a user posts on an SNS.

[0852] "Posted data" refers to data in the form of text, images, and videos posted by users to online platforms such as social networking sites.

[0853] "Natural language processing" is a technology that enables computers to understand, interpret, and generate human language, typically involving tokenization, contextual analysis, and sentiment analysis.

[0854] "Image analysis" refers to the use of computer vision techniques to recognize and analyze objects and text in images. Examples include object detection and OCR.

[0855] "Emotion recognition" is a technology that uses natural language processing and image analysis technologies to detect emotions from text, images, and videos.

[0856] "Flame risk" is an evaluation index of the likelihood that posted data will receive a large amount of negative reactions or criticism on social media.

[0857] The "database of past social media outrage cases" is a database of past social media outrage cases, and is a data set used for risk assessment.

[0858] A "generative AI model" is an artificial intelligence model that generates new data from given input data. A specific example is GPT-3.

[0859] "Similarity search" is a technique for evaluating how similar certain data is to given reference data (in this case, past online scandals).

[0860] A "warning" is a message that notifies the user that there is a high risk of the posted data causing a firestorm.

[0861] "Alternative information" refers to modifications or alternatives suggested by generative AI models to reduce the risk of backlash.

[0862] The present invention is a system that analyzes social media posting data in advance and assesses the risk of a social media outcry. This system combines emotion recognition technology and generative AI models to provide a more comprehensive risk assessment and alternative solutions. Specific embodiments of the present invention are described in detail below.

[0863] Server-side processing

[0864] The server first receives the posted data (text, images, videos) sent from the user's device. This reception process uses a POST request using the REST API. For example, data is received through an API gateway with a set endpoint. The server uses a cloud platform to achieve scalable processing.

[0865] The server's AI analysis module then analyzes the received data using the following techniques:

[0866] Use a natural language processing (NLP) library (e.g., spaCy or BERT) to tokenize the text data and perform contextual and sentiment analysis.

[0867] It uses object detection models (e.g., YOLO and SSD) to identify objects in images, and OCR technology to extract text from images for translation and sentiment analysis.

[0868] In video analysis, the video is divided into frames and each frame is processed in the same way as image analysis. The audio portion of the video is converted into text using automatic speech recognition (ASR), and then emotion and context analysis are performed.

[0869] Emotion recognition is performed to detect user emotions from received posting data. This process uses natural language processing technology to detect emotions in text and identify emotions such as joy, anger, sadness, and happiness. It also uses facial recognition and facial expression analysis technology to analyze the emotions of people in images and videos.

[0870] Based on the analysis results and emotion recognition results, the server compares them with a database of past flaming cases. Using a similarity search technique, the server evaluates how similar the posted data is to past flaming cases and calculates a similarity score. Specifically, the comparison uses techniques such as Cosine similarity.

[0871] The server then assesses the risk of a controversy based on the analysis results, emotion recognition results, and similarity scores. A score of 90 or higher is considered high risk, 70-89 is considered medium risk, and 70 or lower is considered low risk.

[0872] If the risk of outrage is high, the server generates a risk warning and alternative suggestions. Using a generative AI model (e.g., GPT-3), it generates alternative text to reduce the risk. If necessary, it also generates suggested edits for images and videos. The generated alternative suggestions and risk warnings are converted into JSON format and sent to the user's device.

[0873] Terminal side processing

[0874] First, a user creates a post through a social networking application. They enter text and upload images and videos. When the user presses the post button, this post data is sent to the server via a REST API POST request.

[0875] When a risk warning is sent from the server, the device receives it and notifies the user. The risk assessment score and specific warning message are displayed on the interface, and the device also displays the alternatives sent from the server. The user can review the alternatives and decide whether to revise the original post.

[0876] User operations

[0877] A user creates a post in a social networking application. For example, the user enters the sentence, "Today is the worst day. Everything is going wrong." The server determines that this post contains strong negative emotions and sends a warning message to the device indicating a high risk of it starting a flame war. The warning message reads, "This post contains negative emotions and there is a high risk of it starting a flame war."

[0878] The user also checks alternative suggestions provided by the server. For example, an alternative sentence such as "Today didn't go well, but I'll try my best tomorrow!" is presented. The user can then revise the post based on this and finally confirm the post. The revised post is then sent back to the server and posted to the SNS.

[0879] This system allows users to use social media safely and assess the risk of a social media firestorm in advance. Furthermore, by using emotion recognition technology, it is possible to analyze the emotional aspects of posts and improve the accuracy of risk assessment.

[0880] Specific examples

[0881] For example, if a user tries to post "Today is the worst day. Everything is going wrong," the server analyzes it and determines that it is a post with a strong negative sentiment. If it is assessed as having a high risk of causing a flame war, the server generates an alternative suggestion, such as "Today didn't go well, but I'll try my best tomorrow!" and presents it to the user.

[0882] Prompt Sentence Examples

[0883] Input: Today is the worst day. Everything is going wrong.

[0884] Output: Things didn't go well today, but I'll try my best tomorrow!

[0885] This prompt sentence can be used to train a generative AI model (e.g., GPT-3) to generate a less risky alternative sentence.

[0886] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0887] Step 1: Receive post data

[0888] The server receives post data from the user's device. When a user presses the post button in the SNS application, a POST request is sent from the device. The input is the text, image, and video data created by the user, and the output is the post data received by the server. Specifically, the server receives this request using a REST API and retrieves the data through an API gateway with a configured endpoint.

[0889] Step 2: Data analysis

[0890] The AI ​​analysis module on the server analyzes the received post data. Specifically,

[0891] Text analysis: Using a natural language processing (NLP) library (e.g., spaCy or BERT), text data is tokenized and context and sentiment analysis is performed. The input is text data, and the output is the analysis results.

[0892] Image analysis: Identify objects in images using object detection models (such as YOLO or SSD). Also, extract text from images using OCR technology and perform sentiment analysis. The input is image data, and the output is the analysis results.

[0893] Video analysis: Videos are divided into frames, and each frame is processed in the same way as image analysis. The audio portion of the video is converted into text using automatic speech recognition (ASR), and emotion and context analysis is performed. The input is video data, and the output is the analysis results data.

[0894] Step 3: Emotion Recognition

[0895] The server's emotion recognition engine detects the user's emotions from the analyzed post data. Specifically,

[0896] Text data: Detect emotions in text using natural language processing techniques. The input is the result of text analysis, and the output is the emotion recognition result of the text.

[0897] Image data: Using facial recognition and facial expression analysis technology, we identify the emotions of people in an image. The input is the result of image analysis, and the output is the emotion recognition result of the image.

[0898] Video data: Analyzes the voice and facial expressions in the video to perform a comprehensive emotional evaluation. The input is the result of video analysis data, and the output is the emotion recognition results for the video.

[0899] Step 4: Compare with past flame war cases

[0900] The server compares the analysis results and emotion recognition results with a database of past flaming cases. Specifically, it uses a similarity search method to evaluate how similar the posted data is to past flaming cases. The input is the analysis results and emotion recognition results, and the output is a similarity score. Cosine similarity, for example, is used to evaluate the similarity.

[0901] Step 5: Risk assessment

[0902] The server evaluates the risk of a controversy based on the analysis results, emotion recognition results, and similarity score. It calculates a risk score and classifies a score of 90 or above as high risk, 70-89 as medium risk, and 70 or below as low risk. The inputs are the similarity score, emotion recognition results, and analysis results, and the output is the risk assessment score and its judgment result.

[0903] Step 6: Risk notification and alternative generation

[0904] If the server determines that there is a high risk of outrage, it generates a risk warning and alternatives. It uses a generative AI model (e.g., GPT-3) to generate alternative text with less risk. If necessary, it also generates suggested modifications for images and videos. The input is the risk assessment score and the judgment result, and the output is a risk warning message and alternatives.

[0905] Step 7: Sending warnings and alternatives

[0906] The server sends the generated risk warning and alternatives to the user's device. The input is the risk warning message and alternatives, and the output is the data to be sent to the user's device. The risk warning and alternatives are converted to JSON format and sent to the user's device via a REST API.

[0907] Step 8: Receive risk warnings

[0908] The terminal receives the risk warning sent from the server and notifies the user. The input is the risk warning message and alternatives sent from the server, and the output is the notification to the user interface.

[0909] Step 9: View alternatives

[0910] The terminal displays the alternatives sent from the server on the user interface. The user checks them and decides whether to modify the original post. The input is the alternatives, and the output is the display content presented to the user.

[0911] Step 10: User Edits to Posts

[0912] The user modifies the post based on the alternatives provided by the server. The input is the alternatives and the original post, and the output is the modified post.

[0913] Step 11: Finalize and submit your revised post

[0914] The user confirms the revised content of the post and sends it back to the server. The input is the revised content of the post, and the output is the final post data sent to the server. The server receives this revised post and posts it to the SNS.

[0915] (Application example 2)

[0916] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0917] In recent years, the risk of user posts on social media and online platforms causing online outrage has become a social issue. In particular, for brick-and-mortar stores, there is a growing risk that negative customer feedback spreading online could damage their reputation and negatively impact sales. It is also important to quickly analyze customer feedback and provide risk assessments and appropriate countermeasures. The present invention aims to solve these problems by providing a system that enables brick-and-mortar stores to analyze customer feedback and quickly take appropriate countermeasures.

[0918] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0919] In this invention, the server includes means for receiving posted data, means for analyzing the received posted data and comparing it with past cases of online flaming, means for assessing the risk of online flaming based on the analysis results, means for sending a warning to the user when the risk of online flaming is high, means for generating alternative information to reduce the risk of online flaming, means for sending the warning and alternative information to the user terminal, means for analyzing customer feedback at the physical store, means for performing sentiment analysis of the posted data and assessing the risk, and means for generating alternatives that arise when the risk is high. This enables quick and accurate analysis of customer feedback and risk assessment, and appropriate action to protect the reputation of the physical store.

[0920] "Posted data" refers to information such as text, images, and videos that users create and send on social media or online platforms.

[0921] "Means for receiving" refers to the functions and protocols that allow the server to receive posted data sent from the user terminal.

[0922] "Means of analysis" refers to the process of analyzing the content of received posted data using natural language processing and image analysis technology.

[0923] "Past cases of online outrage" refer to cases in the past where a user's post on a social media or online platform sparked criticism or negative reactions.

[0924] "Means of comparison" refers to the algorithms and methods used to evaluate the similarity of newly received posted data with past cases of online outrage.

[0925] "Flame risk" refers to the likelihood that a particular post will provoke criticism or negative reactions in the online community.

[0926] "Sentiment analysis" refers to the process of identifying emotional components in text or images using natural language processing and image recognition techniques.

[0927] "Risk assessment" refers to the process of evaluating the degree of risk of a post causing a controversy using numerical or categorized terms based on the analysis of received post data.

[0928] "Means for sending warnings" refers to the function of sending warning messages to users when there is a high risk of a social media outburst.

[0929] "Alternative information" refers to safer options, such as text or images that have been modified from the original post to reduce the risk of a backlash.

[0930] "Means for generating alternatives" refers to a process that uses generative AI models or other technologies to automatically create safe alternative information.

[0931] "User terminal" refers to devices such as smartphones, tablets, and PCs used to access platforms such as social networking sites.

[0932] "Brick and mortar store" refers to a store that offers goods or services in a physical location.

[0933] "Customer feedback" refers to customer experiences, opinions, and evaluations expressed in the form of posts in physical stores.

[0934] The present invention relates to a system that analyzes posted data in advance and evaluates the risk of a social media firestorm. This system is primarily intended to analyze customer feedback in brick-and-mortar stores, and is designed to prevent negative feedback from damaging the store's reputation. Specific embodiments are described below.

[0935] Server-side processing

[0936] 1. Receiving submitted data

[0937] The server receives the posted data (text, images, videos) sent from the user's device. When the user sends feedback, a POST request is sent from the user's device via the REST API. The server receives this request and prepares to analyze the posted data.

[0938] 2. Data Analysis

[0939] The server program analyzes the submitted data using the following hardware and software:

[0940] Natural Language Processing (NLP) library: Uses TextBlob to tokenize posts and perform contextual and sentiment analysis.

[0941] Image Analysis: Extract text in images using OpenCV and pytesseract and perform analysis on the same.

[0942] The analysis results are compared with a database of past online outrage cases and a similarity search is performed.

[0943] 3. Emotion recognition

[0944] The server analyzes the sentiment of the posted data through the following process:

[0945] Text data: Natural language processing techniques are used to identify sentiment within the text.

[0946] Image data: Use OCR technology to extract text from images and analyze their sentiment.

[0947] 4. Risk assessment and alternative generation

[0948] The server evaluates the risk of a controversy based on the analysis results and emotion recognition results. If the risk is determined to be high, it uses a generative AI model (e.g., GPT-3) to generate alternatives to reduce the risk.

[0949] Terminal side processing

[0950] The user terminal provides the following functions:

[0951] Feedback submission: Users can enter feedback in the form of text, images, or videos using their smartphones.

[0952] Display risk warning: Receives risk warnings sent from the server and notifies the user.

[0953] Displaying alternatives: The alternatives sent from the server are displayed in the user interface, allowing the user to choose whether to revise the post.

[0954] Specific examples

[0955] For example, if a customer posts feedback saying, "Today's customer service was terrible," this post is sent to the server. On the server side, negative sentiment is detected using TextBlob and evaluated as high risk. In response, a generative AI model is used to generate an alternative suggestion, such as, "Today was a bit disappointing, but I look forward to your next visit," and this is presented to the user.

[0956] Prompt Sentence Examples

[0957] "Generate positive alternatives for the user-submitted feedback, 'The customer service today was terrible.'"

[0958] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0959] Step 1:

[0960] The user uses their smartphone to enter feedback, which can be in the form of text, images, or videos. The entered feedback might be something like "Today is the worst day. Everything is going wrong." When the user presses the submit button, the device sends the feedback data to the server as a POST request.

[0961] Step 2:

[0962] The server receives the posted data sent from the user's device. The received data can be text, images, or videos, and is sent via the REST API. The server temporarily stores the received data.

[0963] Step 3:

[0964] The server analyzes the received post data. For text data, it uses TextBlob to tokenize the data, analyzes the context, and performs sentiment analysis. The input for analysis is the post text, and the output is the sentiment polarity (positive, negative, neutral). For example, "Today is the worst day. Everything is going wrong." is analyzed as negative.

[0965] Step 4:

[0966] The server analyzes the image data. It extracts the text from the image using OpenCV and performs OCR analysis using pytesseract. The input for the analysis is the posted image, and the extracted text is again subjected to sentiment analysis using TextBlob. The output is the text from the image and the results of the sentiment analysis.

[0967] Step 5:

[0968] The server analyzes the video data. It divides the video into frames and performs image analysis on each frame. It also converts the audio portion of the video into text using automatic speech recognition (ASR), and performs sentiment analysis on that text. The input for the analysis is the posted video, and the output is the text for each frame and the sentiment analysis results.

[0969] Step 6:

[0970] The server compares the analysis results with a database of past flaming cases. It uses a similarity algorithm to evaluate how similar the analysis data is to past flaming cases. The input is the analysis results and past flaming cases, and the output is a similarity score.

[0971] Step 7:

[0972] The server evaluates the risk of a controversy based on the analysis results and the similarity score. The emotional polarity and similarity scores are combined to calculate an overall risk score. The output is a risk score; for example, a risk score of 90 or higher is considered high risk.

[0973] Step 8:

[0974] If the risk of a firestorm is determined to be high, the server sends a risk warning to the user's device. The warning includes a risk score and a specific warning message. The input is the risk score, and the output is the warning message.

[0975] Step 9:

[0976] To generate alternatives, the server uses a generative AI model. For example, it uses a prompt such as "Generate a positive alternative to the user-posted feedback, 'Today is the worst day. Everything is going wrong.'" The input is the original post text and the prompt, and the output is the alternative text. For example, an alternative such as "Today didn't go well, but let's try our best tomorrow!" is generated.

[0977] Step 10:

[0978] The server sends the generated alternatives to the user's terminal. The user can check the alternatives and choose whether to correct the feedback. The input is the alternative text, and the output is a notification to the user's terminal.

[0979] Step 11:

[0980] The user then takes the suggested alternatives into consideration and finally submits the revised feedback, which is then sent back to the server and posted on the SNS platform.

[0981] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0982] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0983] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0984] [Third embodiment]

[0985] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0986] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0987] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0988] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0989] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0990] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0991] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0992] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0993] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0994] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0995] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0996] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0997] The present invention relates to a system for analyzing posted data on a social networking site in advance and assessing the risk of a social networking site becoming a hot topic. Specific embodiments of the system according to the present invention will be described in detail below.

[0998] Server-side processing

[0999] 1. Receiving submitted data

[1000] The server receives the posted data (content such as text, images, and videos) sent from the user's device. For example, it retrieves the data using a POST request using a REST API.

[1001] 2. Data Analysis

[1002] The AI ​​analysis module on the server analyzes the data in the following steps:

[1003] Sentence analysis:

[1004] Use a natural language processing (NLP) library to tokenize the text and perform morphological analysis.

[1005] Extract relevant keywords and emotional expressions from the text.

[1006] Image analysis:

[1007] Recognize objects in an image using an object detection model (e.g., YOLO or SSD).

[1008] The text in the image is extracted using OCR technology and then further analyzed.

[1009] Video Analysis:

[1010] The video is divided into frames, and each frame is processed in the same way as image analysis.

[1011] The audio portion of the video is converted into text using automatic speech recognition (ASR) technology, and text analysis is performed.

[1012] 3. Comparison with past cases of online outrage

[1013] The server compares the analysis results with a database of existing cases of online outrages and evaluates the risk of online outrage based on the degree of similarity. For example, it searches the database of past cases of online outrages and calculates a similarity score.

[1014] 4. Risk Assessment

[1015] The server scores the risk of a controversy based on the similarity score and the extracted sentiment and keywords. The server then determines the degree of risk based on the assessed risk score. For example, a risk score of 90 or above is high risk, 70-89 is medium risk, and anything below that is low risk.

[1016] 5. Risk Notification and Alternative Generation

[1017] The server sends a warning to users if the risk score is high. To support safe posting, the server uses a derived text generation model (e.g., GPT-3) to generate alternative text and image / video revision suggestions with less risk and sends them to the user's device.

[1018] Terminal side processing

[1019] 1. Create submission data

[1020] Users use SNS applications to create posts, including text they type and images and videos they upload.

[1021] 2. Submitting data

[1022] When the user presses the post button, the terminal sends the created post data to the server, for example, using a REST API POST request.

[1023] 3. Receive risk warnings

[1024] The device receives risk warnings sent from the server and notifies the user, including the risk assessment score and a specific warning message.

[1025] 4. Displaying alternatives

[1026] The terminal displays the alternatives sent from the server for reducing the risk on the user interface, and the user checks them and decides whether to modify the original post.

[1027] User operations

[1028] 1. Enter your post content

[1029] A user creates a post on a social networking app, for example, by typing a sentence such as, "Today is the worst day. Everything is going wrong."

[1030] 2. Risk confirmation

[1031] The user checks the risk assessment notification and warning message from the server on their device. For example, a warning message such as "This post contains negative sentiment and is at high risk of causing a backlash" may be displayed.

[1032] 3. Selection of alternatives

[1033] The user checks the alternatives provided by the server and decides whether to revise the post based on the alternatives or to post the original post as is. For example, an alternative suggestion might be "Today didn't go well, but I'll try my best tomorrow!"

[1034] 4. Confirmation of submission

[1035] The user can then modify the content of their post based on the displayed alternatives and finally confirm the post. The modified content is then sent back to the server and posted to the SNS.

[1036] In this way, the system of the present invention evaluates the risk of a controversy before posting, helping users to use SNS safely.

[1037] The processing flow will be explained below.

[1038] Server-side processing

[1039] Step 1:

[1040] The server receives post data (text, images, videos) from the user's device. When the user presses the post button, a POST request is sent from the device. The server receives this request using the REST API.

[1041] Step 2:

[1042] The server preprocesses the submitted data it receives, for example tokenizing text data and storing image and video data in specific directories.

[1043] Step 3:

[1044] The server's AI analysis module analyzes the posted data.

[1045] Text analysis: Using a natural language processing (NLP) library, posts are tokenized and semantic and sentiment analysis is performed.

[1046] Image analysis: Recognize objects in images using object detection models (e.g., YOLO and SSD), and extract text from images using hooked OCR technology.

[1047] Video analysis: Videos are divided into frames, and each frame is analyzed in the same way as image analysis. Audio in the video is converted into text using automatic speech recognition (ASR), and related sentence analysis is performed.

[1048] Step 4:

[1049] The server compares the analysis results with a database of past flaming cases, performs a similarity search, evaluates the degree to which the posted data is similar to past flaming cases, and calculates a similarity score.

[1050] Step 5:

[1051] The server evaluates the risk of a controversy based on the analysis results and the similarity score. For example, it calculates a risk score, determining that a score of 90 or higher is high risk, 70-89 is medium risk, and 70 or lower is low risk.

[1052] Step 6:

[1053] If the server determines that a post is at high risk of causing an uproar, it generates a risk warning and alternatives. It uses a derived sentence generation model (e.g., GPT-3) to generate alternative sentences with less risk. If necessary, it also generates suggested revisions for images and videos.

[1054] Step 7:

[1055] The server sends risk warnings and alternative solutions to the user's device, and keeps a log containing details of any errors that occur.

[1056] Terminal side processing

[1057] Step 1:

[1058] Users create posts through SNS applications, and can input text and upload images and videos.

[1059] Step 2:

[1060] When the user presses the submit button, the device sends the submission data to the server, for example, using a REST API POST request.

[1061] Step 3:

[1062] The device waits for a response from the server, and if the server returns a risk warning and alternatives, it receives them.

[1063] Step 4:

[1064] The device displays the risk warning and alternatives received from the server to the user, for example, by a pop-up message or a notification box.

[1065] Step 5:

[1066] Users review the risk warnings and alternative suggestions and decide whether to revise their original post. If revisions are necessary, they reconstruct the content using the alternative sentences and suggested revisions as a reference.

[1067] Step 6:

[1068] When the user decides on the final post and presses the post button again, the terminal resends the corrected post data to the server.

[1069] User Action

[1070] Step 1:

[1071] A user types or uploads a post to a social networking application, for example, typing the sentence "Today is the worst day. Everything is going wrong."

[1072] Step 2:

[1073] When the user presses the submit button, the data is sent to the server.

[1074] Step 3:

[1075] The user then checks the risk warning and alternatives from the server. For example, the user may be presented with a warning message such as "This post contains negative emotions and is at high risk of causing a backlash," along with an alternative sentence such as "Today didn't go well, but I'll try my best tomorrow!"

[1076] Step 4:

[1077] The user decides whether to revise the post based on the alternative suggestions, and if so, checks the new post.

[1078] Step 5:

[1079] Once the user has finalized the content of the post and pressed the post button again, the final post data will be posted on the SNS.

[1080] Example 1

[1081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1082] Posting on social media is an important way to share information with many people, but if inappropriate content is included, it can lead to outrage and excessive criticism. In addition, since there are limited ways for users to know about such risks in advance, they may be cautious about using social media. Therefore, a system is needed that evaluates the risk of outrage before posting and suggests safe ways to post.

[1083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1084] In this invention, the server includes means for receiving post data from a user terminal, means for analyzing the received post data using natural language processing, object recognition, and voice recognition, means for comparing the received post data with past flaming cases based on keywords and emotional expressions extracted by the analysis, means for scoring and assessing the risk of flaming based on the comparison results, means for sending a warning to the user if the risk of flaming is high, means for generating alternative information to reduce the risk of flaming, and means for sending the warning and alternative information to the user terminal. This allows users to assess the risk of flaming before posting and learn safe posting methods.

[1085] A "user terminal" is a device operated by a user, and includes a smartphone, tablet, PC, etc.

[1086] "Posted data" refers to content such as text, images, and videos that users share on social media.

[1087] "Server" refers to a high-performance computer system installed at a remote location that analyzes and processes data received from user devices.

[1088] "Natural language processing" is a technology for processing human language using computers, and includes text tokenization, morphological analysis, sentiment analysis, etc.

[1089] "Object recognition" refers to the technology of identifying objects in images and classifying them.

[1090] "Speech recognition" refers to the technology of converting voice data into text.

[1091] "Keywords" are words or phrases that are particularly important in a sentence or piece of content.

[1092] "Sentiment expression" refers to elements in text that indicate sentiment (positive, negative, neutral, etc.) analyzed by natural language processing.

[1093] "Flame war cases" include data that refer to large-scale negative reactions caused by inappropriate posts on social media in the past.

[1094] The "similarity score" is a numerical value that indicates the similarity between the target data and past flame war case data, and is used for risk assessment.

[1095] "Scoring" refers to a method of quantifying risk or performance based on specific criteria.

[1096] "Warning" refers to a caution message to inform the user of a high risk.

[1097] "Alternative information" refers to safe posts and suggested revisions to reduce the risk of outrage.

[1098] MODE FOR CARRYING OUT THE INVENTION

[1099] The present invention relates to a system for analyzing data posted to a social networking site in advance and assessing the risk of a social networking site becoming a hot topic. Specific embodiments of the present invention will be described in detail below.

[1100] Server Processing

[1101] The server receives the posted data (text, images, videos, and other content) sent from the user's device. Specifically, it uses a mechanism to receive data via a POST request via the REST API.

[1102] The server analyzes the received posted data using the following methods. First, it uses a natural language processing (NLP) library (e.g., spaCy) to tokenize the text data and extract emotional expressions and keywords. Next, it uses an object recognition model (e.g., YOLO) to recognize objects within the image. It also uses OCR technology (e.g., Tesseract OCR) to extract text from the image and further analyze it. It then divides the video data into frames, each of which is processed in the same way as image analysis. It then uses automatic speech recognition (ASR) technology to convert the audio portion of the video into text, which is then analyzed.

[1103] The server compares the analysis results with a database of past flaming cases. It calculates a similarity score from the database and evaluates the risk. For this purpose, it uses a scoring method; for example, a similarity score of 90 or higher with past cases is rated as high risk, 70-89 as medium risk, and anything below that is rated as low risk.

[1104] If the server determines that there is a high risk of outrage, it sends a warning to the user. It also uses a generative AI model (e.g., GPT-3) to generate alternative information (suggested revisions to text, images, and videos) with less risk and sends them to the user's device.

[1105] Terminal handling

[1106] Users use SNS applications to create posts, and the text they enter and the images and videos they upload become the post data.

[1107] When the user presses the submit button, the device sends the created post data to the server using a REST API POST request.

[1108] The device receives the risk warning sent from the server and notifies the user, displaying the risk assessment score and a specific warning message.

[1109] The terminal displays the alternatives for reducing the risk sent from the server to the user, who then checks the alternatives and decides whether to revise the content of the post and repost it.

[1110] User operations

[1111] A user inputs content to be posted on the SNS application to create post data. For example, the user inputs a sentence such as "Today is the worst day. Everything is going wrong."

[1112] The user checks the risk assessment notification and warning message from the server on their device. For example, a warning message such as "This post contains negative emotions and is at high risk of causing a backlash" may be displayed.

[1113] The user checks the alternatives provided by the server and modifies the content of the post based on them. For example, an alternative suggestion such as "Today didn't go well, but I'll try my best tomorrow!" may be presented.

[1114] Specific examples

[1115] Example prompt sentence:

[1116] "This sentence contains negative content. Please consider the following alternatives:

[1117] 1. Things didn't go well today, but I'll try my best tomorrow!

[1118] 2. Sometimes you have bad days, but that's life.

[1119] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1120] Step 1: Receiving posted data

[1121] The server receives posted data from user devices. Input includes content such as text, images, and videos. Specifically, data is received via a POST request via the REST API. The output is the received posted data itself.

[1122] Specific behavior:

[1123] When a user presses the "Post" button on their device, the device sends a POST request to the server, and text, image, and video data are sent to the server.

[1124] Step 2: Text analysis

[1125] The server analyzes the received text data using a natural language processing (NLP) library. The input is the received text data, and the output is the analysis results such as keywords and emotional expressions.

[1126] Specific behavior:

[1127] The server uses the spaCy library to tokenize the text and extract important keywords and sentiments. For example, from the sentence "Today is the worst day. Everything is going wrong," it extracts keywords like "worst" and "going wrong."

[1128] Step 3: Image analysis

[1129] The server analyzes the received image data using an object recognition model (e.g., YOLO). The input is the received image data, and the output is the analysis results of objects and text in the image.

[1130] Specific behavior:

[1131] The server uses the YOLO model to identify people and objects in the image and extract their location information, and Tesseract OCR to extract text from the image and analyze important keywords such as "warning" and "caution."

[1132] Step 4: Video Analysis

[1133] The server divides the received video data into frames and processes each frame in the same way as image analysis. The input is the received video data, and the output is the image analysis results for each frame and the speech-to-text results.

[1134] Specific behavior:

[1135] The server uses OpenCV to split the video into frames and analyzes each frame with the YOLO model. At the same time, the audio portion is converted to text using the Google Speech-to-Text API and subjected to NLP processing.

[1136] Step 5: Compare with past flame wars

[1137] The server compares the analysis results with a database of past flaming cases. The input is the analyzed keywords and emotional expressions, and the output is a similarity score.

[1138] Specific behavior:

[1139] The server uses TfidfVectorizer to calculate the similarity score between the analysis results and past flaming cases. For example, if the keywords "worst" and "doesn't work" match many of the past flaming cases, a high similarity score will be calculated.

[1140] Step 6: Risk assessment

[1141] The server scores the flame risk based on the similarity score. The input is the similarity score, and the output is the estimated flame risk score.

[1142] Specific behavior:

[1143] The server applies a scoring algorithm to calculate a risk score. For example, a similarity score of 90 or higher is considered "high risk," and a score between 70 and 89 is considered "medium risk."

[1144] Step 7: Sending an alert

[1145] The server sends a warning to the user if the risk score is high. The input is the assessed risk score and the output is the warning message.

[1146] Specific behavior:

[1147] The server generates a warning message based on the risk score and sends it to the user's device. For example, a message might say, "This post contains negative emotions and is at high risk of causing a backlash."

[1148] Step 8: Generate alternative information

[1149] To reduce the risk of outrage, the server generates alternative information using a generative AI model (e.g., GPT-3). The input is the initial post data, and the output is the generated alternative sentences or suggested revisions.

[1150] Specific behavior:

[1151] If the initial post is "Today is the worst day. Everything is going wrong," GPT-3 will be used to generate an alternative post that says, "Some things didn't go well today, but I'll try my best tomorrow!"

[1152] Step 9: Notification of warnings and alternatives

[1153] The terminal receives the warning and alternatives sent from the server and notifies the user of them. The input is the warning message and alternatives sent from the server, and the output is the notified message.

[1154] Specific behavior:

[1155] The device will use notifications to display warnings and alternatives to the user, including a risk assessment score and specific warning messages.

[1156] Step 10: Edit and finalize your submission

[1157] The user modifies the posted content by referring to the alternatives and finally confirms it. The input is the alternatives and the initial posted data, and the output is the modified posted data.

[1158] Specific behavior:

[1159] The user then modifies the content of the post based on the alternative suggestions and clicks the "Post" button again to send the modified post to the server, where it is evaluated again and posted on the SNS.

[1160] (Application example 1)

[1161] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1162] Systems that can detect the risk of unexpected outrage or fraudulent use and enable users to take appropriate action are needed not only for data posted on social media but also for electronic payment services. However, current systems have limited scope for risk assessment, and do not adequately deter fraudulent use or prevent outrage. In particular, analysis and evaluation of transaction data and messages in electronic payment services is insufficient, potentially exposing users to the risk of fraud and fraudulent use. There is a need to solve these problems and provide a safe and secure user environment.

[1163] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1164] In this invention, the server includes means for receiving posted data, means for analyzing the received posted data and comparing it with past cases of online outrage, means for assessing the risk of online outrage based on the analysis results, means for sending a warning to the user if the risk of online outrage is high, means for generating alternative information to reduce the risk of online outrage, means for sending the warning and alternative information to the user terminal, means for receiving transaction data and messages and assessing the risk of fraud, means for sending a warning to the user if the risk of fraud is high, and means for generating an alternative transaction plan to reduce the risk of fraud. This makes it possible to comprehensively assess and address the risk of online outrage and fraud for SNS posted data and electronic payment transactions.

[1165] "Posted data" refers to various content such as text, images, and videos that users post on platforms such as social networking sites.

[1166] The "receiving means" refers to the technical means for obtaining data sent from a user terminal via the Internet.

[1167] The "analyzing means" is a means for analyzing the content and characteristics of received data using a specific algorithm or model.

[1168] "Flame cases" are records of past events that have caused a large-scale negative reaction on social media or online platforms.

[1169] "Comparative means" refers to a technical means of comparing the analysis results with past cases to evaluate similarities and commonalities.

[1170] "Means of evaluation" refers to technical means for determining the level of risk numerically or in categories based on the results of analysis and comparison.

[1171] "Means for sending warnings" refers to technical means for sending notifications to users to inform them of risks.

[1172] "Means for generating alternative information" refers to technical means that suggest alternative text, images, or videos to reduce risk.

[1173] A "user terminal" is a device such as a computer or smartphone that is directly used by a user.

[1174] "Transaction data" refers to specific transaction information such as buying and selling and remittances in electronic payment services.

[1175] "Fraud risk" is the possibility of loss due to illegal activities such as fraud or unauthorized access.

[1176] An "alternative transaction proposal" is an alternative proposed transaction content or method to reduce the risk of fraudulent use.

[1177] System configuration and program overview

[1178] The system for realizing the present invention comprises the following main hardware and software components.

[1179] Hardware:

[1180] Server: A computer system that performs data analysis and risk assessment.

[1181] User terminal: A user device such as a smartphone or personal computer.

[1182] software:

[1183] NLP library: A library for natural language processing. Example: SpaCy.

[1184] Image analysis models: Models that process and analyze image data. Examples: YOLO, SSD.

[1185] Video analysis technology: Technology that analyzes video data frame by frame. Example: OpenCV.

[1186] Generative AI model: A natural language generation model for generating alternatives. Example: GPT-3.

[1187] Program processing and behavior

[1188] In this system, the following processing is performed:

[1189] 1. Receiving posted data:

[1190] The server receives data sent from the user's device via a REST API, including text, images, and videos.

[1191] 2. Data Analysis:

[1192] Text analysis: The server uses NLP libraries to tokenize and morphologically analyze the text data, as well as sentiment analysis and keyword extraction.

[1193] Image Analysis: The server uses YOLO and SSD models to recognize objects in the image data and analyze it. It also uses OCR technology to extract text from the image.

[1194] Video analysis: The server divides the video data into frames and processes each frame in the same way as image analysis. The audio in the video is converted into text using ASR technology, and then the text is analyzed.

[1195] 3. Comparison with past scandals:

[1196] The server compares the analysis results with a database of past scandals and calculates a similarity score. It also compares with a database of fraud cases.

[1197] 4. Risk Assessment:

[1198] The server scores the risk of online outrage and abuse based on the similarity score, sentiment, and keywords. If the risk score is high, a warning is generated for the user.

[1199] 5. Risk Notification and Alternative Generation:

[1200] If the risk score is high, the server sends a risk warning to the user's device, and at the same time, it uses a generative AI model to generate alternatives to reduce the risk (such as safe trading suggestions or correction suggestions) and provides them to the user.

[1201] Specific examples

[1202] For example, consider the case where a user types "Today is the worst day. Everything is going wrong" in a social media post and then types "Is it okay to buy from this site?" as a transaction message.

[1203] Example prompt sentence:

[1204] The post read: "Today is the worst day. Everything is going wrong."

[1205] Transaction Comments: "Is it safe to buy from this site?"

[1206] The server analyzes this data, performs sentiment analysis and keyword extraction, and if it determines that a transaction is likely to be a scam or a controversy, it sends a warning to the user and offers alternatives such as "Today didn't go well, but I'll try again tomorrow!" or "This transaction is likely to be a scam, so we recommend purchasing from the official online store."

[1207] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1208] Step 1:

[1209] Input: Posted data and transaction data (text, images, videos) from user devices

[1210] Operation: The user device acquires the posting data and transaction data and sends them to the server. Specifically, the user enters text, images, or videos on the posting screen of an SNS or electronic payment app, and presses the send button to send the data.

[1211] Output: The server receives the data sent from the user terminal.

[1212] Step 2:

[1213] Input: Posting data and transaction data collected on the server

[1214] Operation: The server analyzes the received data. For text data, tokenization and morphological analysis are performed using an NLP library (e.g., SpaCy) to extract emotions and keywords. For image data, object detection is performed using YOLO or SSD models, and text is extracted using OCR technology. For video data, OpenCV is used to split the data into frames and perform the same processing as image analysis. The audio portion is also converted to text using ASR technology.

[1215] Output: Analysis results (emotion score, keywords, object recognition results, text extraction results)

[1216] Step 3:

[1217] Input: Analysis results

[1218] How it works: The server compares the analysis results with a database of past scandals and fraud cases. Specifically, it calculates a similarity score and evaluates how closely the results relate to past cases.

[1219] Output: Similarity score

[1220] Step 4:

[1221] Input: Similarity scores and analysis results

[1222] How it works: The server scores the risk of flame wars and abuse based on the similarity score, sentiment, and keywords. For example, if the similarity score exceeds a certain threshold, it is determined to be high risk.

[1223] Output: Risk score (flaming risk score, fraud risk score)

[1224] Step 5:

[1225] Input: Risk Score

[1226] How it works: If the risk score is high, the server generates a warning message for the user. It also uses a generative AI model (e.g., GPT-3) to generate alternatives to reduce the risk (such as suggestions for modifying the post or safe transactions). For example, if a post is overly negative, it suggests modifying it to a more positive one.

[1227] Output: Warning messages and alternatives

[1228] Step 6:

[1229] Input: Warning messages and alternatives

[1230] Operation: The server sends the generated warning message and alternatives to the user terminal, and the user terminal notifies the user of the received warning message and alternatives.

[1231] Output: A warning message and alternatives that are displayed to the user

[1232] Step 7:

[1233] Input: User confirms alternatives

[1234] How it works: The user reviews the warning message and alternatives they receive and decides whether to modify their posting or transaction. They can also submit the modifications to the server again.

[1235] Output: Modified submission and transaction data, or the original data

[1236] Through these steps, the system can assess in advance the risk of online outrage and fraudulent use of social media posting data and electronic transaction data, and provide users with appropriate warnings and alternatives.

[1237] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1238] The present invention relates to a system that analyzes posting data on social networking sites in advance and assesses the risk of a social networking site becoming a hot topic. By incorporating an emotion recognition engine, the present invention also analyzes the emotional aspects of posts, achieving a more comprehensive risk assessment. Specific embodiments of the system according to the present invention are described in detail below.

[1239] Server-side processing

[1240] 1. Receiving submitted data

[1241] The server receives post data (text, images, videos) from the user's device. When the user presses the post button, a POST request is sent from the device. The server receives this request using the REST API.

[1242] 2. Data Analysis

[1243] The server's AI analysis module analyzes the posted data, specifically as follows:

[1244] Sentence analysis:

[1245] A natural language processing (NLP) library is used to tokenize posts and perform contextual and sentiment analysis.

[1246] Image analysis:

[1247] Recognize objects in an image using an object detection model (e.g., YOLO or SSD).

[1248] It uses OCR technology to extract text from images, which can then be translated and analyzed.

[1249] Video Analysis:

[1250] The video is divided into frames, and each frame is processed in the same way as image analysis.

[1251] The audio portion of the video is converted into text using automatic speech recognition (ASR), and then sentiment analysis and related contextual analysis are performed.

[1252] 3. Emotion recognition

[1253] The server's emotion recognition engine detects the user's emotions from the received post data, as follows:

[1254] Text data: Natural language processing techniques are used to analyze emotions within the text. For example, emotions such as joy, anger, sadness, and happiness are identified, and an emotional assessment is made based on these.

[1255] Image data: Facial recognition and facial expression analysis techniques are used to identify the emotions of people in images. This is used in conjunction with image analysis techniques to assess the strength of emotions.

[1256] Video data: Emotions are recognized by analyzing the voice and facial expressions in the video. Image analysis and voice recognition technologies are combined to perform a comprehensive emotional evaluation.

[1257] 4. Comparison with past cases of online outrage

[1258] The server compares the analysis results and emotion recognition results with a database of past flaming cases, performs a similarity search, evaluates the degree of similarity of the posted data to past flaming cases, and calculates a similarity score.

[1259] 5. Risk Assessment

[1260] The server evaluates the risk of a controversy based on the analysis results, emotion recognition results, and similarity scores. For example, it calculates a risk score and determines that a score of 90 or higher is high risk, 70-89 is medium risk, and 70 or lower is low risk.

[1261] 6. Risk Notification and Alternative Generation

[1262] If the server determines that the risk of outrage is high, it generates a risk warning and alternatives. It uses a derived sentence generation model (e.g., GPT-3) to generate alternative sentences with less risk. If necessary, it also generates suggested revisions for images and videos.

[1263] 7. Sending warnings and alternatives

[1264] The server transmits the generated risk warning and alternatives to the user terminal.

[1265] Terminal side processing

[1266] 1. Create submission data

[1267] Users create posts through SNS applications, and can input text and upload images and videos.

[1268] 2. Submitting data

[1269] When the user presses the post button, the terminal sends the created post data to the server, for example, using a REST API POST request.

[1270] 3. Receive risk warnings

[1271] The device receives risk warnings sent from the server and notifies the user, including the risk assessment score and a specific warning message.

[1272] 4. Displaying alternatives

[1273] The terminal displays the alternatives sent from the server for reducing the risk on the user interface, and the user checks them and decides whether to modify the original post.

[1274] User operations

[1275] 1. Enter your post content

[1276] A user creates a post in a social networking application, for example, by typing, "Today is the worst day. Everything is going wrong."

[1277] 2. Risk confirmation

[1278] The user checks the risk assessment notification and warning message from the server on their device. For example, a warning message such as "This post contains negative sentiment and is at high risk of causing a backlash" may be displayed.

[1279] 3. Selection of alternatives

[1280] The user checks the alternatives provided by the server and decides whether to revise the post based on them. For example, an alternative suggestion might be "Today didn't go well, but I'll try my best tomorrow!"

[1281] 4. Confirmation of submission

[1282] The user can then modify the content of their post based on the displayed alternatives and finally confirm the post. The modified content is then sent back to the server and posted to the SNS.

[1283] In this way, the system of the present invention assesses the risk of a social media firestorm before a post is made, helping users to use social media safely. In addition, by combining it with an emotion recognition engine, it is possible to analyze the emotional aspects of posts, enabling more accurate risk assessment.

[1284] The processing flow will be explained below.

[1285] Server-side processing

[1286] Step 1:

[1287] The server receives post data (text, images, videos) from the user's device. When the user presses the post button, a POST request is sent from the device. The server receives this request via the REST API.

[1288] Step 2:

[1289] The server preprocesses the received submission data, tokenizing it for text data and storing image and video data in a specific directory.

[1290] Step 3:

[1291] The server's AI analysis module analyzes the posted data using the following steps:

[1292] Text analysis: Uses natural language processing (NLP) libraries to tokenize text and perform contextual and sentiment analysis.

[1293] Image analysis: Recognize objects and text in images using object detection models (e.g., YOLO or SSD).

[1294] Video analysis: Videos are divided into frames, and each frame is analyzed in the same way as image analysis. The audio portion of the video is converted into text using automatic speech recognition (ASR), and this is then analyzed.

[1295] Step 4:

[1296] The emotion recognition engine of the server recognizes the emotion of the user based on the received posting data.

[1297] Text data: Natural language processing techniques are used to analyze sentiment within the text, for example, identifying positive, negative, or neutral sentiment.

[1298] Image data: Using facial recognition and facial expression analysis techniques, we identify the emotions of people in images.

[1299] Video data: Analyzes the voice and facial expressions in the video, and integrates multiple emotional data to evaluate the user's overall emotions.

[1300] Step 5:

[1301] Based on the analysis results and emotion recognition results, the server compares the posted data with a database of past flaming cases, performs a similarity search, and scores the degree of similarity with past flaming cases.

[1302] Step 6:

[1303] The server evaluates the risk of a controversy based on the analysis results, emotion recognition results, and similarity scores. A score of 90 or above is considered high risk, 70-89 is considered medium risk, and 70 or below is considered low risk.

[1304] Step 7:

[1305] If the risk of a post being criticized is high, the server will notify the user of the risk and generate alternatives. For example, it will use a derived sentence generation model (e.g., GPT-3) to generate alternative sentences with less risk. In some cases, it will also generate suggested revisions for images and videos.

[1306] Step 8:

[1307] The server sends the generated risk warning and alternatives to the user's device, and if an error occurs, it records it in a log.

[1308] Terminal side processing

[1309] Step 1:

[1310] A user creates a post through a social networking application, for example, by inputting text and uploading images and videos.

[1311] Step 2:

[1312] When the user presses the post button, the terminal sends the created post data to the server, for example, using a POST request.

[1313] Step 3:

[1314] The device waits for a response from the server and receives a risk warning and alternatives.

[1315] Step 4:

[1316] The device will display risk warnings and alternatives to the user, for example as a pop-up message or notification.

[1317] User operations

[1318] Step 1:

[1319] A user inputs a post into a social networking application, for example, a sentence such as "Today is the worst day. Everything is going wrong."

[1320] Step 2:

[1321] When the user presses the post button, the post data is sent to the server.

[1322] Step 3:

[1323] The user will see a risk warning and alternatives from the server. For example, a warning message such as "This post contains negative sentiment and is at high risk of causing controversy" will be displayed.

[1324] Step 4:

[1325] The user checks the alternatives and decides whether to revise the post based on them. For example, an alternative suggestion might be "Today didn't go well, but I'll try my best tomorrow!"

[1326] Step 5:

[1327] Once the user has finalized the content of the post and pressed the post button again, the revised post data will be resent to the server and posted on the SNS.

[1328] Example 2

[1329] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1330] There is a need to provide an environment where users can safely use SNS by assessing the risk of data posted on SNS becoming a hot topic in advance. Conventional technologies simply filter the content of posts, without performing comprehensive analysis including emotional aspects and images and videos, making it difficult to accurately assess risk. The purpose of this invention is to solve these problems and provide a more comprehensive and accurate risk assessment system.

[1331] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving post data, means for analyzing the received post data using natural language processing and image analysis techniques, means for performing emotion recognition based on the analysis results, means for comparing the emotion recognition results and the analysis results with a database of past flame war cases, means for evaluating the risk of a flame war based on the comparison results, means for sending a warning to the user when the risk of a flame war is high, means for generating alternative information to reduce the risk of a flame war using a generative AI model, and means for sending the warning and alternative information to the user terminal. This makes it possible to provide a highly accurate evaluation of the risk of a flame war and appropriate alternatives before a user posts on an SNS.

[1332] "Posted data" refers to data in the form of text, images, and videos posted by users to online platforms such as social networking sites.

[1333] "Natural language processing" is a technology that enables computers to understand, interpret, and generate human language, typically involving tokenization, contextual analysis, and sentiment analysis.

[1334] "Image analysis" refers to the use of computer vision techniques to recognize and analyze objects and text in images. Examples include object detection and OCR.

[1335] "Emotion recognition" is a technology that uses natural language processing and image analysis technologies to detect emotions from text, images, and videos.

[1336] "Flame risk" is an evaluation index of the likelihood that posted data will receive a large amount of negative reactions or criticism on social media.

[1337] The "database of past social media outrage cases" is a database of past social media outrage cases, and is a data set used for risk assessment.

[1338] A "generative AI model" is an artificial intelligence model that generates new data from given input data. A specific example is GPT-3.

[1339] "Similarity search" is a technique for evaluating how similar certain data is to given reference data (in this case, past online scandals).

[1340] A "warning" is a message that notifies the user that there is a high risk of the posted data causing a firestorm.

[1341] "Alternative information" refers to modifications or alternatives suggested by generative AI models to reduce the risk of backlash.

[1342] The present invention is a system that analyzes social media posting data in advance and assesses the risk of a social media outcry. This system combines emotion recognition technology and generative AI models to provide a more comprehensive risk assessment and alternative solutions. Specific embodiments of the present invention are described in detail below.

[1343] Server-side processing

[1344] The server first receives the posted data (text, images, videos) sent from the user's device. This reception process uses a POST request using the REST API. For example, data is received through an API gateway with a set endpoint. The server uses a cloud platform to achieve scalable processing.

[1345] The server's AI analysis module then analyzes the received data using the following techniques:

[1346] Use a natural language processing (NLP) library (e.g., spaCy or BERT) to tokenize the text data and perform contextual and sentiment analysis.

[1347] It uses object detection models (e.g., YOLO and SSD) to identify objects in images, and OCR technology to extract text from images for translation and sentiment analysis.

[1348] In video analysis, the video is divided into frames and each frame is processed in the same way as image analysis. The audio portion of the video is converted into text using automatic speech recognition (ASR), and then emotion and context analysis are performed.

[1349] Emotion recognition is performed to detect user emotions from received posting data. This process uses natural language processing technology to detect emotions in text and identify emotions such as joy, anger, sadness, and happiness. It also uses facial recognition and facial expression analysis technology to analyze the emotions of people in images and videos.

[1350] Based on the analysis results and emotion recognition results, the server compares them with a database of past flaming cases. Using a similarity search technique, the server evaluates how similar the posted data is to past flaming cases and calculates a similarity score. Specifically, the comparison uses techniques such as Cosine similarity.

[1351] The server then assesses the risk of a controversy based on the analysis results, emotion recognition results, and similarity scores. A score of 90 or higher is considered high risk, 70-89 is considered medium risk, and 70 or lower is considered low risk.

[1352] If the risk of outrage is high, the server generates a risk warning and alternative suggestions. Using a generative AI model (e.g., GPT-3), it generates alternative text to reduce the risk. If necessary, it also generates suggested edits for images and videos. The generated alternative suggestions and risk warnings are converted into JSON format and sent to the user's device.

[1353] Terminal side processing

[1354] First, a user creates a post through a social networking application. They enter text and upload images and videos. When the user presses the post button, this post data is sent to the server via a REST API POST request.

[1355] When a risk warning is sent from the server, the device receives it and notifies the user. The risk assessment score and specific warning message are displayed on the interface, and the device also displays the alternatives sent from the server. The user can review the alternatives and decide whether to revise the original post.

[1356] User operations

[1357] A user creates a post in a social networking application. For example, the user enters the sentence, "Today is the worst day. Everything is going wrong." The server determines that this post contains strong negative emotions and sends a warning message to the device indicating a high risk of it starting a flame war. The warning message reads, "This post contains negative emotions and there is a high risk of it starting a flame war."

[1358] The user also checks alternative suggestions provided by the server. For example, an alternative sentence such as "Today didn't go well, but I'll try my best tomorrow!" is presented. The user can then revise the post based on this and finally confirm the post. The revised post is then sent back to the server and posted to the SNS.

[1359] This system allows users to use social media safely and assess the risk of a social media firestorm in advance. Furthermore, by using emotion recognition technology, it is possible to analyze the emotional aspects of posts and improve the accuracy of risk assessment.

[1360] Specific examples

[1361] For example, if a user tries to post "Today is the worst day. Everything is going wrong," the server analyzes it and determines that it is a post with a strong negative sentiment. If it is assessed as having a high risk of causing a flame war, the server generates an alternative suggestion, such as "Today didn't go well, but I'll try my best tomorrow!" and presents it to the user.

[1362] Prompt Sentence Examples

[1363] Input: Today is the worst day. Everything is going wrong.

[1364] Output: Things didn't go well today, but I'll try my best tomorrow!

[1365] This prompt sentence can be used to train a generative AI model (e.g., GPT-3) to generate a less risky alternative sentence.

[1366] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1367] Step 1: Receive post data

[1368] The server receives post data from the user's device. When a user presses the post button in the SNS application, a POST request is sent from the device. The input is the text, image, and video data created by the user, and the output is the post data received by the server. Specifically, the server receives this request using a REST API and retrieves the data through an API gateway with a configured endpoint.

[1369] Step 2: Data analysis

[1370] The AI ​​analysis module on the server analyzes the received post data. Specifically,

[1371] Text analysis: Using a natural language processing (NLP) library (e.g., spaCy or BERT), text data is tokenized and context and sentiment analysis is performed. The input is text data, and the output is the analysis results.

[1372] Image analysis: Identify objects in images using object detection models (such as YOLO or SSD). Also, extract text from images using OCR technology and perform sentiment analysis. The input is image data, and the output is the analysis results.

[1373] Video analysis: Videos are divided into frames, and each frame is processed in the same way as image analysis. The audio portion of the video is converted into text using automatic speech recognition (ASR), and emotion and context analysis is performed. The input is video data, and the output is the analysis results data.

[1374] Step 3: Emotion Recognition

[1375] The server's emotion recognition engine detects the user's emotions from the analyzed post data. Specifically,

[1376] Text data: Detect emotions in text using natural language processing techniques. The input is the result of text analysis, and the output is the emotion recognition result of the text.

[1377] Image data: Using facial recognition and facial expression analysis technology, we identify the emotions of people in an image. The input is the result of image analysis, and the output is the emotion recognition result of the image.

[1378] Video data: Analyzes the voice and facial expressions in the video to perform a comprehensive emotional evaluation. The input is the result of video analysis data, and the output is the emotion recognition results for the video.

[1379] Step 4: Compare with past flame war cases

[1380] The server compares the analysis results and emotion recognition results with a database of past flaming cases. Specifically, it uses a similarity search method to evaluate how similar the posted data is to past flaming cases. The input is the analysis results and emotion recognition results, and the output is a similarity score. Cosine similarity, for example, is used to evaluate the similarity.

[1381] Step 5: Risk assessment

[1382] The server evaluates the risk of a controversy based on the analysis results, emotion recognition results, and similarity score. It calculates a risk score and classifies a score of 90 or above as high risk, 70-89 as medium risk, and 70 or below as low risk. The inputs are the similarity score, emotion recognition results, and analysis results, and the output is the risk assessment score and its judgment result.

[1383] Step 6: Risk notification and alternative generation

[1384] If the server determines that there is a high risk of outrage, it generates a risk warning and alternatives. It uses a generative AI model (e.g., GPT-3) to generate alternative text with less risk. If necessary, it also generates suggested modifications for images and videos. The input is the risk assessment score and the judgment result, and the output is a risk warning message and alternatives.

[1385] Step 7: Sending warnings and alternatives

[1386] The server sends the generated risk warning and alternatives to the user's device. The input is the risk warning message and alternatives, and the output is the data to be sent to the user's device. The risk warning and alternatives are converted to JSON format and sent to the user's device via a REST API.

[1387] Step 8: Receive risk warnings

[1388] The terminal receives the risk warning sent from the server and notifies the user. The input is the risk warning message and alternatives sent from the server, and the output is the notification to the user interface.

[1389] Step 9: View alternatives

[1390] The terminal displays the alternatives sent from the server on the user interface. The user checks them and decides whether to modify the original post. The input is the alternatives, and the output is the display content presented to the user.

[1391] Step 10: User Edits to Posts

[1392] The user modifies the post based on the alternatives provided by the server. The input is the alternatives and the original post, and the output is the modified post.

[1393] Step 11: Finalize and submit your revised post

[1394] The user confirms the revised content of the post and sends it back to the server. The input is the revised content of the post, and the output is the final post data sent to the server. The server receives this revised post and posts it to the SNS.

[1395] (Application example 2)

[1396] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1397] In recent years, the risk of user posts on social media and online platforms causing online outrage has become a social issue. In particular, for brick-and-mortar stores, there is a growing risk that negative customer feedback spreading online could damage their reputation and negatively impact sales. It is also important to quickly analyze customer feedback and provide risk assessments and appropriate countermeasures. The present invention aims to solve these problems by providing a system that enables brick-and-mortar stores to analyze customer feedback and quickly take appropriate countermeasures.

[1398] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1399] In this invention, the server includes means for receiving posted data, means for analyzing the received posted data and comparing it with past cases of online flaming, means for assessing the risk of online flaming based on the analysis results, means for sending a warning to the user when the risk of online flaming is high, means for generating alternative information to reduce the risk of online flaming, means for sending the warning and alternative information to the user terminal, means for analyzing customer feedback at the physical store, means for performing sentiment analysis of the posted data and assessing the risk, and means for generating alternatives that arise when the risk is high. This enables quick and accurate analysis of customer feedback and risk assessment, and appropriate action to protect the reputation of the physical store.

[1400] "Posted data" refers to information such as text, images, and videos that users create and send on social media or online platforms.

[1401] "Means for receiving" refers to the functions and protocols that allow the server to receive posted data sent from the user terminal.

[1402] "Means of analysis" refers to the process of analyzing the content of received posted data using natural language processing and image analysis technology.

[1403] "Past cases of online outrage" refer to cases in the past where a user's post on a social media or online platform sparked criticism or negative reactions.

[1404] "Means of comparison" refers to the algorithms and methods used to evaluate the similarity of newly received posted data with past cases of online outrage.

[1405] "Flame risk" refers to the likelihood that a particular post will provoke criticism or negative reactions in the online community.

[1406] "Sentiment analysis" refers to the process of identifying emotional components in text or images using natural language processing and image recognition techniques.

[1407] "Risk assessment" refers to the process of evaluating the degree of risk of a post causing a controversy using numerical or categorized terms based on the analysis of received post data.

[1408] "Means for sending warnings" refers to the function of sending warning messages to users when there is a high risk of a social media outburst.

[1409] "Alternative information" refers to safer options, such as text or images that have been modified from the original post to reduce the risk of a backlash.

[1410] "Means for generating alternatives" refers to a process that uses generative AI models or other technologies to automatically create safe alternative information.

[1411] "User terminal" refers to devices such as smartphones, tablets, and PCs used to access platforms such as social networking sites.

[1412] "Brick and mortar store" refers to a store that offers goods or services in a physical location.

[1413] "Customer feedback" refers to customer experiences, opinions, and evaluations expressed in the form of posts in physical stores.

[1414] The present invention relates to a system that analyzes posted data in advance and evaluates the risk of a social media firestorm. This system is primarily intended to analyze customer feedback in brick-and-mortar stores, and is designed to prevent negative feedback from damaging the store's reputation. Specific embodiments are described below.

[1415] Server-side processing

[1416] 1. Receiving submitted data

[1417] The server receives the posted data (text, images, videos) sent from the user's device. When the user sends feedback, a POST request is sent from the user's device via the REST API. The server receives this request and prepares to analyze the posted data.

[1418] 2. Data Analysis

[1419] The server program analyzes the submitted data using the following hardware and software:

[1420] Natural Language Processing (NLP) library: Uses TextBlob to tokenize posts and perform contextual and sentiment analysis.

[1421] Image Analysis: Extract text in images using OpenCV and pytesseract and perform analysis on the same.

[1422] The analysis results are compared with a database of past online outrage cases and a similarity search is performed.

[1423] 3. Emotion recognition

[1424] The server analyzes the sentiment of the posted data through the following process:

[1425] Text data: Natural language processing techniques are used to identify sentiment within the text.

[1426] Image data: Use OCR technology to extract text from images and analyze their sentiment.

[1427] 4. Risk assessment and alternative generation

[1428] The server evaluates the risk of a controversy based on the analysis results and emotion recognition results. If the risk is determined to be high, it uses a generative AI model (e.g., GPT-3) to generate alternatives to reduce the risk.

[1429] Terminal side processing

[1430] The user terminal provides the following functions:

[1431] Feedback submission: Users can enter feedback in the form of text, images, or videos using their smartphones.

[1432] Display risk warning: Receives risk warnings sent from the server and notifies the user.

[1433] Displaying alternatives: The alternatives sent from the server are displayed in the user interface, allowing the user to choose whether to revise the post.

[1434] Specific examples

[1435] For example, if a customer posts feedback saying, "Today's customer service was terrible," this post is sent to the server. On the server side, negative sentiment is detected using TextBlob and evaluated as high risk. In response, a generative AI model is used to generate an alternative suggestion, such as, "Today was a bit disappointing, but I look forward to your next visit," and this is presented to the user.

[1436] Prompt Sentence Examples

[1437] "Generate positive alternatives for the user-submitted feedback, 'The customer service today was terrible.'"

[1438] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1439] Step 1:

[1440] The user uses their smartphone to enter feedback, which can be in the form of text, images, or videos. The entered feedback might be something like "Today is the worst day. Everything is going wrong." When the user presses the submit button, the device sends the feedback data to the server as a POST request.

[1441] Step 2:

[1442] The server receives the posted data sent from the user's device. The received data can be text, images, or videos, and is sent via the REST API. The server temporarily stores the received data.

[1443] Step 3:

[1444] The server analyzes the received post data. For text data, it uses TextBlob to tokenize the data, analyzes the context, and performs sentiment analysis. The input for analysis is the post text, and the output is the sentiment polarity (positive, negative, neutral). For example, "Today is the worst day. Everything is going wrong." is analyzed as negative.

[1445] Step 4:

[1446] The server analyzes the image data. It extracts the text from the image using OpenCV and performs OCR analysis using pytesseract. The input for the analysis is the posted image, and the extracted text is again subjected to sentiment analysis using TextBlob. The output is the text from the image and the results of the sentiment analysis.

[1447] Step 5:

[1448] The server analyzes the video data. It divides the video into frames and performs image analysis on each frame. It also converts the audio portion of the video into text using automatic speech recognition (ASR), and performs sentiment analysis on that text. The input for the analysis is the posted video, and the output is the text for each frame and the sentiment analysis results.

[1449] Step 6:

[1450] The server compares the analysis results with a database of past flaming cases. It uses a similarity algorithm to evaluate how similar the analysis data is to past flaming cases. The input is the analysis results and past flaming cases, and the output is a similarity score.

[1451] Step 7:

[1452] The server evaluates the risk of a controversy based on the analysis results and the similarity score. The emotional polarity and similarity scores are combined to calculate an overall risk score. The output is a risk score; for example, a risk score of 90 or higher is considered high risk.

[1453] Step 8:

[1454] If the risk of a firestorm is determined to be high, the server sends a risk warning to the user's device. The warning includes a risk score and a specific warning message. The input is the risk score, and the output is the warning message.

[1455] Step 9:

[1456] To generate alternatives, the server uses a generative AI model. For example, it uses a prompt such as "Generate a positive alternative to the user-posted feedback, 'Today is the worst day. Everything is going wrong.'" The input is the original post text and the prompt, and the output is the alternative text. For example, an alternative such as "Today didn't go well, but let's try our best tomorrow!" is generated.

[1457] Step 10:

[1458] The server sends the generated alternatives to the user's terminal. The user can check the alternatives and choose whether to correct the feedback. The input is the alternative text, and the output is a notification to the user's terminal.

[1459] Step 11:

[1460] The user then takes the suggested alternatives into consideration and finally submits the revised feedback, which is then sent back to the server and posted on the SNS platform.

[1461] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1462] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1463] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1464] [Fourth embodiment]

[1465] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1466] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1467] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1468] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1469] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1470] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1471] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1472] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1473] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1474] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1475] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1476] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1477] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1478] The present invention relates to a system for analyzing posted data on a social networking site in advance and assessing the risk of a social networking site becoming a hot topic. Specific embodiments of the system according to the present invention will be described in detail below.

[1479] Server-side processing

[1480] 1. Receiving submitted data

[1481] The server receives the posted data (content such as text, images, and videos) sent from the user's device. For example, it retrieves the data using a POST request using a REST API.

[1482] 2. Data Analysis

[1483] The AI ​​analysis module on the server analyzes the data in the following steps:

[1484] Sentence analysis:

[1485] Use a natural language processing (NLP) library to tokenize the text and perform morphological analysis.

[1486] Extract relevant keywords and emotional expressions from the text.

[1487] Image analysis:

[1488] Recognize objects in an image using an object detection model (e.g., YOLO or SSD).

[1489] The text in the image is extracted using OCR technology and then further analyzed.

[1490] Video Analysis:

[1491] The video is divided into frames, and each frame is processed in the same way as image analysis.

[1492] The audio portion of the video is converted into text using automatic speech recognition (ASR) technology, and text analysis is performed.

[1493] 3. Comparison with past cases of online outrage

[1494] The server compares the analysis results with a database of existing cases of online outrages and evaluates the risk of online outrage based on the degree of similarity. For example, it searches the database of past cases of online outrages and calculates a similarity score.

[1495] 4. Risk Assessment

[1496] The server scores the risk of a controversy based on the similarity score and the extracted sentiment and keywords. The server then determines the degree of risk based on the assessed risk score. For example, a risk score of 90 or above is high risk, 70-89 is medium risk, and anything below that is low risk.

[1497] 5. Risk Notification and Alternative Generation

[1498] The server sends a warning to users if the risk score is high. To support safe posting, the server uses a derived text generation model (e.g., GPT-3) to generate alternative text and image / video revision suggestions with less risk and sends them to the user's device.

[1499] Terminal side processing

[1500] 1. Create submission data

[1501] Users use SNS applications to create posts, including text they type and images and videos they upload.

[1502] 2. Submitting data

[1503] When the user presses the post button, the terminal sends the created post data to the server, for example, using a REST API POST request.

[1504] 3. Receive risk warnings

[1505] The device receives risk warnings sent from the server and notifies the user, including the risk assessment score and a specific warning message.

[1506] 4. Displaying alternatives

[1507] The terminal displays the alternatives sent from the server for reducing the risk on the user interface, and the user checks them and decides whether to modify the original post.

[1508] User operations

[1509] 1. Enter your post content

[1510] A user creates a post on a social networking app, for example, by typing a sentence such as, "Today is the worst day. Everything is going wrong."

[1511] 2. Risk confirmation

[1512] The user checks the risk assessment notification and warning message from the server on their device. For example, a warning message such as "This post contains negative sentiment and is at high risk of causing a backlash" may be displayed.

[1513] 3. Selection of alternatives

[1514] The user checks the alternatives provided by the server and decides whether to revise the post based on the alternatives or to post the original post as is. For example, an alternative suggestion might be "Today didn't go well, but I'll try my best tomorrow!"

[1515] 4. Confirmation of submission

[1516] The user can then modify the content of their post based on the displayed alternatives and finally confirm the post. The modified content is then sent back to the server and posted to the SNS.

[1517] In this way, the system of the present invention evaluates the risk of a controversy before posting, helping users to use SNS safely.

[1518] The processing flow will be explained below.

[1519] Server-side processing

[1520] Step 1:

[1521] The server receives post data (text, images, videos) from the user's device. When the user presses the post button, a POST request is sent from the device. The server receives this request using the REST API.

[1522] Step 2:

[1523] The server preprocesses the submitted data it receives, for example tokenizing text data and storing image and video data in specific directories.

[1524] Step 3:

[1525] The server's AI analysis module analyzes the posted data.

[1526] Text analysis: Using a natural language processing (NLP) library, posts are tokenized and semantic and sentiment analysis is performed.

[1527] Image analysis: Recognize objects in images using object detection models (e.g., YOLO and SSD), and extract text from images using hooked OCR technology.

[1528] Video analysis: Videos are divided into frames, and each frame is analyzed in the same way as image analysis. Audio in the video is converted into text using automatic speech recognition (ASR), and related sentence analysis is performed.

[1529] Step 4:

[1530] The server compares the analysis results with a database of past flaming cases, performs a similarity search, evaluates the degree to which the posted data is similar to past flaming cases, and calculates a similarity score.

[1531] Step 5:

[1532] The server evaluates the risk of a controversy based on the analysis results and the similarity score. For example, it calculates a risk score, determining that a score of 90 or higher is high risk, 70-89 is medium risk, and 70 or lower is low risk.

[1533] Step 6:

[1534] If the server determines that a post is at high risk of causing an uproar, it generates a risk warning and alternatives. It uses a derived sentence generation model (e.g., GPT-3) to generate alternative sentences with less risk. If necessary, it also generates suggested revisions for images and videos.

[1535] Step 7:

[1536] The server sends risk warnings and alternative solutions to the user's device, and keeps a log containing details of any errors that occur.

[1537] Terminal side processing

[1538] Step 1:

[1539] Users create posts through SNS applications, and can input text and upload images and videos.

[1540] Step 2:

[1541] When the user presses the submit button, the device sends the submission data to the server, for example, using a REST API POST request.

[1542] Step 3:

[1543] The device waits for a response from the server, and if the server returns a risk warning and alternatives, it receives them.

[1544] Step 4:

[1545] The device displays the risk warning and alternatives received from the server to the user, for example, by a pop-up message or a notification box.

[1546] Step 5:

[1547] Users review the risk warnings and alternative suggestions and decide whether to revise their original post. If revisions are necessary, they reconstruct the content using the alternative sentences and suggested revisions as a reference.

[1548] Step 6:

[1549] When the user decides on the final post and presses the post button again, the terminal resends the corrected post data to the server.

[1550] User Action

[1551] Step 1:

[1552] A user types or uploads a post to a social networking application, for example, typing the sentence "Today is the worst day. Everything is going wrong."

[1553] Step 2:

[1554] When the user presses the submit button, the data is sent to the server.

[1555] Step 3:

[1556] The user then checks the risk warning and alternatives from the server. For example, the user may be presented with a warning message such as "This post contains negative emotions and is at high risk of causing a backlash," along with an alternative sentence such as "Today didn't go well, but I'll try my best tomorrow!"

[1557] Step 4:

[1558] The user decides whether to revise the post based on the alternative suggestions, and if so, checks the new post.

[1559] Step 5:

[1560] Once the user has finalized the content of the post and pressed the post button again, the final post data will be posted on the SNS.

[1561] Example 1

[1562] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1563] Posting on social media is an important way to share information with many people, but if inappropriate content is included, it can lead to outrage and excessive criticism. In addition, since there are limited ways for users to know about such risks in advance, they may be cautious about using social media. Therefore, a system is needed that evaluates the risk of outrage before posting and suggests safe ways to post.

[1564] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1565] In this invention, the server includes means for receiving post data from a user terminal, means for analyzing the received post data using natural language processing, object recognition, and voice recognition, means for comparing the received post data with past flaming cases based on keywords and emotional expressions extracted by the analysis, means for scoring and assessing the risk of flaming based on the comparison results, means for sending a warning to the user if the risk of flaming is high, means for generating alternative information to reduce the risk of flaming, and means for sending the warning and alternative information to the user terminal. This allows users to assess the risk of flaming before posting and learn safe posting methods.

[1566] A "user terminal" is a device operated by a user, and includes a smartphone, tablet, PC, etc.

[1567] "Posted data" refers to content such as text, images, and videos that users share on social media.

[1568] "Server" refers to a high-performance computer system installed at a remote location that analyzes and processes data received from user devices.

[1569] "Natural language processing" is a technology for processing human language using computers, and includes text tokenization, morphological analysis, sentiment analysis, etc.

[1570] "Object recognition" refers to the technology of identifying objects in images and classifying them.

[1571] "Speech recognition" refers to the technology of converting voice data into text.

[1572] "Keywords" are words or phrases that are particularly important in a sentence or piece of content.

[1573] "Sentiment expression" refers to elements in text that indicate sentiment (positive, negative, neutral, etc.) analyzed by natural language processing.

[1574] "Flame war cases" include data that refer to large-scale negative reactions caused by inappropriate posts on social media in the past.

[1575] The "similarity score" is a numerical value that indicates the similarity between the target data and past flame war case data, and is used for risk assessment.

[1576] "Scoring" refers to a method of quantifying risk or performance based on specific criteria.

[1577] "Warning" refers to a caution message to inform the user of a high risk.

[1578] "Alternative information" refers to safe posts and suggested revisions to reduce the risk of outrage.

[1579] MODE FOR CARRYING OUT THE INVENTION

[1580] The present invention relates to a system for analyzing data posted to a social networking site in advance and assessing the risk of a social networking site becoming a hot topic. Specific embodiments of the present invention will be described in detail below.

[1581] Server Processing

[1582] The server receives the posted data (text, images, videos, and other content) sent from the user's device. Specifically, it uses a mechanism to receive data via a POST request via the REST API.

[1583] The server analyzes the received posted data using the following methods. First, it uses a natural language processing (NLP) library (e.g., spaCy) to tokenize the text data and extract emotional expressions and keywords. Next, it uses an object recognition model (e.g., YOLO) to recognize objects within the image. It also uses OCR technology (e.g., Tesseract OCR) to extract text from the image and further analyze it. It then divides the video data into frames, each of which is processed in the same way as image analysis. It then uses automatic speech recognition (ASR) technology to convert the audio portion of the video into text, which is then analyzed.

[1584] The server compares the analysis results with a database of past flaming cases. It calculates a similarity score from the database and evaluates the risk. For this purpose, it uses a scoring method; for example, a similarity score of 90 or higher with past cases is rated as high risk, 70-89 as medium risk, and anything below that is rated as low risk.

[1585] If the server determines that there is a high risk of outrage, it sends a warning to the user. It also uses a generative AI model (e.g., GPT-3) to generate alternative information (suggested revisions to text, images, and videos) with less risk and sends them to the user's device.

[1586] Terminal handling

[1587] Users use SNS applications to create posts, and the text they enter and the images and videos they upload become the post data.

[1588] When the user presses the submit button, the device sends the created post data to the server using a REST API POST request.

[1589] The device receives the risk warning sent from the server and notifies the user, displaying the risk assessment score and a specific warning message.

[1590] The terminal displays the alternatives for reducing the risk sent from the server to the user, who then checks the alternatives and decides whether to revise the content of the post and repost it.

[1591] User operations

[1592] A user inputs content to be posted on the SNS application to create post data. For example, the user inputs a sentence such as "Today is the worst day. Everything is going wrong."

[1593] The user checks the risk assessment notification and warning message from the server on their device. For example, a warning message such as "This post contains negative emotions and is at high risk of causing a backlash" may be displayed.

[1594] The user checks the alternatives provided by the server and modifies the content of the post based on them. For example, an alternative suggestion such as "Today didn't go well, but I'll try my best tomorrow!" may be presented.

[1595] Specific examples

[1596] Example prompt sentence:

[1597] "This sentence contains negative content. Please consider the following alternatives:

[1598] 1. Things didn't go well today, but I'll try my best tomorrow!

[1599] 2. Sometimes you have bad days, but that's life.

[1600] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1601] Step 1: Receiving posted data

[1602] The server receives posted data from user devices. Input includes content such as text, images, and videos. Specifically, data is received via a POST request via the REST API. The output is the received posted data itself.

[1603] Specific behavior:

[1604] When a user presses the "Post" button on their device, the device sends a POST request to the server, and text, image, and video data are sent to the server.

[1605] Step 2: Text analysis

[1606] The server analyzes the received text data using a natural language processing (NLP) library. The input is the received text data, and the output is the analysis results such as keywords and emotional expressions.

[1607] Specific behavior:

[1608] The server uses the spaCy library to tokenize the text and extract important keywords and sentiments. For example, from the sentence "Today is the worst day. Everything is going wrong," it extracts keywords like "worst" and "going wrong."

[1609] Step 3: Image analysis

[1610] The server analyzes the received image data using an object recognition model (e.g., YOLO). The input is the received image data, and the output is the analysis results of objects and text in the image.

[1611] Specific behavior:

[1612] The server uses the YOLO model to identify people and objects in the image and extract their location information, and Tesseract OCR to extract text from the image and analyze important keywords such as "warning" and "caution."

[1613] Step 4: Video Analysis

[1614] The server divides the received video data into frames and processes each frame in the same way as image analysis. The input is the received video data, and the output is the image analysis results for each frame and the speech-to-text results.

[1615] Specific behavior:

[1616] The server uses OpenCV to split the video into frames and analyzes each frame with the YOLO model. At the same time, the audio portion is converted to text using the Google Speech-to-Text API and subjected to NLP processing.

[1617] Step 5: Compare with past flame wars

[1618] The server compares the analysis results with a database of past flaming cases. The input is the analyzed keywords and emotional expressions, and the output is a similarity score.

[1619] Specific behavior:

[1620] The server uses TfidfVectorizer to calculate the similarity score between the analysis results and past flaming cases. For example, if the keywords "worst" and "doesn't work" match many of the past flaming cases, a high similarity score will be calculated.

[1621] Step 6: Risk assessment

[1622] The server scores the flame risk based on the similarity score. The input is the similarity score, and the output is the estimated flame risk score.

[1623] Specific behavior:

[1624] The server applies a scoring algorithm to calculate a risk score. For example, a similarity score of 90 or higher is considered "high risk," and a score between 70 and 89 is considered "medium risk."

[1625] Step 7: Sending an alert

[1626] The server sends a warning to the user if the risk score is high. The input is the assessed risk score and the output is the warning message.

[1627] Specific behavior:

[1628] The server generates a warning message based on the risk score and sends it to the user's device. For example, a message might say, "This post contains negative emotions and is at high risk of causing a backlash."

[1629] Step 8: Generate alternative information

[1630] To reduce the risk of outrage, the server generates alternative information using a generative AI model (e.g., GPT-3). The input is the initial post data, and the output is the generated alternative sentences or suggested revisions.

[1631] Specific behavior:

[1632] If the initial post is "Today is the worst day. Everything is going wrong," GPT-3 will be used to generate an alternative post that says, "Some things didn't go well today, but I'll try my best tomorrow!"

[1633] Step 9: Notification of warnings and alternatives

[1634] The terminal receives the warning and alternatives sent from the server and notifies the user of them. The input is the warning message and alternatives sent from the server, and the output is the notified message.

[1635] Specific behavior:

[1636] The device will use notifications to display warnings and alternatives to the user, including a risk assessment score and specific warning messages.

[1637] Step 10: Edit and finalize your submission

[1638] The user modifies the posted content by referring to the alternatives and finally confirms it. The input is the alternatives and the initial posted data, and the output is the modified posted data.

[1639] Specific behavior:

[1640] The user then modifies the content of the post based on the alternative suggestions and clicks the "Post" button again to send the modified post to the server, where it is evaluated again and posted on the SNS.

[1641] (Application example 1)

[1642] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1643] Systems that can detect the risk of unexpected outrage or fraudulent use and enable users to take appropriate action are needed not only for data posted on social media but also for electronic payment services. However, current systems have limited scope for risk assessment, and do not adequately deter fraudulent use or prevent outrage. In particular, analysis and evaluation of transaction data and messages in electronic payment services is insufficient, potentially exposing users to the risk of fraud and fraudulent use. There is a need to solve these problems and provide a safe and secure user environment.

[1644] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1645] In this invention, the server includes means for receiving posted data, means for analyzing the received posted data and comparing it with past cases of online outrage, means for assessing the risk of online outrage based on the analysis results, means for sending a warning to the user if the risk of online outrage is high, means for generating alternative information to reduce the risk of online outrage, means for sending the warning and alternative information to the user terminal, means for receiving transaction data and messages and assessing the risk of fraud, means for sending a warning to the user if the risk of fraud is high, and means for generating an alternative transaction plan to reduce the risk of fraud. This makes it possible to comprehensively assess and address the risk of online outrage and fraud for SNS posted data and electronic payment transactions.

[1646] "Posted data" refers to various content such as text, images, and videos that users post on platforms such as social networking sites.

[1647] The "receiving means" refers to the technical means for obtaining data sent from a user terminal via the Internet.

[1648] The "analyzing means" is a means for analyzing the content and characteristics of received data using a specific algorithm or model.

[1649] "Flame cases" are records of past events that have caused a large-scale negative reaction on social media or online platforms.

[1650] "Comparative means" refers to a technical means of comparing the analysis results with past cases to evaluate similarities and commonalities.

[1651] "Means of evaluation" refers to technical means for determining the level of risk numerically or in categories based on the results of analysis and comparison.

[1652] "Means for sending warnings" refers to technical means for sending notifications to users to inform them of risks.

[1653] "Means for generating alternative information" refers to technical means that suggest alternative text, images, or videos to reduce risk.

[1654] A "user terminal" is a device such as a computer or smartphone that is directly used by a user.

[1655] "Transaction data" refers to specific transaction information such as buying and selling and remittances in electronic payment services.

[1656] "Fraud risk" is the possibility of loss due to illegal activities such as fraud or unauthorized access.

[1657] An "alternative transaction proposal" is an alternative proposed transaction content or method to reduce the risk of fraudulent use.

[1658] System configuration and program overview

[1659] The system for realizing the present invention comprises the following main hardware and software components.

[1660] Hardware:

[1661] Server: A computer system that performs data analysis and risk assessment.

[1662] User terminal: A user device such as a smartphone or personal computer.

[1663] software:

[1664] NLP library: A library for natural language processing. Example: SpaCy.

[1665] Image analysis models: Models that process and analyze image data. Examples: YOLO, SSD.

[1666] Video analysis technology: Technology that analyzes video data frame by frame. Example: OpenCV.

[1667] Generative AI model: A natural language generation model for generating alternatives. Example: GPT-3.

[1668] Program processing and behavior

[1669] In this system, the following processing is performed:

[1670] 1. Receiving posted data:

[1671] The server receives data sent from the user's device via a REST API, including text, images, and videos.

[1672] 2. Data Analysis:

[1673] Text analysis: The server uses NLP libraries to tokenize and morphologically analyze the text data, as well as sentiment analysis and keyword extraction.

[1674] Image Analysis: The server uses YOLO and SSD models to recognize objects in the image data and analyze it. It also uses OCR technology to extract text from the image.

[1675] Video analysis: The server divides the video data into frames and processes each frame in the same way as image analysis. The audio in the video is converted into text using ASR technology, and then the text is analyzed.

[1676] 3. Comparison with past scandals:

[1677] The server compares the analysis results with a database of past scandals and calculates a similarity score. It also compares with a database of fraud cases.

[1678] 4. Risk Assessment:

[1679] The server scores the risk of online outrage and abuse based on the similarity score, sentiment, and keywords. If the risk score is high, a warning is generated for the user.

[1680] 5. Risk Notification and Alternative Generation:

[1681] If the risk score is high, the server sends a risk warning to the user's device, and at the same time, it uses a generative AI model to generate alternatives to reduce the risk (such as safe trading suggestions or correction suggestions) and provides them to the user.

[1682] Specific examples

[1683] For example, consider the case where a user types "Today is the worst day. Everything is going wrong" in a social media post and then types "Is it okay to buy from this site?" as a transaction message.

[1684] Example prompt sentence:

[1685] The post read: "Today is the worst day. Everything is going wrong."

[1686] Transaction Comments: "Is it safe to buy from this site?"

[1687] The server analyzes this data, performs sentiment analysis and keyword extraction, and if it determines that a transaction is likely to be a scam or a controversy, it sends a warning to the user and offers alternatives such as "Today didn't go well, but I'll try again tomorrow!" or "This transaction is likely to be a scam, so we recommend purchasing from the official online store."

[1688] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1689] Step 1:

[1690] Input: Posted data and transaction data (text, images, videos) from user devices

[1691] Operation: The user device acquires the posting data and transaction data and sends them to the server. Specifically, the user enters text, images, or videos on the posting screen of an SNS or electronic payment app, and presses the send button to send the data.

[1692] Output: The server receives the data sent from the user terminal.

[1693] Step 2:

[1694] Input: Posting data and transaction data collected on the server

[1695] Operation: The server analyzes the received data. For text data, tokenization and morphological analysis are performed using an NLP library (e.g., SpaCy) to extract emotions and keywords. For image data, object detection is performed using YOLO or SSD models, and text is extracted using OCR technology. For video data, OpenCV is used to split the data into frames and perform the same processing as image analysis. The audio portion is also converted to text using ASR technology.

[1696] Output: Analysis results (emotion score, keywords, object recognition results, text extraction results)

[1697] Step 3:

[1698] Input: Analysis results

[1699] How it works: The server compares the analysis results with a database of past scandals and fraud cases. Specifically, it calculates a similarity score and evaluates how closely the results relate to past cases.

[1700] Output: Similarity score

[1701] Step 4:

[1702] Input: Similarity scores and analysis results

[1703] How it works: The server scores the risk of flame wars and abuse based on the similarity score, sentiment, and keywords. For example, if the similarity score exceeds a certain threshold, it is determined to be high risk.

[1704] Output: Risk score (flaming risk score, fraud risk score)

[1705] Step 5:

[1706] Input: Risk Score

[1707] How it works: If the risk score is high, the server generates a warning message for the user. It also uses a generative AI model (e.g., GPT-3) to generate alternatives to reduce the risk (such as suggestions for modifying the post or safe transactions). For example, if a post is overly negative, it suggests modifying it to a more positive one.

[1708] Output: Warning messages and alternatives

[1709] Step 6:

[1710] Input: Warning messages and alternatives

[1711] Operation: The server sends the generated warning message and alternatives to the user terminal, and the user terminal notifies the user of the received warning message and alternatives.

[1712] Output: A warning message and alternatives that are displayed to the user

[1713] Step 7:

[1714] Input: User confirms alternatives

[1715] How it works: The user reviews the warning message and alternatives they receive and decides whether to modify their posting or transaction. They can also submit the modifications to the server again.

[1716] Output: Modified submission and transaction data, or the original data

[1717] Through these steps, the system can assess in advance the risk of online outrage and fraudulent use of social media posting data and electronic transaction data, and provide users with appropriate warnings and alternatives.

[1718] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1719] The present invention relates to a system that analyzes posting data on social networking sites in advance and assesses the risk of a social networking site becoming a hot topic. By incorporating an emotion recognition engine, the present invention also analyzes the emotional aspects of posts, achieving a more comprehensive risk assessment. Specific embodiments of the system according to the present invention are described in detail below.

[1720] Server-side processing

[1721] 1. Receiving submitted data

[1722] The server receives post data (text, images, videos) from the user's device. When the user presses the post button, a POST request is sent from the device. The server receives this request using the REST API.

[1723] 2. Data Analysis

[1724] The server's AI analysis module analyzes the posted data, specifically as follows:

[1725] Sentence analysis:

[1726] A natural language processing (NLP) library is used to tokenize posts and perform contextual and sentiment analysis.

[1727] Image analysis:

[1728] Recognize objects in an image using an object detection model (e.g., YOLO or SSD).

[1729] It uses OCR technology to extract text from images, which can then be translated and analyzed.

[1730] Video Analysis:

[1731] The video is divided into frames, and each frame is processed in the same way as image analysis.

[1732] The audio portion of the video is converted into text using automatic speech recognition (ASR), and then sentiment analysis and related contextual analysis are performed.

[1733] 3. Emotion recognition

[1734] The server's emotion recognition engine detects the user's emotions from the received post data, as follows:

[1735] Text data: Natural language processing techniques are used to analyze emotions within the text. For example, emotions such as joy, anger, sadness, and happiness are identified, and an emotional assessment is made based on these.

[1736] Image data: Facial recognition and facial expression analysis techniques are used to identify the emotions of people in images. This is used in conjunction with image analysis techniques to assess the strength of emotions.

[1737] Video data: Emotions are recognized by analyzing the voice and facial expressions in the video. Image analysis and voice recognition technologies are combined to perform a comprehensive emotional evaluation.

[1738] 4. Comparison with past cases of online outrage

[1739] The server compares the analysis results and emotion recognition results with a database of past flaming cases, performs a similarity search, evaluates the degree of similarity of the posted data to past flaming cases, and calculates a similarity score.

[1740] 5. Risk Assessment

[1741] The server evaluates the risk of a controversy based on the analysis results, emotion recognition results, and similarity scores. For example, it calculates a risk score and determines that a score of 90 or higher is high risk, 70-89 is medium risk, and 70 or lower is low risk.

[1742] 6. Risk Notification and Alternative Generation

[1743] If the server determines that the risk of outrage is high, it generates a risk warning and alternatives. It uses a derived sentence generation model (e.g., GPT-3) to generate alternative sentences with less risk. If necessary, it also generates suggested revisions for images and videos.

[1744] 7. Sending warnings and alternatives

[1745] The server transmits the generated risk warning and alternatives to the user terminal.

[1746] Terminal side processing

[1747] 1. Create submission data

[1748] Users create posts through SNS applications, and can input text and upload images and videos.

[1749] 2. Submitting data

[1750] When the user presses the post button, the terminal sends the created post data to the server, for example, using a REST API POST request.

[1751] 3. Receive risk warnings

[1752] The device receives risk warnings sent from the server and notifies the user, including the risk assessment score and a specific warning message.

[1753] 4. Displaying alternatives

[1754] The terminal displays the alternatives sent from the server for reducing the risk on the user interface, and the user checks them and decides whether to modify the original post.

[1755] User operations

[1756] 1. Enter your post content

[1757] A user creates a post in a social networking application, for example, by typing, "Today is the worst day. Everything is going wrong."

[1758] 2. Risk confirmation

[1759] The user checks the risk assessment notification and warning message from the server on their device. For example, a warning message such as "This post contains negative sentiment and is at high risk of causing a backlash" may be displayed.

[1760] 3. Selection of alternatives

[1761] The user checks the alternatives provided by the server and decides whether to revise the post based on them. For example, an alternative suggestion might be "Today didn't go well, but I'll try my best tomorrow!"

[1762] 4. Confirmation of submission

[1763] The user can then modify the content of their post based on the displayed alternatives and finally confirm the post. The modified content is then sent back to the server and posted to the SNS.

[1764] In this way, the system of the present invention assesses the risk of a social media firestorm before a post is made, helping users to use social media safely. In addition, by combining it with an emotion recognition engine, it is possible to analyze the emotional aspects of posts, enabling more accurate risk assessment.

[1765] The processing flow will be explained below.

[1766] Server-side processing

[1767] Step 1:

[1768] The server receives post data (text, images, videos) from the user's device. When the user presses the post button, a POST request is sent from the device. The server receives this request via the REST API.

[1769] Step 2:

[1770] The server preprocesses the received submission data, tokenizing it for text data and storing image and video data in a specific directory.

[1771] Step 3:

[1772] The server's AI analysis module analyzes the posted data using the following steps:

[1773] Text analysis: Uses natural language processing (NLP) libraries to tokenize text and perform contextual and sentiment analysis.

[1774] Image analysis: Recognize objects and text in images using object detection models (e.g., YOLO or SSD).

[1775] Video analysis: Videos are divided into frames, and each frame is analyzed in the same way as image analysis. The audio portion of the video is converted into text using automatic speech recognition (ASR), and this is then analyzed.

[1776] Step 4:

[1777] The emotion recognition engine of the server recognizes the emotion of the user based on the received posting data.

[1778] Text data: Natural language processing techniques are used to analyze sentiment within the text, for example, identifying positive, negative, or neutral sentiment.

[1779] Image data: Using facial recognition and facial expression analysis techniques, we identify the emotions of people in images.

[1780] Video data: Analyzes the voice and facial expressions in the video, and integrates multiple emotional data to evaluate the user's overall emotions.

[1781] Step 5:

[1782] Based on the analysis results and emotion recognition results, the server compares the posted data with a database of past flaming cases, performs a similarity search, and scores the degree of similarity with past flaming cases.

[1783] Step 6:

[1784] The server evaluates the risk of a controversy based on the analysis results, emotion recognition results, and similarity scores. A score of 90 or above is considered high risk, 70-89 is considered medium risk, and 70 or below is considered low risk.

[1785] Step 7:

[1786] If the risk of a post being criticized is high, the server will notify the user of the risk and generate alternatives. For example, it will use a derived sentence generation model (e.g., GPT-3) to generate alternative sentences with less risk. In some cases, it will also generate suggested revisions for images and videos.

[1787] Step 8:

[1788] The server sends the generated risk warning and alternatives to the user's device, and if an error occurs, it records it in a log.

[1789] Terminal side processing

[1790] Step 1:

[1791] A user creates a post through a social networking application, for example, by inputting text and uploading images and videos.

[1792] Step 2:

[1793] When the user presses the post button, the terminal sends the created post data to the server, for example, using a POST request.

[1794] Step 3:

[1795] The device waits for a response from the server and receives a risk warning and alternatives.

[1796] Step 4:

[1797] The device will display risk warnings and alternatives to the user, for example as a pop-up message or notification.

[1798] User operations

[1799] Step 1:

[1800] A user inputs a post into a social networking application, for example, a sentence such as "Today is the worst day. Everything is going wrong."

[1801] Step 2:

[1802] When the user presses the post button, the post data is sent to the server.

[1803] Step 3:

[1804] The user will see a risk warning and alternatives from the server. For example, a warning message such as "This post contains negative sentiment and is at high risk of causing controversy" will be displayed.

[1805] Step 4:

[1806] The user checks the alternatives and decides whether to revise the post based on them. For example, an alternative suggestion might be "Today didn't go well, but I'll try my best tomorrow!"

[1807] Step 5:

[1808] Once the user has finalized the content of the post and pressed the post button again, the revised post data will be resent to the server and posted on the SNS.

[1809] Example 2

[1810] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1811] There is a need to provide an environment where users can safely use SNS by assessing the risk of data posted on SNS becoming a hot topic in advance. Conventional technologies simply filter the content of posts, without performing comprehensive analysis including emotional aspects and images and videos, making it difficult to accurately assess risk. The purpose of this invention is to solve these problems and provide a more comprehensive and accurate risk assessment system.

[1812] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving post data, means for analyzing the received post data using natural language processing and image analysis techniques, means for performing emotion recognition based on the analysis results, means for comparing the emotion recognition results and the analysis results with a database of past flame war cases, means for evaluating the risk of a flame war based on the comparison results, means for sending a warning to the user when the risk of a flame war is high, means for generating alternative information to reduce the risk of a flame war using a generative AI model, and means for sending the warning and alternative information to the user terminal. This makes it possible to provide a highly accurate evaluation of the risk of a flame war and appropriate alternatives before a user posts on an SNS.

[1813] "Posted data" refers to data in the form of text, images, and videos posted by users to online platforms such as social networking sites.

[1814] "Natural language processing" is a technology that enables computers to understand, interpret, and generate human language, typically involving tokenization, contextual analysis, and sentiment analysis.

[1815] "Image analysis" refers to the use of computer vision techniques to recognize and analyze objects and text in images. Examples include object detection and OCR.

[1816] "Emotion recognition" is a technology that uses natural language processing and image analysis technologies to detect emotions from text, images, and videos.

[1817] "Flame risk" is an evaluation index of the likelihood that posted data will receive a large amount of negative reactions or criticism on social media.

[1818] The "database of past social media outrage cases" is a database of past social media outrage cases, and is a data set used for risk assessment.

[1819] A "generative AI model" is an artificial intelligence model that generates new data from given input data. A specific example is GPT-3.

[1820] "Similarity search" is a technique for evaluating how similar certain data is to given reference data (in this case, past online scandals).

[1821] A "warning" is a message that notifies the user that there is a high risk of the posted data causing a firestorm.

[1822] "Alternative information" refers to modifications or alternatives suggested by generative AI models to reduce the risk of backlash.

[1823] The present invention is a system that analyzes social media posting data in advance and assesses the risk of a social media outcry. This system combines emotion recognition technology and generative AI models to provide a more comprehensive risk assessment and alternative solutions. Specific embodiments of the present invention are described in detail below.

[1824] Server-side processing

[1825] The server first receives the posted data (text, images, videos) sent from the user's device. This reception process uses a POST request using the REST API. For example, data is received through an API gateway with a set endpoint. The server uses a cloud platform to achieve scalable processing.

[1826] The server's AI analysis module then analyzes the received data using the following techniques:

[1827] Use a natural language processing (NLP) library (e.g., spaCy or BERT) to tokenize the text data and perform contextual and sentiment analysis.

[1828] It uses object detection models (e.g., YOLO and SSD) to identify objects in images, and OCR technology to extract text from images for translation and sentiment analysis.

[1829] In video analysis, the video is divided into frames and each frame is processed in the same way as image analysis. The audio portion of the video is converted into text using automatic speech recognition (ASR), and then emotion and context analysis are performed.

[1830] Emotion recognition is performed to detect user emotions from received posting data. This process uses natural language processing technology to detect emotions in text and identify emotions such as joy, anger, sadness, and happiness. It also uses facial recognition and facial expression analysis technology to analyze the emotions of people in images and videos.

[1831] Based on the analysis results and emotion recognition results, the server compares them with a database of past flaming cases. Using a similarity search technique, the server evaluates how similar the posted data is to past flaming cases and calculates a similarity score. Specifically, the comparison uses techniques such as Cosine similarity.

[1832] The server then assesses the risk of a controversy based on the analysis results, emotion recognition results, and similarity scores. A score of 90 or higher is considered high risk, 70-89 is considered medium risk, and 70 or lower is considered low risk.

[1833] If the risk of outrage is high, the server generates a risk warning and alternative suggestions. Using a generative AI model (e.g., GPT-3), it generates alternative text to reduce the risk. If necessary, it also generates suggested edits for images and videos. The generated alternative suggestions and risk warnings are converted into JSON format and sent to the user's device.

[1834] Terminal side processing

[1835] First, a user creates a post through a social networking application. They enter text and upload images and videos. When the user presses the post button, this post data is sent to the server via a REST API POST request.

[1836] When a risk warning is sent from the server, the device receives it and notifies the user. The risk assessment score and specific warning message are displayed on the interface, and the device also displays the alternatives sent from the server. The user can review the alternatives and decide whether to revise the original post.

[1837] User operations

[1838] A user creates a post in a social networking application. For example, the user enters the sentence, "Today is the worst day. Everything is going wrong." The server determines that this post contains strong negative emotions and sends a warning message to the device indicating a high risk of it starting a flame war. The warning message reads, "This post contains negative emotions and there is a high risk of it starting a flame war."

[1839] The user also checks alternative suggestions provided by the server. For example, an alternative sentence such as "Today didn't go well, but I'll try my best tomorrow!" is presented. The user can then revise the post based on this and finally confirm the post. The revised post is then sent back to the server and posted to the SNS.

[1840] This system allows users to use social media safely and assess the risk of a social media firestorm in advance. Furthermore, by using emotion recognition technology, it is possible to analyze the emotional aspects of posts and improve the accuracy of risk assessment.

[1841] Specific examples

[1842] For example, if a user tries to post "Today is the worst day. Everything is going wrong," the server analyzes it and determines that it is a post with a strong negative sentiment. If it is assessed as having a high risk of causing a flame war, the server generates an alternative suggestion, such as "Today didn't go well, but I'll try my best tomorrow!" and presents it to the user.

[1843] Prompt Sentence Examples

[1844] Input: Today is the worst day. Everything is going wrong.

[1845] Output: Things didn't go well today, but I'll try my best tomorrow!

[1846] This prompt sentence can be used to train a generative AI model (e.g., GPT-3) to generate a less risky alternative sentence.

[1847] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1848] Step 1: Receive post data

[1849] The server receives post data from the user's device. When a user presses the post button in the SNS application, a POST request is sent from the device. The input is the text, image, and video data created by the user, and the output is the post data received by the server. Specifically, the server receives this request using a REST API and retrieves the data through an API gateway with a configured endpoint.

[1850] Step 2: Data analysis

[1851] The AI ​​analysis module on the server analyzes the received post data. Specifically,

[1852] Text analysis: Using a natural language processing (NLP) library (e.g., spaCy or BERT), text data is tokenized and context and sentiment analysis is performed. The input is text data, and the output is the analysis results.

[1853] Image analysis: Identify objects in images using object detection models (such as YOLO or SSD). Also, extract text from images using OCR technology and perform sentiment analysis. The input is image data, and the output is the analysis results.

[1854] Video analysis: Videos are divided into frames, and each frame is processed in the same way as image analysis. The audio portion of the video is converted into text using automatic speech recognition (ASR), and emotion and context analysis is performed. The input is video data, and the output is the analysis results data.

[1855] Step 3: Emotion Recognition

[1856] The server's emotion recognition engine detects the user's emotions from the analyzed post data. Specifically,

[1857] Text data: Detect emotions in text using natural language processing techniques. The input is the result of text analysis, and the output is the emotion recognition result of the text.

[1858] Image data: Using facial recognition and facial expression analysis technology, we identify the emotions of people in an image. The input is the result of image analysis, and the output is the emotion recognition result of the image.

[1859] Video data: Analyzes the voice and facial expressions in the video to perform a comprehensive emotional evaluation. The input is the result of video analysis data, and the output is the emotion recognition results for the video.

[1860] Step 4: Compare with past flame war cases

[1861] The server compares the analysis results and emotion recognition results with a database of past flaming cases. Specifically, it uses a similarity search method to evaluate how similar the posted data is to past flaming cases. The input is the analysis results and emotion recognition results, and the output is a similarity score. Cosine similarity, for example, is used to evaluate the similarity.

[1862] Step 5: Risk assessment

[1863] The server evaluates the risk of a controversy based on the analysis results, emotion recognition results, and similarity score. It calculates a risk score and classifies a score of 90 or above as high risk, 70-89 as medium risk, and 70 or below as low risk. The inputs are the similarity score, emotion recognition results, and analysis results, and the output is the risk assessment score and its judgment result.

[1864] Step 6: Risk notification and alternative generation

[1865] If the server determines that there is a high risk of outrage, it generates a risk warning and alternatives. It uses a generative AI model (e.g., GPT-3) to generate alternative text with less risk. If necessary, it also generates suggested modifications for images and videos. The input is the risk assessment score and the judgment result, and the output is a risk warning message and alternatives.

[1866] Step 7: Sending warnings and alternatives

[1867] The server sends the generated risk warning and alternatives to the user's device. The input is the risk warning message and alternatives, and the output is the data to be sent to the user's device. The risk warning and alternatives are converted to JSON format and sent to the user's device via a REST API.

[1868] Step 8: Receive risk warnings

[1869] The terminal receives the risk warning sent from the server and notifies the user. The input is the risk warning message and alternatives sent from the server, and the output is the notification to the user interface.

[1870] Step 9: View alternatives

[1871] The terminal displays the alternatives sent from the server on the user interface. The user checks them and decides whether to modify the original post. The input is the alternatives, and the output is the display content presented to the user.

[1872] Step 10: User Edits to Posts

[1873] The user modifies the post based on the alternatives provided by the server. The input is the alternatives and the original post, and the output is the modified post.

[1874] Step 11: Finalize and submit your revised post

[1875] The user confirms the revised content of the post and sends it back to the server. The input is the revised content of the post, and the output is the final post data sent to the server. The server receives this revised post and posts it to the SNS.

[1876] (Application example 2)

[1877] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1878] In recent years, the risk of user posts on social media and online platforms causing online outrage has become a social issue. In particular, for brick-and-mortar stores, there is a growing risk that negative customer feedback spreading online could damage their reputation and negatively impact sales. It is also important to quickly analyze customer feedback and provide risk assessments and appropriate countermeasures. The present invention aims to solve these problems by providing a system that enables brick-and-mortar stores to analyze customer feedback and quickly take appropriate countermeasures.

[1879] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1880] In this invention, the server includes means for receiving posted data, means for analyzing the received posted data and comparing it with past cases of online flaming, means for assessing the risk of online flaming based on the analysis results, means for sending a warning to the user when the risk of online flaming is high, means for generating alternative information to reduce the risk of online flaming, means for sending the warning and alternative information to the user terminal, means for analyzing customer feedback at the physical store, means for performing sentiment analysis of the posted data and assessing the risk, and means for generating alternatives that arise when the risk is high. This enables quick and accurate analysis of customer feedback and risk assessment, and appropriate action to protect the reputation of the physical store.

[1881] "Posted data" refers to information such as text, images, and videos that users create and send on social media or online platforms.

[1882] "Means for receiving" refers to the functions and protocols that allow the server to receive posted data sent from the user terminal.

[1883] "Means of analysis" refers to the process of analyzing the content of received posted data using natural language processing and image analysis technology.

[1884] "Past cases of online outrage" refer to cases in the past where a user's post on a social media or online platform sparked criticism or negative reactions.

[1885] "Means of comparison" refers to the algorithms and methods used to evaluate the similarity of newly received posted data with past cases of online outrage.

[1886] "Flame risk" refers to the likelihood that a particular post will provoke criticism or negative reactions in the online community.

[1887] "Sentiment analysis" refers to the process of identifying emotional components in text or images using natural language processing and image recognition techniques.

[1888] "Risk assessment" refers to the process of evaluating the degree of risk of a post causing a controversy using numerical or categorized terms based on the analysis of received post data.

[1889] "Means for sending warnings" refers to the function of sending warning messages to users when there is a high risk of a social media outburst.

[1890] "Alternative information" refers to safer options, such as text or images that have been modified from the original post to reduce the risk of a backlash.

[1891] "Means for generating alternatives" refers to a process that uses generative AI models or other technologies to automatically create safe alternative information.

[1892] "User terminal" refers to devices such as smartphones, tablets, and PCs used to access platforms such as social networking sites.

[1893] "Brick and mortar store" refers to a store that offers goods or services in a physical location.

[1894] "Customer feedback" refers to customer experiences, opinions, and evaluations expressed in the form of posts in physical stores.

[1895] The present invention relates to a system that analyzes posted data in advance and evaluates the risk of a social media firestorm. This system is primarily intended to analyze customer feedback in brick-and-mortar stores, and is designed to prevent negative feedback from damaging the store's reputation. Specific embodiments are described below.

[1896] Server-side processing

[1897] 1. Receiving submitted data

[1898] The server receives the posted data (text, images, videos) sent from the user's device. When the user sends feedback, a POST request is sent from the user's device via the REST API. The server receives this request and prepares to analyze the posted data.

[1899] 2. Data Analysis

[1900] The server program analyzes the submitted data using the following hardware and software:

[1901] Natural Language Processing (NLP) library: Uses TextBlob to tokenize posts and perform contextual and sentiment analysis.

[1902] Image Analysis: Extract text in images using OpenCV and pytesseract and perform analysis on the same.

[1903] The analysis results are compared with a database of past online outrage cases and a similarity search is performed.

[1904] 3. Emotion recognition

[1905] The server analyzes the sentiment of the posted data through the following process:

[1906] Text data: Natural language processing techniques are used to identify sentiment within the text.

[1907] Image data: Use OCR technology to extract text from images and analyze their sentiment.

[1908] 4. Risk assessment and alternative generation

[1909] The server evaluates the risk of a controversy based on the analysis results and emotion recognition results. If the risk is determined to be high, it uses a generative AI model (e.g., GPT-3) to generate alternatives to reduce the risk.

[1910] Terminal side processing

[1911] The user terminal provides the following functions:

[1912] Feedback submission: Users can enter feedback in the form of text, images, or videos using their smartphones.

[1913] Display risk warning: Receives risk warnings sent from the server and notifies the user.

[1914] Displaying alternatives: The alternatives sent from the server are displayed in the user interface, allowing the user to choose whether to revise the post.

[1915] Specific examples

[1916] For example, if a customer posts feedback saying, "Today's customer service was terrible," this post is sent to the server. On the server side, negative sentiment is detected using TextBlob and evaluated as high risk. In response, a generative AI model is used to generate an alternative suggestion, such as, "Today was a bit disappointing, but I look forward to your next visit," and this is presented to the user.

[1917] Prompt Sentence Examples

[1918] "Generate positive alternatives for the user-submitted feedback, 'The customer service today was terrible.'"

[1919] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1920] Step 1:

[1921] The user uses their smartphone to enter feedback, which can be in the form of text, images, or videos. The entered feedback might be something like "Today is the worst day. Everything is going wrong." When the user presses the submit button, the device sends the feedback data to the server as a POST request.

[1922] Step 2:

[1923] The server receives the posted data sent from the user's device. The received data can be text, images, or videos, and is sent via the REST API. The server temporarily stores the received data.

[1924] Step 3:

[1925] The server analyzes the received post data. For text data, it uses TextBlob to tokenize the data, analyzes the context, and performs sentiment analysis. The input for analysis is the post text, and the output is the sentiment polarity (positive, negative, neutral). For example, "Today is the worst day. Everything is going wrong." is analyzed as negative.

[1926] Step 4:

[1927] The server analyzes the image data. It extracts the text from the image using OpenCV and performs OCR analysis using pytesseract. The input for the analysis is the posted image, and the extracted text is again subjected to sentiment analysis using TextBlob. The output is the text from the image and the results of the sentiment analysis.

[1928] Step 5:

[1929] The server analyzes the video data. It divides the video into frames and performs image analysis on each frame. It also converts the audio portion of the video into text using automatic speech recognition (ASR), and performs sentiment analysis on that text. The input for the analysis is the posted video, and the output is the text for each frame and the sentiment analysis results.

[1930] Step 6:

[1931] The server compares the analysis results with a database of past flaming cases. It uses a similarity algorithm to evaluate how similar the analysis data is to past flaming cases. The input is the analysis results and past flaming cases, and the output is a similarity score.

[1932] Step 7:

[1933] The server evaluates the risk of a controversy based on the analysis results and the similarity score. The emotional polarity and similarity scores are combined to calculate an overall risk score. The output is a risk score; for example, a risk score of 90 or higher is considered high risk.

[1934] Step 8:

[1935] If the risk of a firestorm is determined to be high, the server sends a risk warning to the user's device. The warning includes a risk score and a specific warning message. The input is the risk score, and the output is the warning message.

[1936] Step 9:

[1937] To generate alternatives, the server uses a generative AI model. For example, it uses a prompt such as "Generate a positive alternative to the user-posted feedback, 'Today is the worst day. Everything is going wrong.'" The input is the original post text and the prompt, and the output is the alternative text. For example, an alternative such as "Today didn't go well, but let's try our best tomorrow!" is generated.

[1938] Step 10:

[1939] The server sends the generated alternatives to the user's terminal. The user can check the alternatives and choose whether to correct the feedback. The input is the alternative text, and the output is a notification to the user's terminal.

[1940] Step 11:

[1941] The user then takes the suggested alternatives into con...

Claims

1. means for receiving posted data; A method for analyzing received posting data and comparing it with past cases of online outrage, A means for assessing the risk of a firestorm based on the analysis results; a means for sending a warning to users when there is a high risk of flame war; A means for generating alternative information to reduce the risk of flame wars; means for transmitting a warning and alternative information to a user terminal; A system including:

2. 2. The system according to claim 1, wherein the posted data is any one of text, images, and videos.

3. The system according to claim 1, wherein the risk of a flaming incident is evaluated using a database of past flaming incidents.

Images