system

Abstract

To provide a system.SOLUTION: A system includes: means for monitoring SNS utilization of a user's child; means for using an AI model so as to detect an unsafe text; means for monitoring an exchange of SNS by using API of an SNS platform; means for notifying the user or a manager of the SNS in real time when detecting the risk; means for improving the system on the basis of feedback; means for estimating a feeling state of the user's child from text by using a feeling engine; means for detecting a contribution and a message including a specific keyword; and means for sending notification on the basis of a detected feeling state or a keyword.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a problem that the detection and response to dangerous texts when children use social networking sites is not being done properly. Also, even if such systems exist, there is a problem that feedback for improvement is not being provided properly. [Means for solving the problem]

[0005] This invention monitors children's use of social media and detects dangerous texts using an AI model. Furthermore, it monitors social media interactions using the social media platform's API and notifies parents or administrators in real time when a dangerous situation is detected. The system then improves based on feedback, thereby resolving the above-mentioned issues. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] One embodiment of the present invention is a system for monitoring children's use of social networking sites. This system monitors children's social networking site interactions via the API of the social networking site platform used by the children. Specifically, it obtains texts posted by the children and messages received by them.

[0029] "Example 2"

[0030] Furthermore, the system of the present invention inputs the acquired text into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing technology. Specifically, it uses a natural language generation model, such as generative AI, to understand the content of the text and determine whether the text contains dangerous content.

[0031] "Example 3"

[0032] Additionally, the system of the present invention notifies parents or administrators in real time when dangerous text is detected. This notification can be sent via email, SMS, or push notification. Specifically, a notification containing information about the dangerous text, including the time and location of the text, is sent to contacts registered by the parent or administrator.

[0033] "Example 4"

[0034] Furthermore, the system of the present invention improves the system based on feedback. Specifically, it uses feedback from parents or administrators, as well as data automatically collected by the system, to improve the accuracy of the AI ​​model. For example, if a parent or administrator provides feedback such as "this text is not dangerous," the AI ​​model is updated to determine that the text is not dangerous.

[0035] The processing flow of each embodiment will be described below.

[0036] "Example 1"

[0037] Step 1: Monitor the child's social media interactions via the API of the social media platform they use, specifically by capturing text messages posted and received by the child.

[0038] Step 2: The acquired text is fed into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing techniques.

[0039] Step 3: If the AI ​​model detects dangerous text, it notifies parents and administrators in real time via email, SMS, or push notification.

[0040] Step 4: Improve the accuracy of the AI ​​model using feedback from parents and administrators, as well as data collected automatically by the system.

[0041] "Example 2"

[0042] Step 1: Monitor the child's social media interactions via the API of the social media platform they use, specifically by capturing text messages posted and received by the child.

[0043] Step 2: The acquired text is fed into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing techniques.

[0044] Step 3: If the AI ​​model detects dangerous text, it notifies parents and administrators in real time via email, SMS, or push notification.

[0045] Step 4: Improve the accuracy of the AI ​​model using feedback from parents and administrators, as well as data collected automatically by the system.

[0046] "Example 3"

[0047] Step 1: Monitor the child's social media interactions via the API of the social media platform they use, specifically by capturing text messages posted and received by the child.

[0048] Step 2: The acquired text is fed into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing techniques.

[0049] Step 3: If the AI ​​model detects dangerous text, it notifies parents and administrators in real time via email, SMS, or push notification.

[0050] Step 4: Improve the accuracy of the AI ​​model using feedback from parents and administrators, as well as data collected automatically by the system.

[0051] "Example 4"

[0052] Step 1: Monitor the child's social media interactions via the API of the social media platform they use, specifically by capturing text messages posted and received by the child.

[0053] Step 2: The acquired text is fed into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing techniques.

[0054] Step 3: If the AI ​​model detects dangerous text, it notifies parents and administrators in real time via email, SMS, or push notification.

[0055] Step 4: Improve the accuracy of the AI ​​model using feedback from parents and administrators, as well as data collected automatically by the system.

[0056] Example 1

[0057] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0058] Today's children frequently use social media, increasing the risk of being exposed to inappropriate content and dangerous interactions. However, it is difficult for parents and administrators to monitor these interactions in real time and respond appropriately. Conventional monitoring systems have low accuracy in detecting dangerous content and often delay notifications, so a rapid response is required.

[0059] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for monitoring children's use of SNS, a means for using a generative AI model to detect dangerous text, a means for monitoring SNS interactions using the API of the SNS platform, a means for notifying parents or administrators in real time when a risk is detected, a means for improving the system based on feedback, a means for analyzing text using natural language processing technology, a means for analyzing acquired data and detecting inappropriate content, and a means for sending a notification to parents based on the analysis results. This makes it possible to effectively monitor children's use of SNS, quickly detect dangerous interactions, and notify parents.

[0060] "Means for monitoring children's use of social media" refers to technologies for monitoring activities on social media platforms used by children and obtaining the content of posts and messages.

[0061] A "generative AI model for detecting dangerous text" is an artificial intelligence model used to analyze text using natural language processing techniques to detect inappropriate content or dangerous interactions.

[0062] "Means for monitoring social media interactions using the API of a social media platform" refers to technology that utilizes the application programming interface (API) provided by a social media platform to obtain and monitor user posts and messages.

[0063] "Means for notifying parents or administrators in real time when danger is detected" refers to technology that immediately sends a notification to parents or administrators when dangerous content is detected.

[0064] "Means for improving the system based on feedback" refers to techniques for collecting system usage results and feedback from users, and using that information to improve the system's performance and accuracy.

[0065] "Means for analyzing text using natural language processing technology" refers to natural language processing (NLP) technology used to analyze text data and understand its content.

[0066] "Means for analyzing acquired data and detecting inappropriate content" refers to technology for analyzing data acquired from social media platforms and detecting inappropriate content contained therein.

[0067] The "means for sending a notification to the guardian based on the analysis result" is a technology for sending a notification to the guardian as necessary based on the results of the data analysis.

[0068] This invention is a system for monitoring children's use of social networking sites and detecting dangerous texts. This system includes elements of a server, a terminal, and a user, and operates as follows.

[0069] The server first obtains data related to the child's social media account using the social media platform's API. Specifically, the server authenticates using the API key and secret key provided by the social media platform and then obtains the child's posts and messages. For example, the server can use the Twitter API or Facebook Graph API.

[0070] The server then uses natural language processing (NLP) techniques to analyze the retrieved data. Specifically, it uses generative AI models to analyze the text and detect inappropriate content or dangerous interactions. NLP libraries used include NLTK and spaCy.

[0071] Based on the analysis results, the server sends real-time notifications to parents and administrators as needed via email, SMS, or push notifications, allowing parents and administrators to immediately understand their children's social media usage and take appropriate action.

[0072] Furthermore, the server improves the system based on feedback. Specifically, it collects system usage results and user feedback, and uses this information to improve the performance and accuracy of the generative AI model.

[0073] As a concrete example, consider a child using Twitter. The server retrieves the child's latest posts using the Twitter API and analyzes the text using a generative AI model. If inappropriate content is detected based on the analysis results, a notification is sent to the parent via email.

[0074] Examples of prompt sentences include the following:

[0075] "Please create a system to monitor children's social media usage. Use the Twitter API to retrieve children's posts and use a natural language processing library to analyze them for inappropriate content. Send notifications to parents based on the analysis results."

[0076] In this way, the server can effectively monitor children's use of social networking sites, quickly detect dangerous interactions, and notify parents.

[0077] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0078] Step 1: API Authentication

[0079] The server performs authentication using the API key and secret key of the SNS platform. The API key and secret key are required as input. This gives the server permission to obtain data from the SNS platform. Specifically, the server sends an authentication request using the API key and secret key and obtains an authentication token.

[0080] Step 2: Data Acquisition

[0081] After successful authentication, the server retrieves posts and messages related to the child's account. The input requires an authentication token and the child's account information. The output is the retrieved post and message data. Specifically, it sends a GET request using the social media platform's API to retrieve the child's latest posts and messages.

[0082] Step 3: Data analysis

[0083] The server uses natural language processing (NLP) techniques to analyze the retrieved data. The input requires retrieved post and message data. The output is the analysis results. Specifically, it uses a generative AI model to analyze the text and detect inappropriate content or dangerous interactions. NLP libraries used include NLTK and spaCy.

[0084] Step 4: Hazard detection

[0085] The server determines whether dangerous content has been detected based on the results of the data analysis. The input required is the results of the data analysis. The output is a flag indicating a danger detection. Specifically, the server evaluates the analysis results and checks whether they contain inappropriate words or phrases.

[0086] Step 5: Notification

[0087] The server sends real-time notifications to parents and administrators when a risk is detected. The inputs required are the risk detection flag and analysis results. The output is confirmation that a notification has been sent. Specific actions include sending a notification to parents and administrators via email, SMS, or push notification.

[0088] Step 6: Gather feedback

[0089] The server collects system usage results and user feedback. The inputs required are system usage results and user feedback. The output is the collected feedback data. Specifically, the server collects feedback through the user interface and stores it in a database.

[0090] Step 7: System Improvement

[0091] The server improves the system based on the collected feedback. The input requires the collected feedback data. The output is an improved system. Specifically, it makes adjustments to improve the performance and accuracy of the generative AI model, thereby improving the overall system.

[0092] (Application example 1)

[0093] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0094] Today's children frequently use social networking sites, increasing their risk of being exposed to inappropriate content and dangerous interactions. However, it is difficult for parents and administrators to monitor these risks in real time and respond appropriately. Conventional monitoring systems have low accuracy in detecting dangerous content and often delay notification to parents and administrators. Therefore, there is a need for an effective monitoring system to ensure children's safety.

[0095] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0096] In this invention, the server includes means for monitoring children's use of SNS, means for using a generative AI model to detect dangerous text, means for monitoring SNS interactions using the API of the SNS platform, means for notifying parents or administrators in real time when danger is detected, means for improving the system based on feedback, means for generating and providing regular reports to parents or administrators, and means for inputting prompt sentences into the generative AI model to analyze the content of the text, thereby making it possible to effectively monitor children's use of SNS and quickly detect dangerous content and notify parents or administrators.

[0097] "Means for monitoring children's use of social media" refers to technologies for tracking activities on social media platforms used by children and obtaining the content of posts and messages.

[0098] "Means using generative AI models to detect dangerous text" refers to technology that uses generative AI models to analyze text on social media and automatically detect inappropriate or dangerous content.

[0099] "Methods of monitoring social media interactions using the API of a social media platform" refers to technology that utilizes the API provided by a social media platform to obtain and monitor user posts and messages.

[0100] "Means for notifying parents or administrators in real time when danger is detected" refers to technology that sends immediate notifications to parents or administrators when dangerous content is detected.

[0101] "Means for improving the system based on feedback" refers to techniques for collecting feedback from users and improving the performance and accuracy of the system based on that feedback.

[0102] The "means for generating periodic reports and providing them to parents and administrators" refers to a technology for generating reports summarizing the monitoring results at regular intervals and providing them to parents and administrators.

[0103] "Means for inputting prompt sentences into a generative AI model to analyze the content of text" refers to a technology in which a specific prompt sentence is input into a generative AI model and the resulting analysis results are used to evaluate the content of the text.

[0104] As an embodiment of the present invention, the following system can be constructed.

[0105] First, to monitor a child's use of social media, the server uses the social media platform's API to retrieve the child's posts and messages. Specifically, the server accesses the API endpoint provided by the social media platform and uses authentication information to retrieve data related to the child's account. This data includes posted text and received messages.

[0106] The server then inputs the acquired text data into a generative AI model to detect dangerous text. The generative AI model uses natural language processing techniques to analyze the content of the text and automatically detect inappropriate or dangerous content. This analysis can use natural language processing libraries such as Hugging Face's transformers library.

[0107] Examples of prompts to input to a generative AI model include:

[0108] "Analyze the following message for potential risks: 'I hate you. Don't talk to me anymore.'"

[0109] If dangerous text is detected, the server sends real-time notifications to parents and administrators via email, SMS, or push notification, allowing them to take immediate action.

[0110] In addition, the server periodically generates a report summarizing the monitoring results and provides it to parents or administrators. This report includes a list of detected dangerous text and analysis results, making it easier for parents and administrators to understand their children's social media usage.

[0111] Finally, the server collects user feedback and uses it to improve the system's performance and accuracy. Feedback is based on ratings and comments provided by parents and administrators. This allows the system to continually improve and become more effective at monitoring.

[0112] In this way, a system can be realized that can effectively monitor children's use of social networking sites, quickly detect dangerous content, and notify parents or administrators.

[0113] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0114] Step 1:

[0115] The server retrieves the child's posts and messages using the SNS platform's API. The SNS platform's API endpoint and authentication information are used as input. The retrieved text data (posts and messages) is obtained as output. Specifically, the server sends an HTTP request to the API endpoint and receives the text data as a response.

[0116] Step 2:

[0117] The server inputs the acquired text data into the generative AI model to detect dangerous text. The acquired text data and prompt text are used as input. The output is the analysis result of the text (risk assessment). Specifically, the prompt text is input into the generative AI model, and the model analyzes the content of the text and assesses the risk level.

[0118] Step 3:

[0119] The server sends real-time notifications to parents and administrators when dangerous text is detected. It uses the analysis results and notification destination information as inputs. It outputs the notification. Specific behaviors include sending a notification to the parent or administrator via email, SMS, or push notification.

[0120] Step 4:

[0121] The server periodically generates a report summarizing the monitoring results and provides it to the parent or administrator. The monitoring result data is used as input. As output, a report is generated and provided to the parent or administrator. Specifically, the monitoring results are compiled, compiled into a report format, and sent via email or other means.

[0122] Step 5:

[0123] The server collects user feedback and uses it to improve the system's performance and accuracy. It uses feedback from parents and administrators as input. It outputs improvements to the system. Specifically, it stores the feedback in a database, analyzes it, and adjusts the system's algorithms and settings.

[0124] Example 2

[0125] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0126] Modern social networking sites (SNS) pose a growing risk of children posting dangerous texts. These dangerous texts may contain violent, discriminatory, or threatening content, and may pose serious danger to children themselves or others. Conventional monitoring systems face the challenge of detecting these dangerous texts in real time and taking appropriate action.

[0127] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for monitoring children's use of the information exchange service, means for using a generative AI model for detecting dangerous text, means for monitoring interactions on the information exchange service using an application program interface of the information exchange service platform, means for notifying parents or administrators in real time when a risk is detected, means for improving the system based on the feedback, means for acquiring text entered by a user, means for inputting the acquired text into the generative AI model, means for the generative AI model to analyze the text and determine whether the text is dangerous, and means for notifying the user of the analysis result. This makes it possible to detect dangerous text before children post it and take appropriate measures.

[0128] "Children" refers to minors, particularly those who require supervision when using information exchange services.

[0129] "Information Exchange Service" refers to an online platform for users to post and share messages and content.

[0130] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to analyze text and understand and judge its content.

[0131] "Application Program Interface" refers to an interface for exchanging data and functions between different software systems.

[0132] "Real-time notification" refers to a function that notifies you immediately when a specific event occurs.

[0133] "Feedback" refers to evaluations and opinions provided based on system performance and results.

[0134] "Text" refers to written data posted by users to information exchange services.

[0135] "Analysis" refers to the process by which a generative AI model understands and evaluates the content of a text.

[0136] "Notification" refers to a message used to communicate analysis results to users and administrators.

[0137] MODE FOR CARRYING OUT THE INVENTION

[0138] This invention is a system that detects dangerous text in real time when a child uses a social networking service (SNS) and takes appropriate action. A specific embodiment of this system will be described below.

[0139] Hardware and software used

[0140] The system uses the following hardware and software:

[0141] Server: A central processing unit for receiving, analyzing, and notifying text data.

[0142] Device: A device (smartphone, computer, etc.) that allows a user to enter text.

[0143] Generative AI model: An artificial intelligence model that analyzes text using natural language processing techniques (e.g., ChatGPT (registered trademark)).

[0144] Data processing and calculation

[0145] 1. Get the text:

[0146] A user uses a terminal to input text to be posted to the information exchange service, and the terminal transmits this text to the server.

[0147] 2. Text Analysis:

[0148] The server inputs the received text into a generative AI model, which uses natural language processing techniques to analyze the content of the text and determine whether it contains dangerous content.

[0149] 3. Notification of analysis results:

[0150] The server receives the analysis results from the generative AI model and notifies the user based on the results via email, short message service, or push notification.

[0151] Specific examples

[0152] For example, suppose a user is trying to post a message on social media that reads, "I'm going to plant a bomb at school tomorrow." The server retrieves this text and inputs it into a generative AI model.

[0153] Example prompt sentence:

[0154] Please determine whether the following text contains dangerous content:

[0155] "Tomorrow I'm going to plant a bomb in the school."

[0156] The generative AI model analyzes this prompt and determines that the text contains the dangerous content of "planting a bomb." As a result, the server determines that the text is dangerous and issues a warning to the user.

[0157] Example of a warning message:

[0158] WARNING: The text you entered contains potentially dangerous content. Please stop posting.

[0159] In this way, the server can use the generative AI model to determine the risk of the text provided by the user and take appropriate action.This system can detect dangerous texts before children post them and notify parents or administrators in real time.

[0160] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0161] Step 1:

[0162] The user enters text

[0163] A user uses a terminal (such as a smartphone or PC) to input text to be posted to an information exchange service. The input text is temporarily stored in the terminal's memory. The input data is the text entered by the user, and the output data is the text stored in the terminal.

[0164] Step 2:

[0165] The device sends the text to the server

[0166] The terminal sends the text entered by the user to the server, where the text data is securely transmitted over the Internet. The input data is the text stored on the terminal, and the output data is the text sent to the server.

[0167] Step 3:

[0168] The server receives the text

[0169] The server receives the text sent from the terminal. The received text is temporarily stored for the next processing step. The input data is the text sent to the server, and the output data is the text stored on the server.

[0170] Step 4:

[0171] The server inputs the text into the generative AI model

[0172] The server inputs the received text into the generative AI model. At this time, it generates a prompt sentence and sends it to the generative AI model along with the text. The input data is the text stored on the server, and the output data is the prompt sentence and text sent to the generative AI model.

[0173] Specific prompt examples:

[0174] Please determine whether the following text contains dangerous content:

[0175] "Tomorrow I'm going to plant a bomb in the school."

[0176] Step 5:

[0177] Generative AI models parse text

[0178] The generative AI model analyzes the prompts and text sent from the server. It uses natural language processing technology to understand the context and keywords of the text and evaluate whether it contains dangerous content. The input data are the prompts and text sent to the generative AI model, and the output data are the analysis results.

[0179] Step 6:

[0180] The server receives the analysis results

[0181] The server receives the analysis results from the generative AI model, which include a judgment of whether the text is safe or unsafe. The input data is the analysis results sent from the generative AI model, and the output data is the analysis results stored on the server.

[0182] Step 7:

[0183] The server notifies the user of the analysis results

[0184] The server notifies the user based on the analysis results. For example, if the text is deemed dangerous, it displays a warning message to the user. The notification can be via email, a short message service, or a push notification. The input data is the analysis results stored on the server, and the output data is the notification message sent to the user.

[0185] Specific warning message example:

[0186] WARNING: The text you entered contains potentially dangerous content. Please stop posting.

[0187] (Application example 2)

[0188] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0189] Modern information and communication services are increasing the risk that children will receive dangerous texts and messages. In particular, phishing emails and messages containing inappropriate content are becoming a problem. This not only threatens children's safety, but also makes it difficult for parents and administrators to respond appropriately. Conventional systems have had difficulty detecting dangerous texts in real time and suggesting appropriate countermeasures.

[0190] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0191] In this invention, the server includes means for monitoring a child's use of information and communication services, means for using a generative AI model to detect dangerous text, means for monitoring interactions with information and communication services using an application programming interface of an information and communication service platform, means for notifying a parent or administrator in real time when a danger is detected, means for improving the system based on the feedback, means for analyzing the text of messages and emails received by a user in real time to determine whether the messages or emails contain dangerous content, and means for displaying a warning to the user and suggesting an appropriate response method when dangerous content is detected. This allows a child to receive an immediate warning when they receive a dangerous text, enabling parents or administrators to respond quickly.

[0192] "Information and communication services" are means of communication provided via the Internet, such as messaging, email, and social networking.

[0193] A "generative AI model" is an artificial intelligence model that uses natural language processing techniques to analyze text and generate output tailored to a specific task.

[0194] An "application programming interface" is a standardized interface that allows different software systems to communicate with each other.

[0195] "Real-time notification" is a function that immediately notifies users of information when a specific event occurs.

[0196] "Feedback" is information for evaluation and improvement based on the performance and results of a system.

[0197] A "message" is information such as text, images, and audio sent and received between users.

[0198] "Email" is a digital form of letter sent and received over the Internet.

[0199] A "warning" is a notification that notifies the user that a danger or problem has occurred.

[0200] A "solution" is an action or measure taken in response to a particular problem or situation.

[0201] The following system configuration will be described as an embodiment of the present invention.

[0202] System Configuration

[0203] The system includes a server, a user terminal, and a generative AI model. The server receives the text of messages and emails sent from the user terminal and analyzes the text using the generative AI model. If the text contains dangerous content based on the analysis results, the server notifies the user in real time.

[0204] Hardware and software used

[0205] Hardware: Smartphones, servers

[0206] Software: Python, OpenAI (registered trademark) API, smtplib (Python standard library)

[0207] Data processing and calculation

[0208] 1. Text acquisition: The user terminal sends the text of the received message or email to the server.

[0209] 2. Text analysis: The server inputs the received text into a generative AI model and analyzes the text using natural language processing techniques.

[0210] 3. Risk determination: The generative AI model determines whether the text contains risky content.

[0211] 4. Warning Notification: If dangerous content is detected, the server will notify the user in real time via email, short message service, or push notification.

[0212] Specific examples

[0213] For example, if a user receives a phishing email, the content of the email will be analyzed, and if it is deemed dangerous, a warning will be displayed. Specific examples of prompt text are as follows:

[0214] Prompt Sentence Examples

[0215] Determine if the following text is dangerous: "Your account has been hacked. Click this link immediately to change your password."

[0216] In this way, users can be immediately alerted when they receive a dangerous text, allowing parents or administrators to respond quickly.

[0217] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0218] Step 1:

[0219] Obtain the text of messages and emails received by the user terminal.

[0220] Input: Text of a received message or email

[0221] Output: The retrieved text data

[0222] Specific operation: The user device scans the inbox for new messages and emails and extracts their contents in text format.

[0223] Step 2:

[0224] The user terminal transmits the acquired text data to the server.

[0225] Input: Obtained text data

[0226] Output: Text data sent to the server

[0227] Specific operation: The user terminal sends text data to the server using a communication method such as an HTTP request.

[0228] Step 3:

[0229] The server inputs the received text data into the generative AI model.

[0230] Input: Text data sent to the server

[0231] Output: Text data fed into the generative AI model

[0232] Specific operation: The server converts the received text data into an appropriate format for sending to the generative AI model's API.

[0233] Step 4:

[0234] A generative AI model analyzes text data to determine whether it contains dangerous content.

[0235] Input: Text data fed into the generative AI model

[0236] Output: Risk assessment result

[0237] How it works: The generative AI model uses natural language processing techniques to analyze text data and determine whether it contains dangerous content, such as phishing scams or inappropriate content.

[0238] Step 5:

[0239] The server receives the judgment results of the generated AI model and, if dangerous content is detected, notifies the user of a warning.

[0240] Input: Risk assessment result

[0241] Output: A warning notification sent to the user

[0242] Specific operation: The server analyzes the judgment results from the generative AI model and, if they contain dangerous content, sends a warning to the user via email, short message service, or push notification.

[0243] Step 6:

[0244] The user receives a warning notification and confirms the appropriate action to be taken.

[0245] Input: The warning notification sent to the user

[0246] Output: User's response behavior

[0247] Specific actions: The user checks the warning notification they received and takes appropriate action according to the instructions in the notification. For example, if it is a phishing email, they should not click on the link or change their password.

[0248] Example 3

[0249] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0250] The use of modern information and communication networks, especially among children, poses a growing risk of sending and receiving dangerous text messages. Because these dangerous text messages can lead to bullying, threats, and other harmful behavior, parents and administrators must monitor them in real time and take appropriate action. However, current systems lack the means to effectively detect and promptly notify users of dangerous text messages. Furthermore, their ability to improve the system based on feedback is limited. This makes it difficult to prevent dangerous situations before they occur.

[0251] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0252] In this invention, the server includes means for monitoring a child's use of the information and communications network, means for using a generative AI model to detect dangerous text, means for monitoring interactions on the information and communications network using an application programming interface of the information and communications network platform, means for notifying a parent or administrator in real time when a danger is detected, means for improving the system based on the feedback, means for receiving text messages sent by the user from the terminal, means for analyzing the text messages using the generative AI model, means for detecting dangerous text based on the analysis results, and means for updating the generative AI model based on feedback from the parent or administrator. This makes it possible to detect dangerous text messages in real time and quickly notify parents or administrators. Furthermore, by continuously improving the system based on feedback, dangerous text can be detected with greater accuracy.

[0253] "Means for monitoring children's use of information and communications networks" refers to technologies and devices for monitoring activities on information and communications networks used by children and detecting abnormal or dangerous behavior.

[0254] "Means using generative AI models to detect dangerous text" refers to technologies or devices that use generative AI models to analyze the content of text messages and assess their risk.

[0255] "Means for monitoring communications on an information and communications network using the application programming interface of an information and communications network platform" refers to technology or devices for monitoring messages and data communications on the network using the API provided by the information and communications network platform.

[0256] "Means for notifying parents or administrators in real time when danger is detected" refers to technology or devices for sending immediate notifications to parents or administrators when dangerous text is detected.

[0257] "Means for improving the system based on feedback" refers to technologies or devices that improve the performance or accuracy of the system based on feedback from parents or administrators or automatically collected data.

[0258] The "means for receiving a text message sent from a terminal by a user" refers to a technique or device that allows a server to receive a text message sent from a terminal used by a user.

[0259] "Means for analyzing text messages using a generative AI model" refers to technology or devices that use a generative AI model to analyze the content of received text messages and understand their meaning and intent.

[0260] "Means for detecting dangerous text based on analysis results" refers to technologies or devices for determining whether a text message is dangerous based on the analysis results of a generative AI model.

[0261] "Means for updating the generative AI model based on feedback from parents or administrators" refers to technologies or devices that update the learning data of the generative AI model based on feedback information provided by parents or administrators, thereby improving the accuracy of the model.

[0262] MODE FOR CARRYING OUT THE INVENTION

[0263] The present invention is a system for monitoring children's use of information and communication networks and detecting dangerous text messages. A specific embodiment of this system will be described below.

[0264] System configuration

[0265] This system consists of three main components: a server, a device, and a user. The server analyzes text messages using a generative AI model and detects dangerous text. The device is the device used by the user to send text messages, such as a smartphone or PC. The user is the person who actually sends the text messages.

[0266] Hardware and software used

[0267] The server is a computer system equipped with a high-performance processor and a large amount of memory. Specifically, a server equipped with an Intel Xeon processor or an AMD EPYC processor is suitable. The software running on the server includes the Python libraries spaCy and Transformers. These libraries are used to analyze text messages using natural language processing techniques.

[0268] A terminal is a device that allows a user to input and send text messages. This can be a smartphone or a computer, and these devices must be connected to the Internet.

[0269] System Operation

[0270] When a user sends a text message from their device, the device sends the message to a server. The server inputs the received text message into a generative AI model for analysis. The generative AI model evaluates the content of the text based on pre-trained data and determines whether it is dangerous.

[0271] For example, if a user sends a message saying, "I'm waiting for you at school. You can't escape," the server receives this message and inputs it into the generative AI model, which analyzes it and determines that it contains violent content.

[0272] Notifications and Feedback

[0273] If dangerous text is detected, the server sends a real-time notification to parents or administrators via email, short message service (SMS), or push notification, including information such as the content of the dangerous text, the time it occurred, and its location.

[0274] Parents or administrators can provide feedback based on the received notification, for example, "This text is not dangerous," and the server will use this feedback to update the generative AI model and reflect it in future decisions.

[0275] Specific examples

[0276] As a concrete example, the following prompt sentence is input to the generative AI model:

[0277] Example prompt sentence:

[0278] "It analyzes text messages sent by users to determine if they contain dangerous content, such as messages like: 'I'm waiting for you at school. You can't escape.'"

[0279] In this way, the system can detect dangerous text messages in real time and quickly notify parents or administrators. Furthermore, by continuously improving the system based on feedback, it is possible to detect dangerous text with higher accuracy. The flow of the identification process in Example 3 will be described with reference to FIG. 15.

[0280] Step 1:

[0281] A user sends a text message.

[0282] A user uses a device such as a smartphone or PC to input a text message and press the send button. The input is the text message entered by the user. The output is a command to send the message to the device.

[0283] Step 2:

[0284] The device sends a text message to the server.

[0285] The terminal sends the text message entered by the user to a server over the Internet, encrypting the data using the HTTPS protocol. The input is the text message entered by the user. The output is the text message sent to the server.

[0286] Step 3:

[0287] The server receives the text message.

[0288] The server receives text messages sent from the device. The received data is temporarily stored in a database. The input is the text message sent from the device. The output is the text message stored in the database.

[0289] Step 4:

[0290] The server analyzes the text message using the generative AI model.

[0291] The server uses the Python libraries spaCy and Transformers to analyze text messages. The generative AI model evaluates the content of the text based on pre-trained data. The input is the text message stored in the database. The output is the analysis result.

[0292] Step 5:

[0293] The server detects dangerous text.

[0294] The server determines whether the text is dangerous based on the analysis results of the generative AI model. For example, if the text contains "violent language" or "threatening content," it is deemed dangerous. The input is the analysis results of the generative AI model. The output is the detection result of dangerous text.

[0295] Step 6:

[0296] The server sends a notification to the parent or administrator.

[0297] The server sends real-time notifications to parents and administrators when dangerous text is detected. Notifications can be sent via email, SMS, or push notification. The notification includes information such as the content of the dangerous text, the time it occurred, and its location. The input is the detection result of the dangerous text. The output is the notification sent to the parent or administrator.

[0298] Step 7:

[0299] Parents and administrators provide feedback.

[0300] The parent / administrator provides feedback based on the received notification, for example, "This text is not dangerous" and sends the feedback to the server. The input is the feedback from the parent / administrator. The output is the feedback sent to the server.

[0301] Step 8:

[0302] The server updates the generative AI model based on the feedback.

[0303] The server updates the generative AI model based on feedback from parents and administrators. Specifically, it uses the feedback as training data to improve the accuracy of the model. The input is the feedback from parents and administrators. The output is the updated generative AI model.

[0304] (Application example 3)

[0305] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0306] Today's children frequently use online platforms and social media, increasing their risk of coming into contact with dangerous text or inappropriate content. However, it is difficult for parents and administrators to monitor these dangers in real time and respond appropriately. Furthermore, existing systems have low accuracy in detecting dangerous text, resulting in false positives and missed results. Furthermore, there is a lack of a mechanism for utilizing feedback to improve the system's accuracy, making continuous improvement difficult.

[0307] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0308] In this invention, the server includes means for monitoring a child's online activity, means for using a generative AI model to detect dangerous texts, means for monitoring communications using an online platform's API, means for notifying parents or administrators in real time when a risk is detected, means for improving the system based on feedback, means for sending the notification by email, SMS, or push notification, and means for collecting and learning from feedback to improve the accuracy of the AI ​​model. This makes it possible to detect dangerous texts in real time and quickly notify parents or administrators. The accuracy of the system can also be continuously improved by utilizing feedback.

[0309] "Children's online activities" refers to all of a child's actions and communications that occur using the Internet.

[0310] A "generative AI model" is an artificial intelligence model that has been trained using machine learning techniques and is generated to perform a specific task.

[0311] An "online platform" is a general term for services and applications provided over the Internet, and is a place where users can share information and communicate.

[0312] "API" stands for Application Programming Interface, an interface for exchanging data and functions between different software systems.

[0313] "Communication" refers to the act or process of sending or receiving data or information.

[0314] "Real-time notification" is a function that sends a notification immediately the moment a specific event occurs.

[0315] "Feedback" refers to the act of providing evaluations and opinions on the performance and results of a system, and refers to information that is useful for improving the system.

[0316] "Email" is a form of digital message sent and received over the Internet.

[0317] "SMS" stands for Short Message Service, a service that allows you to send and receive short text messages over a mobile phone network.

[0318] "Push notifications" are notifications sent directly to a user's device from a mobile application or web service.

[0319] "Accuracy" refers to the ability of a system or model to perform a task accurately.

[0320] "Learning" is the process by which a machine learning model finds patterns and rules based on data and improves its performance.

[0321] A system for implementing this invention uses a generative AI model to monitor children's online activity and detect dangerous texts. The system uses the API of an online platform to monitor communications, notifies parents or administrators in real time when a risk is detected, and has the ability to improve the system based on feedback.

[0322] Hardware and software used

[0323] Hardware: Smartphones, servers

[0324] Software: Python, smtplib (sending email), twilio (sending SMS), requests (sending push notifications)

[0325] Data processing and calculation

[0326] 1. Loading the generative AI model: The server loads a generative AI model for detecting dangerous text. This model analyzes text using natural language processing techniques.

[0327] 2. Text evaluation: The server evaluates the text data obtained through the online platform's API using a generative AI model to determine whether the text is dangerous.

[0328] 3. Sending notifications: If dangerous text is detected, the server will send real-time notifications to parents or administrators via email, SMS, or push notification.

[0329] 4. Feedback collection and learning: Feedback is collected from parents and administrators, and the server uses this information to update the generative AI model, thereby improving the accuracy of the system.

[0330] Specific examples

[0331] For example, if a child receives a dangerous message on their smartphone, the content of the message will be notified to the parent in real time. If the parent gives feedback that the text is not dangerous, the server will learn from that information and will not judge similar texts as dangerous in the future.

[0332] Prompt Sentence Examples

[0333] Develop a system to notify parents and administrators of dangerous texts in real time via email, SMS, or push notification, and implement a feedback mechanism to improve the accuracy of the AI ​​model.

[0334] The above is an embodiment of the present invention.

[0335] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0336] Step 1:

[0337] The server loads the generative AI model.

[0338] Input: File path of the generated AI model

[0339] Data processing: Read the model file and expand it in memory

[0340] Output: The loaded generative AI model

[0341] Specific operation: The server loads the generative AI model from the specified file path and extracts it in memory. This model is used to analyze text using natural language processing techniques.

[0342] Step 2:

[0343] The server obtains the text data through the API of the online platform.

[0344] Input: API endpoint, authentication information

[0345] Data processing: Send an API request and retrieve text data

[0346] Output: The retrieved text data

[0347] What it does: The server sends a request to the online platform's API endpoint using the authentication information to retrieve text data about the child's online activities.

[0348] Step 3:

[0349] The server evaluates the acquired text data using a generative AI model.

[0350] Input: Obtained text data, generative AI model

[0351] Data processing: Inputting text data into a generative AI model to assess risk

[0352] Output: Risk assessment results

[0353] Specific operation: The server inputs the acquired text data into the generative AI model and evaluates whether the text is dangerous. The evaluation result is output as a flag indicating whether the text is dangerous.

[0354] Step 4:

[0355] The server sends real-time notifications to parents and administrators if dangerous text is detected.

[0356] Input: Risk assessment results, parent / caregiver contact information

[0357] Data processing: Generate notification messages based on the evaluation results and send them to designated contacts

[0358] Output: Notification message sent

[0359] What it does: When dangerous text is detected, the server generates a notification message with the parent or administrator's contact information and sends it via email, SMS, or push notification.

[0360] Step 5:

[0361] The server collects feedback from parents and administrators and updates the generative AI model.

[0362] Input: Feedback data, generative AI model

[0363] Data processing: Analyzing feedback data and retraining generative AI models

[0364] Output: An updated generative AI model

[0365] How it works: The server collects feedback data from parents and administrators and uses that data to retrain the generative AI model, improving the model's accuracy and resulting in better evaluation results in future evaluations.

[0366] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0367] "Example 1"

[0368] One embodiment of the present invention incorporates an emotion engine. This emotion engine infers a user's emotional state from their text. Specifically, it analyzes text posted by the user on the social networking site and determines whether the text expresses an emotion, such as joy, anger, sadness, surprise, or fear. This emotion engine is implemented by combining natural language processing and machine learning technologies.

[0369] "Example 2"

[0370] Furthermore, the emotion engine assesses the risk level based on the estimated emotional state. For example, if the user's text expresses strong feelings of anger or fear, it is determined that the text is likely to be dangerous. This risk assessment is performed by an AI model, and the results are notified to parents or administrators.

[0371] "Example 3"

[0372] The emotion engine also controls notifications based on the evaluation results. Specifically, notifications are sent to parents or administrators only when the risk level exceeds a certain threshold. This prevents parents or administrators from being confused by unnecessary notifications.

[0373] "Example 4"

[0374] Additionally, the emotion engine tracks changes in the user's emotional state and uses that information to improve the system. Specifically, it analyzes how the user's emotional state changes over time and uses the results to train the AI ​​model. This allows the system to more accurately understand the user's emotional state and deliver more appropriate notifications.

[0375] The processing flow of each embodiment will be described below.

[0376] "Example 1"

[0377] Step 1: A user posts text on a social networking site.

[0378] Step 2: The system takes that text and sends it to the emotion engine.

[0379] Step 3: The emotion engine analyzes the text and infers the user's emotional state.

[0380] "Example 2"

[0381] Step 1: Obtain the emotional state estimated by the emotion engine.

[0382] Step 2: The emotion engine assesses the risk level based on the emotional state.

[0383] Step 3: Submit the evaluation results to the system.

[0384] "Example 3"

[0385] Step 1: The system obtains the evaluation results from the emotion engine.

[0386] Step 2: If the evaluation results exceed a certain threshold, the system sends a notification to parents and administrators.

[0387] "Example 4"

[0388] Step 1: The system tracks changes in the user's emotional state.

[0389] Step 2: The system sends the emotional state change information to the emotion engine.

[0390] Step 3: The emotion engine uses the information to update its model and improve the system.

[0391] Example 1

[0392] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0393] Today's children frequently use information exchange services, which can lead to inappropriate or dangerous interactions. However, it is difficult for parents and administrators to monitor these interactions in real time and take appropriate action. It is also difficult to understand children's emotional states, so early detection and response to problems is required.

[0394] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0395] In this invention, the server includes means for monitoring a child's use of the information exchange service, means for using an artificial intelligence model to estimate an emotional state, means for monitoring information exchange transactions using an application programming interface of the information exchange service, means for notifying a parent or administrator of the results of the estimated emotional state in real time, and means for improving the system based on the feedback. This makes it possible to effectively monitor a child's use of the information exchange service and understand their emotional state, thereby detecting problems early and taking appropriate measures.

[0396] "Means for monitoring children's use of information exchange services" refers to a function for monitoring interactions with information exchange services used by children and collecting appropriate data.

[0397] The "artificial intelligence model for estimating emotional states" is an algorithm that combines natural language processing technology and machine learning technology to estimate emotional states from text data.

[0398] "Application programming interface of an information exchange service" means an interface for accessing data and functions within a service using the API provided by the information exchange service.

[0399] "Means for notifying parents or administrators of the results of estimated emotional state in real time" is a notification function for immediately conveying the results of the analysis of emotional state to parents or administrators.

[0400] "Means for improving the system based on feedback" refers to methods for improving the performance and functionality of a system based on feedback from users and operational data of the system.

[0401] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0402] "Email, SMS, or Push Notification" refers to a communication method for conveying information to a user. Email is a message sent over the Internet, SMS is a short text message sent over a mobile phone network, and Push Notification is a notification sent directly from an application to a user's device.

[0403] MODE FOR CARRYING OUT THE INVENTION

[0404] This invention is a system that monitors children's use of information exchange services and estimates their emotional state. The system operates in cooperation with a server, a terminal, and a user.

[0405] Hardware and software used

[0406] The server is a server machine with a high-performance CPU and sufficient memory. The software uses an information exchange service application programming interface (API), a natural language processing library (e.g., NLTK or spaCy), and a machine learning framework (e.g., TENSORFLOW (registered trademark) or PyTorch).

[0407] Program processing

[0408] The server uses the API of the information exchange service to obtain text posted and messages received by the child. Specifically, the server periodically calls the API to collect the latest posts and messages. The server then passes the collected text data to the emotion engine. The emotion engine combines natural language processing (NLP) and machine learning technologies to infer emotional states from the text. For example, the emotion engine analyzes the text and determines whether the text expresses an emotion, such as joy, anger, sadness, surprise, or fear.

[0409] Specific examples

[0410] For example, if a child posts "Today was so much fun!" to an information exchange service, the server retrieves this post and passes it to the emotion engine. The emotion engine analyzes this text and determines that it expresses the emotion of "joy." The server stores the analysis results in a database and notifies parents or administrators in real time. The notification may be, for example, a push notification with a message such as "Your child posted 'Today was so much fun!'. Their emotional state is 'joy.'"

[0411] Prompt Sentence Examples

[0412] Examples of prompts to input to a generative AI model might include:

[0413] "Analyze text posted by a child on an information exchange service and infer their emotional state. For example, if a child posts, 'Today was so much fun!', determine which emotion they are expressing."

[0414] In this way, the server monitors the child's use of the information exchange service and realizes a system that uses an emotion engine to infer the child's emotional state from the text.

[0415] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0416] Program processing flow

[0417] Step 1:

[0418] The server calls the API of the information exchange service.

[0419] Input: API endpoint URL, authentication token, required parameters

[0420] What happens: The server sends a GET request to a URL like "https: / / api.exchangeplatform.com / v1 / messages?user_id=12345" containing the authentication token "Bearer abcdef123456".

[0421] Output: Response from the API (JSON format data)

[0422] Step 2:

[0423] The server retrieves the child's posts and messages.

[0424] Input: Response from the API (JSON format data)

[0425] How it works: The server receives the response from the API and extracts the text posted and messages received by the child. For example, it receives the following JSON data:

[0426] json

[0427] {

[0428] "messages": [

[0429] {"text": "Today was so much fun!", "timestamp": "2023-10-01T12:34:56Z"},

[0430] {"text": "I have too much homework and I'm tired...", "timestamp": "2023-10-01T14:56:78Z"}

[0431] ]

[0432] }

[0433] The server extracts the text and timestamp from this data.

[0434] Output: Extracted text data and timestamps

[0435] Step 3:

[0436] The server passes the acquired text data to the emotion engine.

[0437] Input: Extracted text data

[0438] How it works: The server passes the acquired text data, "Today was so much fun!", to the emotion engine, which uses a natural language processing library to analyze this text.

[0439] Output: Text data passed to the emotion engine

[0440] Step 4:

[0441] The emotion engine analyzes the text data and estimates the emotional state.

[0442] Input: Text data passed to the emotion engine

[0443] How it works: The emotion engine analyzes the text "I had so much fun today!" and infers the emotion "joy." The analysis uses a pre-trained model.

[0444] Output: Estimated emotional state (e.g., "joy")

[0445] Step 5:

[0446] The server stores the analysis results of the emotion engine.

[0447] Input: Estimated emotional state, text data, timestamp

[0448] How it works: The server stores the analysis results returned by the emotion engine in a database. The stored data includes the text "I had so much fun today!", the emotional state "Joy", and the timestamp "2023-10-01T12:34:56Z".

[0449] Output: Analysis results stored in a database

[0450] Step 6:

[0451] The server notifies the terminal of the analysis results.

[0452] Input: Analysis results stored in the database

[0453] Operation: The server notifies the device of the analysis result, "Joy." For example, the notification may be a push notification with a message such as, "Your child posted, 'Today was so much fun!' Their emotional state is 'Joy.'"

[0454] Output: Notification sent to the device

[0455] In this way, the server monitors the child's use of the information exchange service, estimates the child's emotional state from the text using an emotion engine, and notifies parents or administrators in real time.

[0456] (Application example 1)

[0457] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0458] Today's children frequently use social networking sites (SNS), increasing the likelihood that they will encounter dangerous situations such as bullying, suicide, and drug use. However, it is difficult for parents and administrators to monitor their children's SNS usage in real time and take appropriate action. Conventional monitoring systems are inadequate in estimating emotional states or detecting specific keywords, and may miss dangerous situations that children are in. Therefore, there is a need for a system that can effectively monitor children's SNS usage, detect dangerous situations early, and notify parents or administrators.

[0459] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for monitoring a child's use of SNS, means for using an AI model to detect dangerous text, means for monitoring SNS interactions using the SNS platform's API, means for notifying parents or administrators in real time when danger is detected, means for improving the system based on feedback, means for estimating an emotional state from text using an emotion engine, means for detecting posts or messages containing specific keywords, and means for sending a notification based on the detected emotional state or keyword. This makes it possible to quickly and accurately notify parents or administrators when a child encounters a dangerous situation on SNS.

[0460] "Methods for monitoring children's use of social media" refers to technologies for tracking activity on social media accounts used by children and obtaining the content of posts and messages.

[0461] "Means for using AI models to detect dangerous text" refers to technology that uses machine learning algorithms to automatically identify dangerous content from text on social media.

[0462] "Methods for monitoring social media interactions using the API of a social media platform" refers to technology that utilizes the application programming interface (API) provided by a social media platform to obtain and monitor user posts and messages.

[0463] "Means for notifying parents or administrators in real time when danger is detected" refers to technology that immediately sends a warning to parents or administrators when a dangerous situation is detected.

[0464] "Means for improving the system based on feedback" refers to techniques for improving the performance and accuracy of a system based on feedback from users and operational data from the system.

[0465] The "means for inferring an emotional state from text using an emotion engine" is a technology that combines natural language processing technology and machine learning technology to infer a user's emotional state from text.

[0466] "Means for detecting posts and messages containing specific keywords" refers to technology that automatically identifies posts and messages containing pre-set dangerous keywords.

[0467] "Means for sending notifications based on detected emotional states or keywords" refers to technology that sends notifications to parents or administrators when a dangerous situation is identified by the emotion engine or keyword detection function.

[0468] The following system configuration will be described as an embodiment of the present invention.

[0469] System Configuration

[0470] This system monitors children's use of social media, detects dangerous situations, and notifies parents or administrators. The system consists of the following main components:

[0471] 1. How to monitor your child's social media use

[0472] The server uses the API of the social media platform to retrieve the child's posts and messages, allowing for real-time monitoring of the child's social media usage.

[0473] 2. Using AI models to detect dangerous text

[0474] The server automatically identifies dangerous content from captured text using an AI model with machine learning algorithms, which analyzes the text using natural language processing techniques.

[0475] 3. A method for inferring emotional states from text using an emotion engine

[0476] The server uses an emotion engine to infer the user's emotional state from the text. The emotion engine is realized by combining natural language processing and machine learning technologies.

[0477] 4. How to find posts or messages containing specific keywords

[0478] The server automatically identifies posts and messages containing pre-defined dangerous keywords, a list of which can be configured by parents or administrators.

[0479] 5. Real-time notification to parents and administrators when danger is detected

[0480] The server instantly sends email, SMS or push notifications to parents or administrators when a dangerous situation is detected.

[0481] 6. How to improve the system based on feedback

[0482] The server makes improvements to improve the performance of the AI ​​model and emotion engine based on user feedback and system operation data.

[0483] Hardware and software used

[0484] Hardware: Smartphones, servers

[0485] Software: Python, SNS platform API, TextBlob (natural language processing library), scikit-learn (machine learning library)

[0486] Data processing and calculation

[0487] 1. Data acquisition: The server acquires the child's posts and messages via the social media platform's API.

[0488] 2. Sentiment Analysis: The server uses TextBlob to perform sentiment analysis on the text and estimate the emotional state.

[0489] 3. Risk detection: The server uses machine learning models to analyze the text and detect risky content and specific keywords.

[0490] 4. Sending notifications: The server sends notifications to parents or administrators based on the detected emotional state or keywords.

[0491] Specific examples

[0492] If a child posts that they are being bullied at school, the emotion engine will detect "sadness" and send a notification to the parents.

[0493] If a child sends a message saying "I want to commit suicide," the set keyword "suicide" will be detected and a warning will be sent to the parents.

[0494] Prompt Sentence Examples

[0495] Create a program that analyzes text posted by children on social media and determines whether the text expresses any emotion, such as joy, anger, sadness, surprise, fear, etc. Additionally, add a function that sends a notification to parents if posts or messages containing specific keywords (e.g., bullying, suicide, drugs, etc.) are detected.

[0496] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0497] Step 1:

[0498] The server uses the social media platform's API to retrieve the child's posts and messages. The input requires the social media platform's API endpoint and the child's account information. The output is the retrieved post and message data, which is often provided in JSON format.

[0499] Step 2:

[0500] The server inputs the text data of the posts and messages it retrieves into the emotion engine. The emotion engine performs sentiment analysis on the text using TextBlob. The input requires text data. The output is the emotional state expressed by the text (e.g., joy, anger, sadness, surprise, fear).

[0501] Step 3:

[0502] The server uses a machine learning model to detect dangerous text based on the output of the emotion engine. The inputs are the emotional state and the text data. The output is a decision on whether the text contains dangerous content. This decision is often expressed as a risk score.

[0503] Step 4:

[0504] The server detects posts and messages containing specific keywords. As input, it requires text data and a predefined list of keywords. As output, it determines whether the keyword was detected or not. This determination result is provided as keyword match information.

[0505] Step 5:

[0506] The server sends real-time notifications to parents and administrators when dangerous text or specific keywords are detected. The inputs required are risk scores and keyword matches. The output is a generated notification message, which can be sent via email, SMS, or push notification.

[0507] Step 6:

[0508] The server makes improvements to improve the performance of the AI ​​model and emotion engine based on user feedback and system operation data. Feedback data and operation data are required as input. Improved AI models and emotion engines are obtained as output. This improvement process is carried out periodically.

[0509] Example 2

[0510] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0511] Today's children frequently use online communication, increasing the risk of coming into contact with dangerous texts and harmful content. However, it is difficult for parents and administrators to monitor these interactions in real time and respond appropriately. Furthermore, conventional systems lack the ability to analyze the emotional state of text in detail and evaluate the level of risk, so they may miss dangerous situations.

[0512] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for monitoring children's online communication usage, a means for using a generative AI model to detect dangerous text, a means for monitoring communication exchanges using an interface of an online platform, a means for notifying parents or administrators in real time when danger is detected, a means for improving the system based on feedback, and a means for estimating the emotional state of the text using an emotion engine and assessing the level of danger. This enables parents or administrators to respond quickly when children encounter dangerous situations online.

[0513] "Means for monitoring children's online communication use" refers to technologies and devices that monitor and record children's online activities, such as sending and receiving messages and postings, in real time.

[0514] "Means using generative AI models to detect dangerous text" refers to technologies and devices that utilize generative AI models that use natural language processing technology to analyze and determine whether input text contains dangerous content.

[0515] "Means of monitoring communication exchanges using the interfaces of online platforms" refers to technologies and devices that use APIs and interfaces provided by online platforms such as social media platforms and messaging apps to monitor messages and posts between users.

[0516] "Means for notifying parents or administrators in real time when danger is detected" refers to technology or devices for sending immediate notifications to parents or administrators when dangerous text is detected.

[0517] "Means for improving the system based on feedback" refers to technologies and devices for improving the performance and accuracy of the system based on the system's usage status and detection results.

[0518] "Means for estimating the emotional state of text using an emotion engine and assessing its risk level" refers to technology or devices for analyzing the emotions contained in text and assessing the risk level of the text based on that emotional state.

[0519] The present invention is a system for monitoring children's online communication and detecting risky texts. The system uses a generative AI model and an emotion engine to analyze the content and emotional state of the text and assess its risk level.

[0520] Hardware and software used

[0521] Device: A device (e.g., smartphone, tablet, or computer) through which a user enters text.

[0522] Server: A computer system for receiving text and analyzing it using a generative AI model and emotion engine.

[0523] Generative AI models: Models that use natural language processing techniques (e.g., ChatGPT).

[0524] Emotion engine: Software for estimating the emotional state of text and assessing its riskiness.

[0525] Program processing

[0526] 1. Getting the text

[0527] A user inputs text through a terminal.

[0528] The terminal sends the entered text to the server.

[0529] 2. Text Analysis

[0530] The server inputs the received text into a generative AI model.

[0531] The generative AI model uses a natural language generation model such as ChatGPT.

[0532] The model analyzes the content of the text and determines whether the text contains dangerous content.

[0533] 3. Evaluation by Emotion Engine

[0534] The server passes the analysis results to the emotion engine.

[0535] The emotion engine estimates the emotional state contained in the text.

[0536] For example, if the text expresses strong feelings of anger or fear, the emotional state is estimated.

[0537] The emotion engine reassess the risk level based on the estimated emotional state.

[0538] 4. Risk Assessment and Notification

[0539] The server determines the final risk level based on the evaluation results obtained from the emotion engine.

[0540] If the risk is deemed high, the server will send a notification to parents or administrators.

[0541] The notification includes the text content and the risk assessment result.

[0542] Specific examples

[0543] For example, if a user enters text such as "I can't take it anymore, I want to hurt someone," this text is sent from the device to the server. The server inputs this text into ChatGPT and analyzes the content of the text. Based on the analysis results, the emotion engine infers that the text expresses strong anger and evaluates it as high risk. This evaluation result is notified to parents or administrators.

[0544] Prompt Sentence Examples

[0545] Examples of prompts to input to a generative AI model might include:

[0546] "A user entered the following text. Please determine if this text contains dangerous content: 'I can't take it anymore, I want to hurt someone.'"

[0547] In this way, the system can analyze the user's text, assess the risk level, and prompt appropriate responses.

[0548] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0549] Step 1: User Enters Text

[0550] Specific operation: The user enters text into an input field on the terminal.

[0551] Input: Text typed by the user (e.g., "I can't take it anymore, I want to hurt someone").

[0552] Output: The entered text is saved to the terminal.

[0553] Step 2: The device sends the text to the server

[0554] Specific operation: The terminal sends the entered text to the server as an HTTP request.

[0555] Input: The text entered by the user.

[0556] Output: The text is sent to the server in JSON format.

[0557] Step 3: The server inputs the text into the generative AI model

[0558] Specific operation: The server generates a prompt sentence to input the received text into the generative AI model.

[0559] Input: Text sent from the terminal.

[0560] Output: Prompt sentence to be input to the generative AI model (e.g., "The user entered the following text. Please determine whether this text contains dangerous content: 'I can't take it anymore, I want to hurt someone.'").

[0561] Step 4: The generative AI model analyzes the text

[0562] Specific operation: The generative AI model receives a prompt sentence and analyzes the content of the text.

[0563] Input: The prompt sentence fed into the generative AI model.

[0564] Output: Analysis result of the text (e.g., whether the text contains dangerous content).

[0565] Step 5: The server passes the analysis results to the emotion engine

[0566] Specific operation: The server passes the analysis results obtained from the generative AI model to the emotion engine.

[0567] Input: Analysis results from a generative AI model.

[0568] Output: The analysis results that are passed to the emotion engine.

[0569] Step 6: The emotion engine estimates the emotional state

[0570] Specific behavior: The emotion engine performs detailed analysis and estimates the emotional state of the text.

[0571] Input: Analysis results passed from the server.

[0572] Output: Estimated emotional state (e.g., the text expresses strong anger).

[0573] Step 7: The server evaluates the risk and notifies you of the result

[0574] Specific operation: The server determines the final risk level based on the evaluation results obtained from the emotion engine. If the risk level is determined to be high, a notification is sent to parents or administrators.

[0575] Input: Rating results from the sentiment engine.

[0576] Output: Notification sent to parent or administrator (e.g. email, SMS, or push notification).

[0577] In this way, the system can analyze the user's text, assess the risk level, and prompt appropriate responses.

[0578] (Application example 2)

[0579] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0580] Modern communication services, especially those used by children, are prone to dangerous text messages and inappropriate content, which can pose a threat to children's safety. It is also difficult for parents and administrators to detect these dangers in real time and take appropriate action. Furthermore, risk assessments do not take into account emotional states, so dangerous situations may be overlooked.

[0581] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0582] In this invention, the server includes means for monitoring a child's use of a communication service, means for using a generative AI model to detect dangerous text, means for monitoring communication exchanges using an API of a communication service platform, means for notifying a parent or administrator in real time when a danger is detected, means for improving the system based on the feedback, means for estimating the emotional state of the text using an emotion engine and assessing the level of danger, and means for sending a notification when the level of danger is high. This makes it possible to detect dangerous text messages in a child's use of a communication service in real time, assess the level of danger taking the emotional state into account, and promptly notify a parent or administrator.

[0583] "Means for monitoring children's use of communication services" refers to technology for monitoring the activities of communication services used by children in real time and detecting abnormal or dangerous behavior.

[0584] "Means using generative AI models to detect dangerous text" refers to technology that uses generative AI models to analyze text messages and determine whether their content is dangerous.

[0585] "Means for monitoring communication exchanges using APIs of communication service platforms" refers to technology that uses APIs provided by communication services to monitor message exchanges between users.

[0586] "Means for notifying parents or administrators in real time when danger is detected" refers to technology that immediately sends a notification to parents or administrators when dangerous text is detected.

[0587] "Means for improving the system based on feedback" refers to techniques for collecting feedback from users and improving the performance and accuracy of the system based on that feedback.

[0588] The "means for estimating the emotional state of text using an emotion engine and assessing the risk level" is a technology that uses an emotion engine to analyze the emotional state of text and assesses the risk level based on the results.

[0589] The "means for sending a notification when the risk is high" is a technology for sending a notification to a parent or administrator when the risk is determined to be high.

[0590] The following system configuration will be described as an embodiment of the present invention.

[0591] System Configuration

[0592] Hardware and Software

[0593] Hardware: Smartphone

[0594] Software: Python, OpenAI API, TextBlob

[0595] Program processing explanation

[0596] How to monitor your child's use of communication services

[0597] The server monitors the activities of the communication services used by the child in real time, including obtaining message exchanges between users using the communication service platform's API.

[0598] How to use generative AI models to detect dangerous text

[0599] The server inputs the acquired text message into a generative AI model to analyze whether its content is dangerous. Specifically, it uses OpenAI's generative AI model (e.g., GPT-3 (registered trademark)) to determine whether the text is dangerous.

[0600] A means of using an emotion engine to estimate the emotional state of text and assess its riskiness

[0601] The server uses TextBlob to estimate the emotional state of the text: if the text expresses strong anger or fear, the emotion engine will rate the text as highly dangerous.

[0602] A means to notify parents and administrators in real time when danger is detected

[0603] The server sends real-time notifications to parents and administrators when dangerous texts are detected via email, short message service, or push notification.

[0604] A means of improving the system based on feedback

[0605] The server collects user feedback and uses it to improve the system's performance and accuracy, allowing the system to be continuously improved.

[0606] Specific examples

[0607] For example, if a child sends a message saying "Come see me now, or else...", the server inputs this message into the generative AI model to assess its risk level. The generative AI model determines that this message is dangerous and uses TextBlob to estimate the emotional state. If the emotion engine determines that this message expresses anger, it is assessed as high risk. The server then sends a notification to the parent or administrator saying "Warning: Dangerous text detected: Come see me now, or else...".

[0608] Prompt Sentence Examples

[0609] "Decide if this text is dangerous: Come see me now, or..."

[0610] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0611] Step 1:

[0612] The server uses the API of the communication service platform to obtain message exchanges between users. The input is message data obtained from the communication service platform, and the output is a text message to be analyzed. This step includes the specific operations of making an API call and obtaining message data.

[0613] Step 2:

[0614] The server inputs the acquired text message into the generative AI model and analyzes whether its content is dangerous. The input is the text message acquired in step 1, and the output is the risk assessment result by the generative AI model. This step includes the specific operation of inputting a prompt sentence into the generative AI model (e.g., GPT-3) and obtaining the analysis result.

[0615] Step 3:

[0616] The server uses TextBlob to estimate the emotional state of the text. The input is the text message obtained in step 1, and the output is the evaluation result of the emotional state. This step includes the specific operation of performing sentiment analysis of the text using the TextBlob library.

[0617] Step 4:

[0618] The server evaluates the risk level of the text based on the analysis results of the generative AI model and the evaluation results of the emotional state. The input is the output of Step 2 and Step 3, and the output is the final risk assessment result. This step includes specific operations to integrate the analysis results and the emotional state and evaluate the risk level.

[0619] Step 5:

[0620] If the risk is assessed as high, the server sends a real-time notification to parents or administrators. The input is the risk assessment result from step 4, and the output is the notification message. This step includes specific actions to send the notification via email, short message service, or push notification.

[0621] Step 6:

[0622] The server collects user feedback and uses it to improve the system's performance and accuracy. The input is user feedback data, and the output is improved system settings and model parameters. This step includes collecting and analyzing feedback data and taking specific actions to improve the system.

[0623] Example 3

[0624] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0625] Today's children frequently use online communication platforms, which can contain dangerous texts or inappropriate content. Parents and administrators are required to monitor these dangerous texts in real time and take appropriate action. However, manual monitoring is labor-intensive and inefficient. It is also difficult to track changes in emotional state and provide timely notifications. A system to solve these challenges is needed.

[0626] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0627] In this invention, the server includes means for monitoring a child's online communication usage, means for using a machine learning model to detect dangerous text, means for monitoring interactions using an interface of an online communication platform, means for notifying parents or administrators in real time when a risk is detected, means for improving the system based on the feedback, means for evaluating a user's emotional state using an emotion engine and controlling notifications, and means for tracking changes in the user's emotional state and using the information to improve the system, thereby enabling real-time detection and notification of dangerous text, tracking changes in emotional state, and continuous improvement of the system.

[0628] "Means for monitoring children's online communication usage" refers to technologies for monitoring interactions on online communication platforms used by children and for collecting and analyzing text messages and other content in real time.

[0629] A "machine learning model for detecting dangerous text" is an algorithm that uses natural language processing techniques to analyze text messages and automatically detect text that contains violent, threatening, or other dangerous content.

[0630] "Means for monitoring interactions using the interface of an online communication platform" refers to technology that links with an online communication platform through an API or other interface to monitor user interactions.

[0631] "Means for real-time notification to parents or administrators when dangerous text is detected" refers to technology that sends immediate notifications to parents or administrators when dangerous text is detected, via email, short message service, or push notification.

[0632] "Means for improving the system based on feedback" refers to techniques for improving the performance and accuracy of the system using feedback from parents and administrators or automatically collected data.

[0633] The "means for evaluating a user's emotional state using an emotion engine and controlling notifications" is a technology for analyzing a user's text messages to evaluate their emotional state and controlling the sending of notifications based on the evaluation results.

[0634] "Means for tracking changes in the user's emotional state and using that information to improve the system" refers to technology that continuously tracks changes in the user's emotional state and uses that data to train the system and improve its accuracy and performance.

[0635] MODE FOR CARRYING OUT THE INVENTION

[0636] This invention is a system that monitors children's online communication, detects dangerous texts, and notifies parents or administrators in real time if necessary. Furthermore, the system can be improved based on feedback, and by assessing and tracking the user's emotional state, it can provide more appropriate notifications.

[0637] System configuration

[0638] The server has the following main functions:

[0639] 1. How to monitor your child's online communication

[0640] 2. Using machine learning models to detect dangerous text

[0641] 3. Using the interface of online communication platforms to monitor interactions

[0642] 4. Real-time notification to parents and administrators when danger is detected

[0643] 5. How to improve the system based on feedback

[0644] 6. A means to evaluate the user's emotional state using an emotion engine and control notifications

[0645] 7. A way to track changes in the user's emotional state and use that information to improve the system

[0646] Hardware and software used

[0647] The servers will be built using a cloud-based infrastructure, specifically using the following software and services:

[0648] Natural Language Processing (NLP) technology: Google® Cloud Natural Language API, IBM Watson® Natural Language Understanding

[0649] Notification services: email, short message service (SMS), push notifications

[0650] Data processing and calculation

[0651] The server receives text messages sent by users in real time and analyzes them using natural language processing technology. If dangerous text is detected based on the analysis results, a notification is sent to parents or administrators. The notification includes information such as the content of the dangerous text, the time it occurred, and the location where it occurred.

[0652] Additionally, the system receives feedback from parents and administrators and uses that feedback to update the AI ​​model, improving the system's accuracy.

[0653] The emotion engine analyzes users' text messages and evaluates their emotional state. Based on the evaluation results, notifications are sent only when the risk level exceeds a certain threshold, preventing parents and administrators from being overwhelmed by unnecessary notifications.

[0654] The emotion engine also tracks changes in the user's emotional state and uses that data to improve the system, allowing it to more accurately understand the user's emotional state and provide more appropriate notifications.

[0655] Specific examples

[0656] If a user sends a message such as "I'm being bullied at school," the server will detect this message as dangerous text and send a notification to the parent or administrator stating, "A user said they are being bullied at school. Time: 14:30, Location: School."

[0657] Example prompt sentence:

[0658] "Please create a program that analyzes text messages sent by users and notifies parents or administrators if dangerous content is found." The flow of the identification process in the third embodiment will be described with reference to FIG. 21.

[0659] Step 1:

[0660] A user sends a text message using a device (such as a smartphone or PC). The input is the text message entered by the user, and the output is that message being sent to the server. Specifically, the user enters the message "I'm being bullied at school" and presses the send button.

[0661] Step 2:

[0662] The server receives text messages sent by users. The input is the text message sent by the user, and the output is that message being prepared for analysis. In concrete terms, the server receives the message "I'm being bullied at school."

[0663] Step 3:

[0664] The server analyzes the received text message using natural language processing (NLP) technology. The input is the received text message, and the output is the analysis result. Specifically, the server analyzes the message "I'm being bullied at school" using Google Cloud Natural Language API and IBM Watson Natural Language Understanding.

[0665] Step 4:

[0666] If the server detects dangerous text based on the analysis results, it prepares to send a notification to parents or administrators. The input is the analysis result, and the output is the content of the notification. Specifically, the server determines that the message "I'm being bullied at school" is dangerous and prepares the content of the notification.

[0667] Step 5:

[0668] The server sends the prepared notification to the parent or administrator. The input is the notification content, and the output is the notification sent to the parent or administrator. Specifically, the server sends a notification to the parent or administrator with the content "A user said they are being bullied at school. Time: 14:30, Location: School."

[0669] Step 6:

[0670] Parents or administrators provide feedback on notifications from the system. The input is the notification from the system, and the output is the feedback. A specific action is for a parent to send feedback saying, "This text is not dangerous."

[0671] Step 7:

[0672] The server receives feedback from parents or administrators and analyzes its content. The input is the feedback, and the output is the analysis result. Specifically, the server receives feedback such as "This text is not dangerous" and analyzes it.

[0673] Step 8:

[0674] The server uses the text that received the feedback as training data to update the AI ​​model. The input is the analysis result, and the output is the updated AI model. Specifically, the server uses the text "I'm being bullied at school" as training data and updates the AI ​​model.

[0675] Step 9:

[0676] The emotion engine analyzes the user's text message and evaluates its emotional state. The input is the user's text message, and the output is the evaluation result of the emotional state. Specifically, the emotion engine analyzes the message "very sad" and evaluates the danger level.

[0677] Step 10:

[0678] The server receives the emotion engine's evaluation results and sends notifications as necessary. The input is the evaluation result of the emotional state, and the output is the notification sent to the parent or administrator. Specifically, the server receives the emotion engine's evaluation results and sends a notification only if it determines that the risk is high.

[0679] Step 11:

[0680] The emotion engine tracks changes in the user's emotional state and uses that data to improve the system. The input is data on changes in the user's emotional state, and the output is an improved system. Specifically, the emotion engine analyzes messages like "I'm happy today" and "I'm sad now" and records the changes in emotional state.

[0681] Step 12:

[0682] The server receives data from the emotion engine and updates the AI ​​model. The input is data on changes in emotional state, and the output is an updated AI model. Specifically, the server updates the AI ​​model based on the analysis results to more accurately understand the user's emotional state.

[0683] (Application example 3)

[0684] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0685] Today's children access a great deal of information and communicate through social networking sites. However, this also increases the risk of children coming into contact with dangerous texts and inappropriate content. It is difficult for parents and administrators to grasp these risks in real time and respond appropriately. Excessive notifications can also confuse parents and administrators. Furthermore, to improve the accuracy of the system, it is necessary to accurately understand the user's emotional state and incorporate their feedback. Effective methods to solve these issues are needed.

[0686] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0687] In this invention, the server includes: means for monitoring a child's use of social media; means for using an AI model to detect dangerous texts; means for monitoring social media interactions using the social media platform's API; means for notifying parents or administrators in real time when a risk is detected; means for improving the system based on feedback; means for tracking the user's emotional state using an emotion engine and using that information to improve the system; and means for sending a notification only when the risk level exceeds a certain threshold. This allows parents and administrators to grasp the risks of their children's use of social media in real time and respond appropriately. It also prevents excessive notifications and improves the accuracy of the system.

[0688] "Means for monitoring children's use of social media" refers to technology for tracking and recording social media activity used by children in real time.

[0689] "Means using AI models to detect dangerous text" refers to techniques that use natural language processing techniques to analyze text and utilize artificial intelligence models to automatically identify dangerous content.

[0690] "Methods for monitoring social media interactions using the API of a social media platform" refers to technology that monitors messages and posts on social media using the application programming interface (API) provided by the social media platform.

[0691] "Means for notifying parents or administrators in real time when danger is detected" refers to technology that sends immediate notifications to parents or administrators when dangerous text is detected.

[0692] "Means for improving the system based on feedback" refers to techniques for improving the accuracy and performance of the system using feedback from parents and administrators or data collected automatically by the system.

[0693] "Means of tracking the user's emotional state using an emotion engine and using that information to improve the system" is a technology that analyzes the user's emotional state and tracks changes in it to improve the accuracy of the system.

[0694] The "means for sending a notification only when the risk level exceeds a certain threshold" is a technology that sends a notification to parents or administrators only when the risk level of the detected dangerous text exceeds a preset threshold.

[0695] A system for implementing this invention uses an AI model to monitor children's use of social media and detect dangerous texts. The system monitors social media interactions using the social media platform's API and sends real-time notifications to parents and administrators when dangerous texts are detected. The system also improves the system based on feedback, using an emotion engine to track the user's emotional state and using that information to improve the system's accuracy. Furthermore, the system has the ability to send notifications only when the level of danger exceeds a certain threshold.

[0696] Hardware and software used

[0697] Hardware: Smartphones, servers

[0698] Software: Python, smtplib (sending emails), twilio (sending SMS), AI model (emotion engine)

[0699] Data processing and calculation

[0700] 1. Text sentiment analysis: The server uses an emotion engine to analyze text on SNS in real time and evaluate the risk level and emotional state.

[0701] 2. Sending notifications: If the risk exceeds a certain threshold, the server will send notifications to parents or administrators via email and SMS.

[0702] 3. Collecting feedback and improving the system: Feedback from parents and administrators will be collected and used to improve the accuracy of the AI ​​model.

[0703] Specific examples

[0704] For example, if a child sends the text "This is a dangerous message" in a chat app, the server analyzes the text using an emotion engine and sends a real-time notification to the parent if it determines that the message is highly dangerous.

[0705] Prompt Sentence Examples

[0706] Analyze the text below and assess the level of danger and emotional state.

[0707] Text: "This is a dangerous message"

[0708] In this way, the system allows parents and administrators to understand the risks of their children's use of social media in real time and take appropriate action, while also preventing excessive notifications and improving the accuracy of the system.

[0709] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0710] Step 1:

[0711] The server monitors the child's social media usage using the social media platform's API. As input, it receives message data obtained from the social media platform and generates monitored text data as output. This text data is used for sentiment analysis in the next step.

[0712] Step 2:

[0713] The server uses an emotion engine to analyze the text data of the monitored object. It receives the text data generated in step 1 as input and generates the risk and emotional state assessment results as output. Specifically, the emotion engine analyzes the content of the text and calculates the risk score and emotional state (e.g., anger, sadness, etc.).

[0714] Step 3:

[0715] The server determines whether the risk level exceeds a certain threshold. As input, it receives the risk level score generated in step 2, and as output, it generates a flag indicating whether a notification is required. Specifically, if the risk level score exceeds a preset threshold, it sets the notification flag.

[0716] Step 4:

[0717] The server sends real-time notifications to parents and administrators when a notification flag is set. It receives the notification flag, text data, risk score, and emotional state as input, and generates email and SMS notification messages as output. Specifically, it uses smtplib to send emails and twilio to send SMS.

[0718] Step 5:

[0719] The server collects feedback from parents and administrators and improves the system. It receives feedback data from parents and administrators as input and generates updated data for the AI ​​model as output. Specifically, it analyzes the feedback content and reflects it as learning data for the AI ​​model.

[0720] Step 6:

[0721] The server uses an emotion engine to track the user's emotional state and uses that information to improve the system. It receives the user's emotional state data as input and generates system improvement data as output. Specifically, it analyzes changes in the user's emotional state and uses the results as training data for the AI ​​model.

[0722] In this way, the system allows parents and administrators to understand the risks of their children's use of social media in real time and take appropriate action, while also preventing excessive notifications and improving the accuracy of the system.

[0723] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0724] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0725] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0726] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0727] [Second embodiment]

[0728] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0729] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0730] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0731] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0732] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0733] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0734] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0735] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0736] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0737] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0738] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0739] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0740] "Example 1"

[0741] One embodiment of the present invention is a system for monitoring children's use of social networking sites. This system monitors children's social networking site interactions via the API of the social networking site platform used by the children. Specifically, it obtains texts posted by the children and messages received by them.

[0742] "Example 2"

[0743] Furthermore, the system of the present invention inputs the acquired text into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing technology. Specifically, it uses a natural language generation model, such as generative AI, to understand the content of the text and determine whether the text contains dangerous content.

[0744] "Example 3"

[0745] Additionally, the system of the present invention notifies parents or administrators in real time when dangerous text is detected. This notification can be sent via email, SMS, or push notification. Specifically, a notification containing information about the dangerous text, including the time and location of the text, is sent to contacts registered by the parent or administrator.

[0746] "Example 4"

[0747] Furthermore, the system of the present invention improves the system based on feedback. Specifically, it uses feedback from parents or administrators, as well as data automatically collected by the system, to improve the accuracy of the AI ​​model. For example, if a parent or administrator provides feedback such as "this text is not dangerous," the AI ​​model is updated to determine that the text is not dangerous.

[0748] The processing flow of each embodiment will be described below.

[0749] "Example 1"

[0750] Step 1: Monitor the child's social media interactions via the API of the social media platform they use, specifically by capturing text messages posted and received by the child.

[0751] Step 2: The acquired text is fed into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing techniques.

[0752] Step 3: If the AI ​​model detects dangerous text, it notifies parents and administrators in real time via email, SMS, or push notification.

[0753] Step 4: Improve the accuracy of the AI ​​model using feedback from parents and administrators, as well as data collected automatically by the system.

[0754] "Example 2"

[0755] Step 1: Monitor the child's social media interactions via the API of the social media platform they use, specifically by capturing text messages posted and received by the child.

[0756] Step 2: The acquired text is fed into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing techniques.

[0757] Step 3: If the AI ​​model detects dangerous text, it notifies parents and administrators in real time via email, SMS, or push notification.

[0758] Step 4: Improve the accuracy of the AI ​​model using feedback from parents and administrators, as well as data collected automatically by the system.

[0759] "Example 3"

[0760] Step 1: Monitor the child's social media interactions via the API of the social media platform they use, specifically by capturing text messages posted and received by the child.

[0761] Step 2: The acquired text is fed into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing techniques.

[0762] Step 3: If the AI ​​model detects dangerous text, it notifies parents and administrators in real time via email, SMS, or push notification.

[0763] Step 4: Improve the accuracy of the AI ​​model using feedback from parents and administrators, as well as data collected automatically by the system.

[0764] "Example 4"

[0765] Step 1: Monitor the child's social media interactions via the API of the social media platform they use, specifically by capturing text messages posted and received by the child.

[0766] Step 2: The acquired text is fed into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing techniques.

[0767] Step 3: If the AI ​​model detects risky text, it notifies parents and administrators in real time via email, SMS, or push notification.

[0768] Step 4: Improve the accuracy of the AI ​​model using feedback from parents and administrators, as well as data collected automatically by the system.

[0769] Example 1

[0770] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0771] Today's children frequently use social media, increasing the risk of being exposed to inappropriate content and dangerous interactions. However, it is difficult for parents and administrators to monitor these interactions in real time and respond appropriately. Conventional monitoring systems have low accuracy in detecting dangerous content and often delay notifications, so a rapid response is required.

[0772] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for monitoring children's use of SNS, a means for using a generative AI model to detect dangerous text, a means for monitoring SNS interactions using the API of the SNS platform, a means for notifying parents or administrators in real time when a risk is detected, a means for improving the system based on feedback, a means for analyzing text using natural language processing technology, a means for analyzing acquired data and detecting inappropriate content, and a means for sending a notification to parents based on the analysis results. This makes it possible to effectively monitor children's use of SNS, quickly detect dangerous interactions, and notify parents.

[0773] "Means for monitoring children's use of social media" refers to technologies for monitoring activities on social media platforms used by children and obtaining the content of posts and messages.

[0774] A "generative AI model for detecting dangerous text" is an artificial intelligence model used to analyze text using natural language processing techniques to detect inappropriate content or dangerous interactions.

[0775] "Means for monitoring social media interactions using the API of a social media platform" refers to technology that utilizes the application programming interface (API) provided by a social media platform to obtain and monitor user posts and messages.

[0776] "Means for notifying parents or administrators in real time when danger is detected" refers to technology that immediately sends a notification to parents or administrators when dangerous content is detected.

[0777] "Means for improving the system based on feedback" refers to techniques for collecting system usage results and feedback from users, and using that information to improve the system's performance and accuracy.

[0778] "Means for analyzing text using natural language processing technology" refers to natural language processing (NLP) technology used to analyze text data and understand its content.

[0779] "Means for analyzing acquired data and detecting inappropriate content" refers to technology for analyzing data acquired from social media platforms and detecting inappropriate content contained therein.

[0780] The "means for sending a notification to the guardian based on the analysis result" is a technology for sending a notification to the guardian as necessary based on the results of the data analysis.

[0781] This invention is a system for monitoring children's use of social networking sites and detecting dangerous texts. This system includes elements of a server, a terminal, and a user, and operates as follows.

[0782] The server first obtains data related to the child's social media account using the social media platform's API. Specifically, the server authenticates using the API key and secret key provided by the social media platform and then obtains the child's posts and messages. For example, the server can use the Twitter API or Facebook Graph API.

[0783] The server then uses natural language processing (NLP) techniques to analyze the retrieved data. Specifically, it uses generative AI models to analyze the text and detect inappropriate content or dangerous interactions. NLP libraries used include NLTK and spaCy.

[0784] Based on the analysis results, the server sends real-time notifications to parents and administrators as needed via email, SMS, or push notifications, allowing parents and administrators to immediately understand their children's social media usage and take appropriate action.

[0785] Furthermore, the server improves the system based on feedback. Specifically, it collects system usage results and user feedback, and uses this information to improve the performance and accuracy of the generative AI model.

[0786] As a concrete example, consider a child using Twitter. The server retrieves the child's latest posts using the Twitter API and analyzes the text using a generative AI model. If inappropriate content is detected based on the analysis results, a notification is sent to the parent via email.

[0787] Examples of prompt sentences include the following:

[0788] "Please create a system to monitor children's social media usage. Use the Twitter API to retrieve children's posts and use a natural language processing library to analyze them for inappropriate content. Send notifications to parents based on the analysis results."

[0789] In this way, the server can effectively monitor children's use of social networking sites, quickly detect dangerous interactions, and notify parents.

[0790] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0791] Step 1: API Authentication

[0792] The server performs authentication using the API key and secret key of the SNS platform. The API key and secret key are required as input. This gives the server permission to obtain data from the SNS platform. Specifically, the server sends an authentication request using the API key and secret key and obtains an authentication token.

[0793] Step 2: Data Acquisition

[0794] After successful authentication, the server retrieves posts and messages related to the child's account. The input requires an authentication token and the child's account information. The output is the retrieved post and message data. Specifically, it sends a GET request using the social media platform's API to retrieve the child's latest posts and messages.

[0795] Step 3: Data analysis

[0796] The server uses natural language processing (NLP) techniques to analyze the retrieved data. The input requires retrieved post and message data. The output is the analysis results. Specifically, it uses a generative AI model to analyze the text and detect inappropriate content or dangerous interactions. NLP libraries used include NLTK and spaCy.

[0797] Step 4: Hazard detection

[0798] The server determines whether dangerous content has been detected based on the results of the data analysis. The input required is the results of the data analysis. The output is a flag indicating a danger detection. Specifically, the server evaluates the analysis results and checks whether they contain inappropriate words or phrases.

[0799] Step 5: Notification

[0800] The server sends real-time notifications to parents and administrators when a risk is detected. The inputs required are the risk detection flag and analysis results. The output is confirmation that a notification has been sent. Specific actions include sending a notification to parents and administrators via email, SMS, or push notification.

[0801] Step 6: Gather feedback

[0802] The server collects system usage results and user feedback. The inputs required are system usage results and user feedback. The output is the collected feedback data. Specifically, the server collects feedback through the user interface and stores it in a database.

[0803] Step 7: System Improvement

[0804] The server improves the system based on the collected feedback. The input requires the collected feedback data. The output is an improved system. Specifically, it makes adjustments to improve the performance and accuracy of the generative AI model, thereby improving the overall system.

[0805] (Application example 1)

[0806] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0807] Today's children frequently use social networking sites, increasing their risk of being exposed to inappropriate content and dangerous interactions. However, it is difficult for parents and administrators to monitor these risks in real time and respond appropriately. Conventional monitoring systems have low accuracy in detecting dangerous content and often delay notification to parents and administrators. Therefore, there is a need for an effective monitoring system to ensure children's safety.

[0808] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0809] In this invention, the server includes means for monitoring children's use of SNS, means for using a generative AI model to detect dangerous text, means for monitoring SNS interactions using the API of the SNS platform, means for notifying parents or administrators in real time when danger is detected, means for improving the system based on feedback, means for generating and providing regular reports to parents or administrators, and means for inputting prompt sentences into the generative AI model to analyze the content of the text, thereby making it possible to effectively monitor children's use of SNS and quickly detect dangerous content and notify parents or administrators.

[0810] "Means for monitoring children's use of social media" refers to technologies for tracking activities on social media platforms used by children and obtaining the content of posts and messages.

[0811] "Means using generative AI models to detect dangerous text" refers to technology that uses generative AI models to analyze text on social media and automatically detect inappropriate or dangerous content.

[0812] "Methods for monitoring social media interactions using the API of a social media platform" refers to technology that utilizes the API provided by a social media platform to obtain and monitor user posts and messages.

[0813] "Means for notifying parents or administrators in real time when danger is detected" refers to technology that sends immediate notifications to parents or administrators when dangerous content is detected.

[0814] "Means for improving the system based on feedback" refers to techniques for collecting feedback from users and improving the performance and accuracy of the system based on that feedback.

[0815] The "means for generating periodic reports and providing them to parents and administrators" refers to a technology for generating reports summarizing the monitoring results at regular intervals and providing them to parents and administrators.

[0816] "Means for inputting prompt sentences into a generative AI model to analyze the content of text" refers to a technology in which a specific prompt sentence is input into a generative AI model and the resulting analysis results are used to evaluate the content of the text.

[0817] As an embodiment of the present invention, the following system can be constructed.

[0818] First, to monitor a child's use of social media, the server uses the social media platform's API to retrieve the child's posts and messages. Specifically, the server accesses the API endpoint provided by the social media platform and uses authentication information to retrieve data related to the child's account. This data includes posted text and received messages.

[0819] The server then inputs the acquired text data into a generative AI model to detect dangerous text. The generative AI model uses natural language processing techniques to analyze the content of the text and automatically detect inappropriate or dangerous content. This analysis can use natural language processing libraries such as Hugging Face's transformers library.

[0820] Examples of prompts to input to a generative AI model include:

[0821] "Analyze the following message for potential risks: 'I hate you. Don't talk to me anymore.'"

[0822] If dangerous text is detected, the server sends real-time notifications to parents and administrators via email, SMS, or push notification, allowing them to take immediate action.

[0823] In addition, the server periodically generates a report summarizing the monitoring results and provides it to parents or administrators. This report includes a list of detected dangerous text and analysis results, making it easier for parents and administrators to understand their children's social media usage.

[0824] Finally, the server collects user feedback and uses it to improve the system's performance and accuracy. Feedback is based on ratings and comments provided by parents and administrators. This allows the system to continually improve and become more effective at monitoring.

[0825] In this way, a system can be realized that can effectively monitor children's use of social networking sites, quickly detect dangerous content, and notify parents or administrators.

[0826] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0827] Step 1:

[0828] The server retrieves the child's posts and messages using the SNS platform's API. The SNS platform's API endpoint and authentication information are used as input. The retrieved text data (posts and messages) is obtained as output. Specifically, the server sends an HTTP request to the API endpoint and receives the text data as a response.

[0829] Step 2:

[0830] The server inputs the acquired text data into the generative AI model to detect dangerous text. The acquired text data and prompt text are used as input. The output is the analysis result of the text (risk assessment). Specifically, the prompt text is input into the generative AI model, and the model analyzes the content of the text and assesses the risk level.

[0831] Step 3:

[0832] The server sends real-time notifications to parents and administrators when dangerous text is detected. It uses the analysis results and notification destination information as inputs. It outputs the notification. Specific behaviors include sending a notification to the parent or administrator via email, SMS, or push notification.

[0833] Step 4:

[0834] The server periodically generates a report summarizing the monitoring results and provides it to the parent or administrator. The monitoring result data is used as input. As output, a report is generated and provided to the parent or administrator. Specifically, the monitoring results are compiled, compiled into a report format, and sent via email or other means.

[0835] Step 5:

[0836] The server collects user feedback and uses it to improve the system's performance and accuracy. It uses feedback from parents and administrators as input. It outputs improvements to the system. Specifically, it stores the feedback in a database, analyzes it, and adjusts the system's algorithms and settings.

[0837] Example 2

[0838] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0839] Modern social networking sites (SNS) pose a growing risk of children posting dangerous texts. These dangerous texts may contain violent, discriminatory, or threatening content, and may pose serious danger to children themselves or others. Conventional monitoring systems face the challenge of detecting these dangerous texts in real time and taking appropriate action.

[0840] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for monitoring children's use of the information exchange service, means for using a generative AI model for detecting dangerous text, means for monitoring interactions on the information exchange service using an application program interface of the information exchange service platform, means for notifying parents or administrators in real time when a risk is detected, means for improving the system based on the feedback, means for acquiring text entered by a user, means for inputting the acquired text into the generative AI model, means for the generative AI model to analyze the text and determine whether the text is dangerous, and means for notifying the user of the analysis result. This makes it possible to detect dangerous text before children post it and take appropriate measures.

[0841] "Children" refers to minors, particularly those who require supervision when using information exchange services.

[0842] "Information Exchange Service" refers to an online platform for users to post and share messages and content.

[0843] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to analyze text and understand and judge its content.

[0844] "Application Program Interface" refers to an interface for exchanging data and functions between different software systems.

[0845] "Real-time notification" refers to a function that notifies you immediately when a specific event occurs.

[0846] "Feedback" refers to evaluations and opinions provided based on system performance and results.

[0847] "Text" refers to written data posted by users to information exchange services.

[0848] "Analysis" refers to the process by which a generative AI model understands and evaluates the content of a text.

[0849] "Notification" refers to a message used to communicate analysis results to users and administrators.

[0850] MODE FOR CARRYING OUT THE INVENTION

[0851] This invention is a system that detects dangerous text in real time when a child uses a social networking service (SNS) and takes appropriate action. A specific embodiment of this system will be described below.

[0852] Hardware and software used

[0853] The system uses the following hardware and software:

[0854] Server: A central processing unit for receiving, analyzing, and notifying text data.

[0855] Device: A device (smartphone, computer, etc.) that allows a user to enter text.

[0856] Generative AI model: An artificial intelligence model that uses natural language processing techniques to analyze text (e.g., ChatGPT).

[0857] Data processing and calculation

[0858] 1. Get the text:

[0859] A user uses a terminal to input text to be posted to the information exchange service, and the terminal transmits this text to the server.

[0860] 2. Text Analysis:

[0861] The server inputs the received text into a generative AI model, which uses natural language processing techniques to analyze the content of the text and determine whether it contains dangerous content.

[0862] 3. Notification of analysis results:

[0863] The server receives the analysis results from the generative AI model and notifies the user based on the results via email, short message service, or push notification.

[0864] Specific examples

[0865] For example, suppose a user is trying to post a message on social media that reads, "I'm going to plant a bomb at school tomorrow." The server retrieves this text and inputs it into a generative AI model.

[0866] Example prompt sentence:

[0867] Please determine whether the following text contains dangerous content:

[0868] "Tomorrow I'm going to plant a bomb in the school."

[0869] The generative AI model analyzes this prompt and determines that the text contains the dangerous content of "planting a bomb." As a result, the server determines that the text is dangerous and issues a warning to the user.

[0870] Example of a warning message:

[0871] WARNING: The text you entered contains potentially dangerous content. Please stop posting.

[0872] In this way, the server can use the generative AI model to determine the risk of the text provided by the user and take appropriate action.This system can detect dangerous texts before children post them and notify parents or administrators in real time.

[0873] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0874] Step 1:

[0875] The user enters text

[0876] A user uses a terminal (such as a smartphone or PC) to input text to be posted to an information exchange service. The input text is temporarily stored in the terminal's memory. The input data is the text entered by the user, and the output data is the text stored in the terminal.

[0877] Step 2:

[0878] The device sends the text to the server

[0879] The terminal sends the text entered by the user to the server, where the text data is securely transmitted over the Internet. The input data is the text stored on the terminal, and the output data is the text sent to the server.

[0880] Step 3:

[0881] The server receives the text

[0882] The server receives the text sent from the terminal. The received text is temporarily stored for the next processing step. The input data is the text sent to the server, and the output data is the text stored on the server.

[0883] Step 4:

[0884] The server inputs the text into the generative AI model

[0885] The server inputs the received text into the generative AI model. At this time, it generates a prompt sentence and sends it to the generative AI model along with the text. The input data is the text stored on the server, and the output data is the prompt sentence and text sent to the generative AI model.

[0886] Specific prompt examples:

[0887] Please determine whether the following text contains dangerous content:

[0888] "Tomorrow I'm going to plant a bomb in the school."

[0889] Step 5:

[0890] Generative AI models parse text

[0891] The generative AI model analyzes the prompts and text sent from the server. It uses natural language processing technology to understand the context and keywords of the text and evaluate whether it contains dangerous content. The input data are the prompts and text sent to the generative AI model, and the output data are the analysis results.

[0892] Step 6:

[0893] The server receives the analysis results

[0894] The server receives the analysis results from the generative AI model, which include a judgment of whether the text is safe or unsafe. The input data is the analysis results sent from the generative AI model, and the output data is the analysis results stored on the server.

[0895] Step 7:

[0896] The server notifies the user of the analysis results

[0897] The server notifies the user based on the analysis results. For example, if the text is deemed dangerous, it displays a warning message to the user. The notification can be via email, a short message service, or a push notification. The input data is the analysis results stored on the server, and the output data is the notification message sent to the user.

[0898] Specific warning message example:

[0899] WARNING: The text you entered contains potentially dangerous content. Please stop posting.

[0900] (Application example 2)

[0901] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0902] Modern information and communication services are increasing the risk that children will receive dangerous texts and messages. In particular, phishing emails and messages containing inappropriate content are becoming a problem. This not only threatens children's safety, but also makes it difficult for parents and administrators to respond appropriately. Conventional systems have had difficulty detecting dangerous texts in real time and suggesting appropriate countermeasures.

[0903] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0904] In this invention, the server includes means for monitoring a child's use of information and communication services, means for using a generative AI model to detect dangerous text, means for monitoring interactions with information and communication services using an application programming interface of an information and communication service platform, means for notifying a parent or administrator in real time when a danger is detected, means for improving the system based on the feedback, means for analyzing the text of messages and emails received by a user in real time to determine whether the messages or emails contain dangerous content, and means for displaying a warning to the user and suggesting an appropriate response method when dangerous content is detected. This allows a child to receive an immediate warning when they receive a dangerous text, enabling parents or administrators to respond quickly.

[0905] "Information and communication services" are means of communication provided via the Internet, such as messaging, email, and social networking.

[0906] A "generative AI model" is an artificial intelligence model that uses natural language processing techniques to analyze text and generate output tailored to a specific task.

[0907] An "application programming interface" is a standardized interface that allows different software systems to communicate with each other.

[0908] "Real-time notification" is a function that immediately notifies users of information when a specific event occurs.

[0909] "Feedback" is information for evaluation and improvement based on the performance and results of a system.

[0910] A "message" is information such as text, images, and audio sent and received between users.

[0911] "Email" is a digital form of letter sent and received over the Internet.

[0912] A "warning" is a notification that notifies the user that a danger or problem has occurred.

[0913] A "solution" is an action or measure taken in response to a particular problem or situation.

[0914] The following system configuration will be described as an embodiment of the present invention.

[0915] System Configuration

[0916] The system includes a server, a user terminal, and a generative AI model. The server receives the text of messages and emails sent from the user terminal and analyzes the text using the generative AI model. Based on the analysis results, if the text contains dangerous content, the server notifies the user in real time.

[0917] Hardware and software used

[0918] Hardware: Smartphones, servers

[0919] Software: Python, OpenAI API, smtplib (Python standard library)

[0920] Data processing and calculation

[0921] 1. Text acquisition: The user terminal sends the text of the received message or email to the server.

[0922] 2. Text analysis: The server inputs the received text into a generative AI model and analyzes the text using natural language processing techniques.

[0923] 3. Risk determination: The generative AI model determines whether the text contains risky content.

[0924] 4. Warning Notification: If dangerous content is detected, the server will notify the user in real time via email, short message service, or push notification.

[0925] Specific examples

[0926] For example, if a user receives a phishing email, the content of the email will be analyzed, and if it is deemed dangerous, a warning will be displayed. Specific examples of prompt text are as follows:

[0927] Prompt Sentence Examples

[0928] Determine if the following text is dangerous: "Your account has been hacked. Click this link immediately to change your password."

[0929] In this way, users can be immediately alerted when they receive a dangerous text, allowing parents or administrators to respond quickly.

[0930] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0931] Step 1:

[0932] Obtain the text of messages and emails received by the user terminal.

[0933] Input: Text of a received message or email

[0934] Output: The retrieved text data

[0935] Specific operation: The user device scans the inbox for new messages and emails and extracts their contents in text format.

[0936] Step 2:

[0937] The user terminal transmits the acquired text data to the server.

[0938] Input: Obtained text data

[0939] Output: Text data sent to the server

[0940] Specific operation: The user terminal sends text data to the server using a communication method such as an HTTP request.

[0941] Step 3:

[0942] The server inputs the received text data into the generative AI model.

[0943] Input: Text data sent to the server

[0944] Output: Text data fed into the generative AI model

[0945] Specific operation: The server converts the received text data into an appropriate format for sending to the generative AI model's API.

[0946] Step 4:

[0947] A generative AI model analyzes text data to determine whether it contains dangerous content.

[0948] Input: Text data fed into the generative AI model

[0949] Output: Risk assessment result

[0950] How it works: The generative AI model uses natural language processing techniques to analyze text data and determine whether it contains dangerous content, such as phishing scams or inappropriate content.

[0951] Step 5:

[0952] The server receives the judgment results of the generated AI model and, if dangerous content is detected, notifies the user of a warning.

[0953] Input: Risk assessment result

[0954] Output: A warning notification sent to the user

[0955] Specific operation: The server analyzes the judgment results from the generative AI model and, if they contain dangerous content, sends a warning to the user via email, short message service, or push notification.

[0956] Step 6:

[0957] The user receives a warning notification and confirms the appropriate action to be taken.

[0958] Input: The warning notification sent to the user

[0959] Output: User's response behavior

[0960] Specific actions: The user checks the warning notification they received and takes appropriate action according to the instructions in the notification. For example, if it is a phishing email, they should not click on the link or change their password.

[0961] Example 3

[0962] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0963] The use of modern information and communication networks, especially among children, poses a growing risk of sending and receiving dangerous text messages. Because these dangerous text messages can lead to bullying, threats, and other harmful behavior, parents and administrators must monitor them in real time and take appropriate action. However, current systems lack the means to effectively detect and promptly notify users of dangerous text messages. Furthermore, their ability to improve the system based on feedback is limited. This makes it difficult to prevent dangerous situations before they occur.

[0964] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0965] In this invention, the server includes means for monitoring a child's use of the information and communications network, means for using a generative AI model to detect dangerous text, means for monitoring interactions on the information and communications network using an application programming interface of the information and communications network platform, means for notifying a parent or administrator in real time when a danger is detected, means for improving the system based on the feedback, means for receiving text messages sent by the user from the terminal, means for analyzing the text messages using the generative AI model, means for detecting dangerous text based on the analysis results, and means for updating the generative AI model based on feedback from the parent or administrator. This makes it possible to detect dangerous text messages in real time and quickly notify parents or administrators. Furthermore, by continuously improving the system based on feedback, dangerous text can be detected with greater accuracy.

[0966] "Means for monitoring children's use of information and communications networks" refers to technologies and devices for monitoring activities on information and communications networks used by children and detecting abnormal or dangerous behavior.

[0967] "Means using generative AI models to detect dangerous text" refers to technologies or devices that use generative AI models to analyze the content of text messages and assess their risk.

[0968] "Means for monitoring communications on an information and communications network using the application programming interface of an information and communications network platform" refers to technology or devices for monitoring messages and data communications on the network using the API provided by the information and communications network platform.

[0969] "Means for notifying parents or administrators in real time when danger is detected" refers to technology or devices for sending immediate notifications to parents or administrators when dangerous text is detected.

[0970] "Means for improving the system based on feedback" refers to technologies or devices that improve the performance or accuracy of the system based on feedback from parents or administrators or automatically collected data.

[0971] The "means for receiving a text message sent from a terminal by a user" refers to a technique or device that allows a server to receive a text message sent from a terminal used by a user.

[0972] "Means for analyzing text messages using a generative AI model" refers to technology or devices that use a generative AI model to analyze the content of received text messages and understand their meaning and intent.

[0973] "Means for detecting dangerous text based on analysis results" refers to technologies or devices for determining whether a text message is dangerous based on the analysis results of a generative AI model.

[0974] "Means for updating the generative AI model based on feedback from parents or administrators" refers to technologies or devices that update the learning data of the generative AI model based on feedback information provided by parents or administrators, thereby improving the accuracy of the model.

[0975] MODE FOR CARRYING OUT THE INVENTION

[0976] The present invention is a system for monitoring children's use of information and communication networks and detecting dangerous text messages. A specific embodiment of this system will be described below.

[0977] System configuration

[0978] This system consists of three main components: a server, a device, and a user. The server analyzes text messages using a generative AI model and detects dangerous text. The device is the device used by the user to send text messages, such as a smartphone or PC. The user is the person who actually sends the text messages.

[0979] Hardware and software used

[0980] The server is a computer system equipped with a high-performance processor and a large amount of memory. Specifically, a server equipped with an Intel Xeon processor or an AMD EPYC processor is suitable. The software running on the server includes the Python libraries spaCy and Transformers. These libraries are used to analyze text messages using natural language processing techniques.

[0981] A terminal is a device that allows a user to input and send text messages. This can be a smartphone or a computer, and these devices must be connected to the Internet.

[0982] System Operation

[0983] When a user sends a text message from their device, the device sends the message to a server. The server inputs the received text message into a generative AI model for analysis. The generative AI model evaluates the content of the text based on pre-trained data and determines whether it is dangerous.

[0984] For example, if a user sends a message saying, "I'm waiting for you at school. You can't escape," the server receives this message and inputs it into the generative AI model, which analyzes it and determines that it contains violent content.

[0985] Notifications and Feedback

[0986] If dangerous text is detected, the server sends a real-time notification to parents or administrators via email, short message service (SMS), or push notification, including information such as the content of the dangerous text, the time it occurred, and its location.

[0987] Parents or administrators can provide feedback based on the received notification, for example, "This text is not dangerous," and the server will use this feedback to update the generative AI model and reflect it in future decisions.

[0988] Specific examples

[0989] As a concrete example, the following prompt sentence is input to the generative AI model:

[0990] Example prompt sentence:

[0991] "It analyzes text messages sent by users to determine if they contain dangerous content, such as messages like: 'I'm waiting for you at school. You can't escape.'"

[0992] In this way, the system can detect dangerous text messages in real time and quickly notify parents or administrators. Furthermore, by continuously improving the system based on feedback, it is possible to detect dangerous text with higher accuracy. The flow of the identification process in Example 3 will be described with reference to FIG. 15.

[0993] Step 1:

[0994] A user sends a text message.

[0995] A user uses a device such as a smartphone or PC to input a text message and press the send button. The input is the text message entered by the user. The output is a command to send the message to the device.

[0996] Step 2:

[0997] The device sends a text message to the server.

[0998] The terminal sends the text message entered by the user to a server over the Internet, encrypting the data using the HTTPS protocol. The input is the text message entered by the user. The output is the text message sent to the server.

[0999] Step 3:

[1000] The server receives the text message.

[1001] The server receives text messages sent from the device. The received data is temporarily stored in a database. The input is the text message sent from the device. The output is the text message stored in the database.

[1002] Step 4:

[1003] The server analyzes the text message using the generative AI model.

[1004] The server uses the Python libraries spaCy and Transformers to analyze text messages. The generative AI model evaluates the content of the text based on pre-trained data. The input is the text message stored in the database. The output is the analysis result.

[1005] Step 5:

[1006] The server detects dangerous text.

[1007] The server determines whether the text is dangerous based on the analysis results of the generative AI model. For example, if the text contains "violent language" or "threatening content," it is deemed dangerous. The input is the analysis results of the generative AI model. The output is the detection result of dangerous text.

[1008] Step 6:

[1009] The server sends a notification to the parent or administrator.

[1010] The server sends real-time notifications to parents and administrators when dangerous text is detected. Notifications can be sent via email, SMS, or push notification. The notification includes information such as the content of the dangerous text, the time it occurred, and its location. The input is the detection result of the dangerous text. The output is the notification sent to the parent or administrator.

[1011] Step 7:

[1012] Parents and administrators provide feedback.

[1013] The parent / administrator provides feedback based on the received notification, for example, "This text is not dangerous" and sends the feedback to the server. The input is the feedback from the parent / administrator. The output is the feedback sent to the server.

[1014] Step 8:

[1015] The server updates the generative AI model based on the feedback.

[1016] The server updates the generative AI model based on feedback from parents and administrators. Specifically, it uses the feedback as training data to improve the accuracy of the model. The input is the feedback from parents and administrators. The output is the updated generative AI model.

[1017] (Application example 3)

[1018] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1019] Today's children frequently use online platforms and social media, increasing their risk of coming into contact with dangerous text or inappropriate content. However, it is difficult for parents and administrators to monitor these dangers in real time and respond appropriately. Furthermore, existing systems have low accuracy in detecting dangerous text, resulting in false positives and missed results. Furthermore, there is a lack of a mechanism for utilizing feedback to improve the system's accuracy, making continuous improvement difficult.

[1020] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1021] In this invention, the server includes means for monitoring a child's online activity, means for using a generative AI model to detect dangerous texts, means for monitoring communications using an online platform's API, means for notifying parents or administrators in real time when a risk is detected, means for improving the system based on feedback, means for sending the notification by email, SMS, or push notification, and means for collecting and learning from feedback to improve the accuracy of the AI ​​model. This makes it possible to detect dangerous texts in real time and quickly notify parents or administrators. The accuracy of the system can also be continuously improved by utilizing feedback.

[1022] "Children's online activities" refers to all of a child's actions and communications that occur using the Internet.

[1023] A "generative AI model" is an artificial intelligence model that has been trained using machine learning techniques and is generated to perform a specific task.

[1024] An "online platform" is a general term for services and applications provided over the Internet, and is a place where users can share information and communicate.

[1025] "API" stands for Application Programming Interface, an interface for exchanging data and functions between different software systems.

[1026] "Communication" refers to the act or process of sending or receiving data or information.

[1027] "Real-time notification" is a function that sends a notification immediately the moment a specific event occurs.

[1028] "Feedback" refers to the act of providing evaluations and opinions on the performance and results of a system, and refers to information that is useful for improving the system.

[1029] "Email" is a form of digital message sent and received over the Internet.

[1030] "SMS" stands for Short Message Service, a service that allows you to send and receive short text messages over a mobile phone network.

[1031] "Push notifications" are notifications sent directly to a user's device from a mobile application or web service.

[1032] "Accuracy" refers to the ability of a system or model to perform a task accurately.

[1033] "Learning" is the process by which a machine learning model finds patterns and rules based on data and improves its performance.

[1034] A system for implementing this invention uses a generative AI model to monitor children's online activity and detect dangerous texts. The system uses the API of an online platform to monitor communications, notifies parents or administrators in real time when a risk is detected, and has the ability to improve the system based on feedback.

[1035] Hardware and software used

[1036] Hardware: Smartphones, servers

[1037] Software: Python, smtplib (sending email), twilio (sending SMS), requests (sending push notifications)

[1038] Data processing and calculation

[1039] 1. Loading the generative AI model: The server loads a generative AI model for detecting dangerous text. This model analyzes text using natural language processing techniques.

[1040] 2. Text evaluation: The server evaluates the text data obtained through the online platform's API using a generative AI model to determine whether the text is dangerous.

[1041] 3. Sending notifications: If dangerous text is detected, the server will send real-time notifications to parents or administrators via email, SMS, or push notification.

[1042] 4. Feedback collection and learning: Feedback is collected from parents and administrators, and the server uses this information to update the generative AI model, thereby improving the accuracy of the system.

[1043] Specific examples

[1044] For example, if a child receives a dangerous message on their smartphone, the content of the message will be notified to the parent in real time. If the parent gives feedback that the text is not dangerous, the server will learn from that information and will not judge similar texts as dangerous in the future.

[1045] Prompt Sentence Examples

[1046] Develop a system to notify parents and administrators of dangerous texts in real time via email, SMS, or push notification, and implement a feedback mechanism to improve the accuracy of the AI ​​model.

[1047] The above is an embodiment of the present invention.

[1048] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1049] Step 1:

[1050] The server loads the generative AI model.

[1051] Input: File path of the generated AI model

[1052] Data processing: Read the model file and expand it in memory

[1053] Output: The loaded generative AI model

[1054] Specific operation: The server loads the generative AI model from the specified file path and extracts it in memory. This model is used to analyze text using natural language processing techniques.

[1055] Step 2:

[1056] The server obtains the text data through the API of the online platform.

[1057] Input: API endpoint, authentication information

[1058] Data processing: Send an API request and retrieve text data

[1059] Output: The retrieved text data

[1060] What it does: The server sends a request to the online platform's API endpoint using the authentication information to retrieve text data about the child's online activities.

[1061] Step 3:

[1062] The server evaluates the acquired text data using a generative AI model.

[1063] Input: Obtained text data, generative AI model

[1064] Data processing: Inputting text data into a generative AI model to assess risk

[1065] Output: Risk assessment results

[1066] Specific operation: The server inputs the acquired text data into the generative AI model and evaluates whether the text is dangerous. The evaluation result is output as a flag indicating whether the text is dangerous.

[1067] Step 4:

[1068] The server sends real-time notifications to parents and administrators if dangerous text is detected.

[1069] Input: Risk assessment results, parent / caregiver contact information

[1070] Data processing: Generate notification messages based on the evaluation results and send them to designated contacts

[1071] Output: Notification message sent

[1072] What it does: When dangerous text is detected, the server generates a notification message with the parent or administrator's contact information and sends it via email, SMS, or push notification.

[1073] Step 5:

[1074] The server collects feedback from parents and administrators and updates the generative AI model.

[1075] Input: Feedback data, generative AI model

[1076] Data processing: Analyzing feedback data and retraining generative AI models

[1077] Output: An updated generative AI model

[1078] How it works: The server collects feedback data from parents and administrators and uses that data to retrain the generative AI model, improving the model's accuracy and resulting in better evaluation results in future evaluations.

[1079] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1080] "Example 1"

[1081] One embodiment of the present invention incorporates an emotion engine. This emotion engine infers a user's emotional state from their text. Specifically, it analyzes text posted by the user on the social networking site and determines whether the text expresses an emotion, such as joy, anger, sadness, surprise, or fear. This emotion engine is implemented by combining natural language processing and machine learning technologies.

[1082] "Example 2"

[1083] Furthermore, the emotion engine assesses the risk level based on the estimated emotional state. For example, if the user's text expresses strong feelings of anger or fear, it is determined that the text is likely to be dangerous. This risk assessment is performed by an AI model, and the results are notified to parents or administrators.

[1084] "Example 3"

[1085] The emotion engine also controls notifications based on the evaluation results. Specifically, notifications are sent to parents or administrators only when the risk level exceeds a certain threshold. This prevents parents or administrators from being confused by unnecessary notifications.

[1086] "Example 4"

[1087] Additionally, the emotion engine tracks changes in the user's emotional state and uses that information to improve the system. Specifically, it analyzes how the user's emotional state changes over time and uses the results to train the AI ​​model. This allows the system to more accurately understand the user's emotional state and provide more appropriate notifications.

[1088] The processing flow of each embodiment will be described below.

[1089] "Example 1"

[1090] Step 1: A user posts text on a social networking site.

[1091] Step 2: The system takes that text and sends it to the emotion engine.

[1092] Step 3: The emotion engine analyzes the text and infers the user's emotional state.

[1093] "Example 2"

[1094] Step 1: Obtain the emotional state estimated by the emotion engine.

[1095] Step 2: The emotion engine assesses the risk level based on the emotional state.

[1096] Step 3: Submit the evaluation results to the system.

[1097] "Example 3"

[1098] Step 1: The system obtains the evaluation results from the emotion engine.

[1099] Step 2: If the evaluation results exceed a certain threshold, the system sends a notification to parents and administrators.

[1100] "Example 4"

[1101] Step 1: The system tracks changes in the user's emotional state.

[1102] Step 2: The system sends the emotional state change information to the emotion engine.

[1103] Step 3: The emotion engine uses the information to update its model and improve the system.

[1104] Example 1

[1105] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1106] Today's children frequently use information exchange services, which can lead to inappropriate or dangerous interactions. However, it is difficult for parents and administrators to monitor these interactions in real time and take appropriate action. It is also difficult to understand children's emotional states, so early detection and response to problems is required.

[1107] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1108] In this invention, the server includes means for monitoring a child's use of the information exchange service, means for using an artificial intelligence model to estimate an emotional state, means for monitoring information exchange transactions using an application programming interface of the information exchange service, means for notifying a parent or administrator of the results of the estimated emotional state in real time, and means for improving the system based on the feedback. This makes it possible to effectively monitor a child's use of the information exchange service and understand their emotional state, thereby detecting problems early and taking appropriate measures.

[1109] "Means for monitoring children's use of information exchange services" refers to a function for monitoring interactions with information exchange services used by children and collecting appropriate data.

[1110] The "artificial intelligence model for estimating emotional states" is an algorithm that combines natural language processing technology and machine learning technology to estimate emotional states from text data.

[1111] "Application programming interface of an information exchange service" means an interface for accessing data and functions within a service using the API provided by the information exchange service.

[1112] "Means for notifying parents or administrators of the results of estimated emotional state in real time" is a notification function for immediately conveying the results of the analysis of emotional state to parents or administrators.

[1113] "Means for improving the system based on feedback" refers to methods for improving the performance and functionality of a system based on feedback from users and operational data of the system.

[1114] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[1115] "Email, SMS, or Push Notification" refers to a communication method for conveying information to a user. Email is a message sent over the Internet, SMS is a short text message sent over a mobile phone network, and Push Notification is a notification sent directly from an application to a user's device.

[1116] MODE FOR CARRYING OUT THE INVENTION

[1117] This invention is a system that monitors children's use of information exchange services and estimates their emotional state. The system operates in cooperation with a server, a terminal, and a user.

[1118] Hardware and software used

[1119] The server is a server machine with a high-performance CPU and sufficient memory. The software uses the information exchange service's application programming interface (API), a natural language processing library (e.g., NLTK or spaCy), and a machine learning framework (e.g., TensorFlow or PyTorch).

[1120] Program processing

[1121] The server uses the API of the information exchange service to obtain text posted and messages received by the child. Specifically, the server periodically calls the API to collect the latest posts and messages. The server then passes the collected text data to the emotion engine. The emotion engine combines natural language processing (NLP) and machine learning technologies to infer emotional states from the text. For example, the emotion engine analyzes the text and determines whether the text expresses an emotion, such as joy, anger, sadness, surprise, or fear.

[1122] Specific examples

[1123] For example, if a child posts "Today was so much fun!" to an information exchange service, the server retrieves this post and passes it to the emotion engine. The emotion engine analyzes this text and determines that it expresses the emotion of "joy." The server stores the analysis results in a database and notifies parents or administrators in real time. The notification may be, for example, a push notification with a message such as "Your child posted 'Today was so much fun!'. Their emotional state is 'joy.'"

[1124] Prompt Sentence Examples

[1125] Examples of prompts to input to a generative AI model might include:

[1126] "Analyze text posted by a child on an information exchange service and infer their emotional state. For example, if a child posts, 'Today was so much fun!', determine which emotion they are expressing."

[1127] In this way, the server monitors the child's use of the information exchange service and realizes a system that uses an emotion engine to infer the child's emotional state from the text.

[1128] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1129] Program processing flow

[1130] Step 1:

[1131] The server calls the API of the information exchange service.

[1132] Input: API endpoint URL, authentication token, required parameters

[1133] What happens: The server sends a GET request to a URL like "https: / / api.exchangeplatform.com / v1 / messages?user_id=12345" containing the authentication token "Bearer abcdef123456".

[1134] Output: Response from the API (JSON format data)

[1135] Step 2:

[1136] The server retrieves the child's posts and messages.

[1137] Input: Response from the API (JSON format data)

[1138] How it works: The server receives the response from the API and extracts the text posted and messages received by the child. For example, it receives the following JSON data:

[1139] json

[1140] {

[1141] "messages": [

[1142] {"text": "Today was so much fun!", "timestamp": "2023-10-01T12:34:56Z"},

[1143] {"text": "I have too much homework and I'm tired...", "timestamp": "2023-10-01T14:56:78Z"}

[1144] ]

[1145] }

[1146] The server extracts the text and timestamp from this data.

[1147] Output: Extracted text data and timestamps

[1148] Step 3:

[1149] The server passes the acquired text data to the emotion engine.

[1150] Input: Extracted text data

[1151] How it works: The server passes the acquired text data, "Today was so much fun!", to the emotion engine, which uses a natural language processing library to analyze this text.

[1152] Output: Text data passed to the emotion engine

[1153] Step 4:

[1154] The emotion engine analyzes the text data and estimates the emotional state.

[1155] Input: Text data passed to the emotion engine

[1156] How it works: The emotion engine analyzes the text "I had so much fun today!" and infers the emotion "joy." The analysis uses a pre-trained model.

[1157] Output: Estimated emotional state (e.g., "joy")

[1158] Step 5:

[1159] The server stores the analysis results of the emotion engine.

[1160] Input: Estimated emotional state, text data, timestamp

[1161] How it works: The server stores the analysis results returned by the emotion engine in a database. The stored data includes the text "I had so much fun today!", the emotional state "Joy", and the timestamp "2023-10-01T12:34:56Z".

[1162] Output: Analysis results stored in a database

[1163] Step 6:

[1164] The server notifies the terminal of the analysis results.

[1165] Input: Analysis results stored in the database

[1166] Operation: The server notifies the device of the analysis result, "Joy." For example, the notification may be a push notification with a message such as, "Your child posted, 'Today was so much fun!' Their emotional state is 'Joy.'"

[1167] Output: Notification sent to the device

[1168] In this way, the server monitors the child's use of the information exchange service, estimates the child's emotional state from the text using an emotion engine, and notifies parents or administrators in real time.

[1169] (Application example 1)

[1170] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1171] Today's children frequently use social networking sites (SNS), increasing the likelihood that they will encounter dangerous situations such as bullying, suicide, and drug use. However, it is difficult for parents and administrators to monitor their children's SNS usage in real time and take appropriate action. Conventional monitoring systems are inadequate in estimating emotional states or detecting specific keywords, and may miss dangerous situations that children are in. Therefore, there is a need for a system that can effectively monitor children's SNS usage, detect dangerous situations early, and notify parents or administrators.

[1172] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for monitoring a child's use of SNS, means for using an AI model to detect dangerous text, means for monitoring SNS interactions using the SNS platform's API, means for notifying parents or administrators in real time when danger is detected, means for improving the system based on feedback, means for estimating an emotional state from text using an emotion engine, means for detecting posts or messages containing specific keywords, and means for sending a notification based on the detected emotional state or keyword. This makes it possible to quickly and accurately notify parents or administrators when a child encounters a dangerous situation on SNS.

[1173] "Methods for monitoring children's use of social media" refers to technologies for tracking activity on social media accounts used by children and obtaining the content of posts and messages.

[1174] "Means for using AI models to detect dangerous text" refers to technology that uses machine learning algorithms to automatically identify dangerous content from text on social media.

[1175] "Methods for monitoring social media interactions using the API of a social media platform" refers to technology that utilizes the application programming interface (API) provided by a social media platform to obtain and monitor user posts and messages.

[1176] "Means for notifying parents or administrators in real time when danger is detected" refers to technology that immediately sends a warning to parents or administrators when a dangerous situation is detected.

[1177] "Means for improving the system based on feedback" refers to techniques for improving the performance and accuracy of a system based on feedback from users and operational data from the system.

[1178] The "means for inferring an emotional state from text using an emotion engine" is a technology that combines natural language processing technology and machine learning technology to infer a user's emotional state from text.

[1179] "Means for detecting posts and messages containing specific keywords" refers to technology that automatically identifies posts and messages containing pre-set dangerous keywords.

[1180] "Means for sending notifications based on detected emotional states or keywords" refers to technology that sends notifications to parents or administrators when a dangerous situation is identified by the emotion engine or keyword detection function.

[1181] The following system configuration will be described as an embodiment of the present invention.

[1182] System Configuration

[1183] This system monitors children's use of social media, detects dangerous situations, and notifies parents or administrators. The system consists of the following main components:

[1184] 1. How to monitor your child's social media use

[1185] The server uses the API of the social media platform to retrieve the child's posts and messages, allowing for real-time monitoring of the child's social media usage.

[1186] 2. Using AI models to detect dangerous text

[1187] The server automatically identifies dangerous content from captured text using an AI model with machine learning algorithms, which analyzes the text using natural language processing techniques.

[1188] 3. A method for inferring emotional states from text using an emotion engine

[1189] The server uses an emotion engine to infer the user's emotional state from the text. The emotion engine is realized by combining natural language processing and machine learning technologies.

[1190] 4. How to find posts or messages containing specific keywords

[1191] The server automatically identifies posts and messages containing pre-defined dangerous keywords, a list of which can be configured by parents or administrators.

[1192] 5. Real-time notification to parents and administrators when danger is detected

[1193] The server instantly sends email, SMS or push notifications to parents or administrators when a dangerous situation is detected.

[1194] 6. How to improve the system based on feedback

[1195] The server makes improvements to improve the performance of the AI ​​model and emotion engine based on user feedback and system operation data.

[1196] Hardware and software used

[1197] Hardware: Smartphones, servers

[1198] Software: Python, SNS platform API, TextBlob (natural language processing library), scikit-learn (machine learning library)

[1199] Data processing and calculation

[1200] 1. Data acquisition: The server acquires the child's posts and messages via the social media platform's API.

[1201] 2. Sentiment Analysis: The server uses TextBlob to perform sentiment analysis on the text and estimate the emotional state.

[1202] 3. Risk detection: The server uses machine learning models to analyze the text and detect risky content and specific keywords.

[1203] 4. Sending notifications: The server sends notifications to parents or administrators based on the detected emotional state or keywords.

[1204] Specific examples

[1205] If a child posts that they are being bullied at school, the emotion engine will detect "sadness" and send a notification to the parents.

[1206] If a child sends a message saying "I want to commit suicide," the set keyword "suicide" will be detected and a warning will be sent to the parents.

[1207] Prompt Sentence Examples

[1208] Create a program that analyzes text posted by children on social media and determines whether the text expresses any emotion, such as joy, anger, sadness, surprise, fear, etc. Additionally, add a function that sends a notification to parents if posts or messages containing specific keywords (e.g., bullying, suicide, drugs, etc.) are detected.

[1209] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1210] Step 1:

[1211] The server uses the social media platform's API to retrieve the child's posts and messages. The input requires the social media platform's API endpoint and the child's account information. The output is the retrieved post and message data, which is often provided in JSON format.

[1212] Step 2:

[1213] The server inputs the text data of the posts and messages it retrieves into the emotion engine. The emotion engine performs sentiment analysis on the text using TextBlob. The input requires text data. The output is the emotional state expressed by the text (e.g., joy, anger, sadness, surprise, fear).

[1214] Step 3:

[1215] The server uses a machine learning model to detect dangerous text based on the output of the emotion engine. The inputs are the emotional state and the text data. The output is a judgment result indicating whether the text contains dangerous content. This judgment result is often expressed as a risk score.

[1216] Step 4:

[1217] The server detects posts and messages containing specific keywords. As input, it requires text data and a predefined list of keywords. As output, it determines whether the keyword was detected or not. This determination result is provided as keyword match information.

[1218] Step 5:

[1219] The server sends real-time notifications to parents and administrators when dangerous text or specific keywords are detected. The inputs required are risk scores and keyword matches. The output is a generated notification message, which can be sent via email, SMS, or push notification.

[1220] Step 6:

[1221] The server makes improvements to improve the performance of the AI ​​model and emotion engine based on user feedback and system operation data. Feedback data and operation data are required as input. Improved AI models and emotion engines are obtained as output. This improvement process is carried out periodically.

[1222] Example 2

[1223] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1224] Today's children frequently use online communication, increasing the risk of coming into contact with dangerous texts and harmful content. However, it is difficult for parents and administrators to monitor these interactions in real time and respond appropriately. Furthermore, conventional systems lack the ability to analyze the emotional state of text in detail and evaluate the level of risk, so they may miss dangerous situations.

[1225] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for monitoring children's online communication usage, a means for using a generative AI model to detect dangerous text, a means for monitoring communication exchanges using an interface of an online platform, a means for notifying parents or administrators in real time when danger is detected, a means for improving the system based on feedback, and a means for estimating the emotional state of the text using an emotion engine and assessing the level of danger. This enables parents or administrators to respond quickly when children encounter dangerous situations online.

[1226] "Means for monitoring children's online communication use" refers to technologies and devices that monitor and record children's online activities, such as sending and receiving messages and postings, in real time.

[1227] "Means using generative AI models to detect dangerous text" refers to technologies and devices that utilize generative AI models that use natural language processing technology to analyze and determine whether input text contains dangerous content.

[1228] "Means of monitoring communication exchanges using the interfaces of online platforms" refers to technologies and devices that use APIs and interfaces provided by online platforms such as social media platforms and messaging apps to monitor messages and posts between users.

[1229] "Means for notifying parents or administrators in real time when danger is detected" refers to technology or devices for sending immediate notifications to parents or administrators when dangerous text is detected.

[1230] "Means for improving the system based on feedback" refers to technologies and devices for improving the performance and accuracy of the system based on the system's usage status and detection results.

[1231] "Means for estimating the emotional state of text using an emotion engine and assessing its risk level" refers to technology or devices for analyzing the emotions contained in text and assessing the risk level of the text based on that emotional state.

[1232] The present invention is a system for monitoring children's online communication and detecting risky texts. The system uses a generative AI model and an emotion engine to analyze the content and emotional state of the text and assess its risk level.

[1233] Hardware and software used

[1234] Device: A device (e.g., smartphone, tablet, or computer) through which a user enters text.

[1235] Server: A computer system for receiving text and analyzing it using a generative AI model and emotion engine.

[1236] Generative AI models: Models that use natural language processing techniques (e.g., ChatGPT).

[1237] Emotion engine: Software for estimating the emotional state of text and assessing its riskiness.

[1238] Program processing

[1239] 1. Getting the text

[1240] A user inputs text through a terminal.

[1241] The terminal sends the entered text to the server.

[1242] 2. Text Analysis

[1243] The server inputs the received text into a generative AI model.

[1244] The generative AI model uses a natural language generation model such as ChatGPT.

[1245] The model analyzes the content of the text and determines whether the text contains dangerous content.

[1246] 3. Evaluation by Emotion Engine

[1247] The server passes the analysis results to the emotion engine.

[1248] The emotion engine estimates the emotional state contained in the text.

[1249] For example, if the text expresses strong feelings of anger or fear, the emotional state is estimated.

[1250] The emotion engine reassess the risk level based on the estimated emotional state.

[1251] 4. Risk Assessment and Notification

[1252] The server determines the final risk level based on the evaluation results obtained from the emotion engine.

[1253] If the risk is deemed high, the server will send a notification to parents or administrators.

[1254] The notification includes the text content and the risk assessment result.

[1255] Specific examples

[1256] For example, if a user enters text such as "I can't take it anymore, I want to hurt someone," this text is sent from the device to the server. The server inputs this text into ChatGPT and analyzes the content of the text. Based on the analysis results, the emotion engine infers that the text expresses strong anger and evaluates it as high risk. This evaluation result is notified to parents or administrators.

[1257] Prompt Sentence Examples

[1258] Examples of prompts to input to a generative AI model might include:

[1259] "A user entered the following text. Please determine if this text contains dangerous content: 'I can't take it anymore, I want to hurt someone.'"

[1260] In this way, the system can analyze the user's text, assess the risk level, and prompt appropriate responses.

[1261] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1262] Step 1: User Enters Text

[1263] Specific operation: The user enters text into an input field on the terminal.

[1264] Input: Text typed by the user (e.g., "I can't take it anymore, I want to hurt someone").

[1265] Output: The entered text is saved to the terminal.

[1266] Step 2: The device sends the text to the server

[1267] Specific operation: The terminal sends the entered text to the server as an HTTP request.

[1268] Input: The text entered by the user.

[1269] Output: The text is sent to the server in JSON format.

[1270] Step 3: The server inputs the text into the generative AI model

[1271] Specific operation: The server generates a prompt sentence to input the received text into the generative AI model.

[1272] Input: Text sent from the terminal.

[1273] Output: Prompt sentence to be input to the generative AI model (e.g., "The user entered the following text. Please determine whether this text contains dangerous content: 'I can't take it anymore, I want to hurt someone.'").

[1274] Step 4: The generative AI model analyzes the text

[1275] Specific operation: The generative AI model receives a prompt sentence and analyzes the content of the text.

[1276] Input: The prompt sentence fed into the generative AI model.

[1277] Output: Analysis result of the text (e.g., whether the text contains dangerous content).

[1278] Step 5: The server passes the analysis results to the emotion engine

[1279] Specific operation: The server passes the analysis results obtained from the generative AI model to the emotion engine.

[1280] Input: Analysis results from a generative AI model.

[1281] Output: The analysis results that are passed to the emotion engine.

[1282] Step 6: The emotion engine estimates the emotional state

[1283] Specific behavior: The emotion engine performs detailed analysis and estimates the emotional state of the text.

[1284] Input: Analysis results passed from the server.

[1285] Output: Estimated emotional state (e.g., the text expresses strong anger).

[1286] Step 7: The server evaluates the risk and notifies you of the result

[1287] Specific operation: The server determines the final risk level based on the evaluation results obtained from the emotion engine. If the risk level is determined to be high, a notification is sent to parents or administrators.

[1288] Input: Rating results from the sentiment engine.

[1289] Output: Notification sent to parent or administrator (e.g. email, SMS, or push notification).

[1290] In this way, the system can analyze the user's text, assess the risk level, and prompt appropriate responses.

[1291] (Application example 2)

[1292] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1293] Modern communication services, especially those used by children, are prone to dangerous text messages and inappropriate content, which can pose a threat to children's safety. It is also difficult for parents and administrators to detect these dangers in real time and take appropriate action. Furthermore, risk assessments do not take into account emotional states, so dangerous situations may be overlooked.

[1294] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1295] In this invention, the server includes means for monitoring a child's use of a communication service, means for using a generative AI model to detect dangerous text, means for monitoring communication exchanges using an API of a communication service platform, means for notifying a parent or administrator in real time when a danger is detected, means for improving the system based on the feedback, means for estimating the emotional state of the text using an emotion engine and assessing the level of danger, and means for sending a notification when the level of danger is high. This makes it possible to detect dangerous text messages in a child's use of a communication service in real time, assess the level of danger taking the emotional state into account, and promptly notify a parent or administrator.

[1296] "Means for monitoring children's use of communication services" refers to technology for monitoring the activities of communication services used by children in real time and detecting abnormal or dangerous behavior.

[1297] "Means using generative AI models to detect dangerous text" refers to technology that uses generative AI models to analyze text messages and determine whether their content is dangerous.

[1298] "Means for monitoring communication exchanges using APIs of communication service platforms" refers to technology that uses APIs provided by communication services to monitor message exchanges between users.

[1299] "Means for notifying parents or administrators in real time when danger is detected" refers to technology that immediately sends a notification to parents or administrators when dangerous text is detected.

[1300] "Means for improving the system based on feedback" refers to techniques for collecting feedback from users and improving the performance and accuracy of the system based on that feedback.

[1301] The "means for estimating the emotional state of text using an emotion engine and assessing the risk level" is a technology that uses an emotion engine to analyze the emotional state of text and assesses the risk level based on the results.

[1302] The "means for sending a notification when the risk is high" is a technology for sending a notification to a parent or administrator when the risk is determined to be high.

[1303] The following system configuration will be described as an embodiment of the present invention.

[1304] System Configuration

[1305] Hardware and Software

[1306] Hardware: Smartphone

[1307] Software: Python, OpenAI API, TextBlob

[1308] Program processing explanation

[1309] How to monitor your child's use of communication services

[1310] The server monitors the activities of the communication services used by the child in real time, including obtaining message exchanges between users using the communication service platform's API.

[1311] How to use generative AI models to detect dangerous text

[1312] The server inputs the acquired text message into a generative AI model to analyze whether its content is dangerous. Specifically, it uses OpenAI's generative AI model (e.g., GPT-3) to determine whether the text is dangerous.

[1313] A means of using an emotion engine to estimate the emotional state of text and assess its riskiness

[1314] The server uses TextBlob to estimate the emotional state of the text: if the text expresses strong anger or fear, the emotion engine will rate the text as highly dangerous.

[1315] A means to notify parents and administrators in real time when danger is detected

[1316] The server sends real-time notifications to parents and administrators when dangerous texts are detected via email, short message service, or push notification.

[1317] A means of improving the system based on feedback

[1318] The server collects user feedback and uses it to improve the system's performance and accuracy, allowing the system to be continuously improved.

[1319] Specific examples

[1320] For example, if a child sends a message saying "Come see me now, or else...", the server inputs this message into the generative AI model to assess its risk level. The generative AI model determines that this message is dangerous and uses TextBlob to estimate the emotional state. If the emotion engine determines that this message expresses anger, it is assessed as high risk. The server then sends a notification to the parent or administrator saying "Warning: Dangerous text detected: Come see me now, or else...".

[1321] Prompt Sentence Examples

[1322] "Decide if this text is dangerous: Come see me now, or..."

[1323] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1324] Step 1:

[1325] The server uses the API of the communication service platform to obtain message exchanges between users. The input is message data obtained from the communication service platform, and the output is a text message to be analyzed. This step includes the specific operations of making an API call and obtaining message data.

[1326] Step 2:

[1327] The server inputs the acquired text message into the generative AI model and analyzes whether its content is dangerous. The input is the text message acquired in step 1, and the output is the risk assessment result by the generative AI model. This step includes the specific operation of inputting a prompt sentence into the generative AI model (e.g., GPT-3) and obtaining the analysis result.

[1328] Step 3:

[1329] The server uses TextBlob to estimate the emotional state of the text. The input is the text message obtained in step 1, and the output is the evaluation result of the emotional state. This step includes the specific operation of performing sentiment analysis of the text using the TextBlob library.

[1330] Step 4:

[1331] The server evaluates the risk level of the text based on the analysis results of the generative AI model and the evaluation results of the emotional state. The input is the output of Step 2 and Step 3, and the output is the final risk assessment result. This step includes specific operations to integrate the analysis results and the emotional state and evaluate the risk level.

[1332] Step 5:

[1333] If the risk is assessed as high, the server sends a real-time notification to parents or administrators. The input is the risk assessment result from step 4, and the output is the notification message. This step includes specific actions to send the notification via email, short message service, or push notification.

[1334] Step 6:

[1335] The server collects user feedback and uses it to improve the system's performance and accuracy. The input is user feedback data, and the output is improved system settings and model parameters. This step includes collecting and analyzing feedback data and taking specific actions to improve the system.

[1336] Example 3

[1337] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1338] Today's children frequently use online communication platforms, which can contain dangerous texts or inappropriate content. Parents and administrators are required to monitor these dangerous texts in real time and take appropriate action. However, manual monitoring is labor-intensive and inefficient. It is also difficult to track changes in emotional state and provide timely notifications. A system to solve these challenges is needed.

[1339] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1340] In this invention, the server includes means for monitoring a child's online communication usage, means for using a machine learning model to detect dangerous text, means for monitoring interactions using an interface of an online communication platform, means for notifying parents or administrators in real time when a risk is detected, means for improving the system based on the feedback, means for evaluating a user's emotional state using an emotion engine and controlling notifications, and means for tracking changes in the user's emotional state and using the information to improve the system, thereby enabling real-time detection and notification of dangerous text, tracking changes in emotional state, and continuous improvement of the system.

[1341] "Means for monitoring children's online communication usage" refers to technologies for monitoring interactions on online communication platforms used by children and for collecting and analyzing text messages and other content in real time.

[1342] A "machine learning model for detecting dangerous text" is an algorithm that uses natural language processing techniques to analyze text messages and automatically detect text that contains violent, threatening, or other dangerous content.

[1343] "Means for monitoring interactions using the interface of an online communication platform" refers to technology that links with an online communication platform through an API or other interface to monitor user interactions.

[1344] "Means for real-time notification to parents or administrators when dangerous text is detected" refers to technology that sends immediate notifications to parents or administrators when dangerous text is detected, via email, short message service, or push notification.

[1345] "Means for improving the system based on feedback" refers to techniques for improving the performance and accuracy of the system using feedback from parents and administrators or automatically collected data.

[1346] The "means for evaluating a user's emotional state using an emotion engine and controlling notifications" is a technology for analyzing a user's text messages to evaluate their emotional state and controlling the sending of notifications based on the evaluation results.

[1347] "Means for tracking changes in the user's emotional state and using that information to improve the system" refers to technology that continuously tracks changes in the user's emotional state and uses that data to train the system and improve its accuracy and performance.

[1348] MODE FOR CARRYING OUT THE INVENTION

[1349] This invention is a system that monitors children's online communication, detects dangerous texts, and notifies parents or administrators in real time if necessary. Furthermore, the system can be improved based on feedback, and by assessing and tracking the user's emotional state, it can provide more appropriate notifications.

[1350] System configuration

[1351] The server has the following main functions:

[1352] 1. How to monitor your child's online communication

[1353] 2. Using machine learning models to detect dangerous text

[1354] 3. Using the interface of online communication platforms to monitor interactions

[1355] 4. Real-time notification to parents and administrators when danger is detected

[1356] 5. How to improve the system based on feedback

[1357] 6. A means to evaluate the user's emotional state using an emotion engine and control notifications

[1358] 7. A way to track changes in the user's emotional state and use that information to improve the system

[1359] Hardware and software used

[1360] The servers will be built using a cloud-based infrastructure, specifically using the following software and services:

[1361] Natural language processing (NLP) technology: Google Cloud Natural Language API, IBM Watson Natural Language Understanding

[1362] Notification services: email, short message service (SMS), push notifications

[1363] Data processing and calculation

[1364] The server receives text messages sent by users in real time and analyzes them using natural language processing technology. If dangerous text is detected based on the analysis results, a notification is sent to parents or administrators. The notification includes information such as the content of the dangerous text, the time it occurred, and the location where it occurred.

[1365] Additionally, the system receives feedback from parents and administrators and uses that feedback to update the AI ​​model, improving the system's accuracy.

[1366] The emotion engine analyzes users' text messages and evaluates their emotional state. Based on the evaluation results, notifications are sent only when the risk level exceeds a certain threshold, preventing parents and administrators from being overwhelmed by unnecessary notifications.

[1367] The emotion engine also tracks changes in the user's emotional state and uses that data to improve the system, allowing it to more accurately understand the user's emotional state and provide more appropriate notifications.

[1368] Specific examples

[1369] If a user sends a message such as "I'm being bullied at school," the server will detect this message as dangerous text and send a notification to the parent or administrator stating, "A user said they are being bullied at school. Time: 14:30, Location: School."

[1370] Example prompt sentence:

[1371] "Please create a program that analyzes text messages sent by users and notifies parents or administrators if dangerous content is found." The flow of the identification process in the third embodiment will be described with reference to FIG. 21.

[1372] Step 1:

[1373] A user sends a text message using a device (such as a smartphone or PC). The input is the text message entered by the user, and the output is that message being sent to the server. Specifically, the user enters the message "I'm being bullied at school" and presses the send button.

[1374] Step 2:

[1375] The server receives text messages sent by users. The input is the text message sent by the user, and the output is that message being prepared for analysis. In concrete terms, the server receives the message "I'm being bullied at school."

[1376] Step 3:

[1377] The server analyzes the received text message using natural language processing (NLP) technology. The input is the received text message, and the output is the analysis result. Specifically, the server analyzes the message "I'm being bullied at school" using Google Cloud Natural Language API and IBM Watson Natural Language Understanding.

[1378] Step 4:

[1379] If the server detects dangerous text based on the analysis results, it prepares to send a notification to parents or administrators. The input is the analysis result, and the output is the content of the notification. Specifically, the server determines that the message "I'm being bullied at school" is dangerous and prepares the content of the notification.

[1380] Step 5:

[1381] The server sends the prepared notification to the parent or administrator. The input is the notification content, and the output is the notification sent to the parent or administrator. Specifically, the server sends a notification to the parent or administrator with the content "A user said they are being bullied at school. Time: 14:30, Location: School."

[1382] Step 6:

[1383] Parents or administrators provide feedback on notifications from the system. The input is the notification from the system, and the output is the feedback. A specific action is for a parent to send feedback saying, "This text is not dangerous."

[1384] Step 7:

[1385] The server receives feedback from parents or administrators and analyzes its content. The input is the feedback, and the output is the analysis result. Specifically, the server receives feedback such as "This text is not dangerous" and analyzes it.

[1386] Step 8:

[1387] The server uses the text that received the feedback as training data to update the AI ​​model. The input is the analysis result, and the output is the updated AI model. Specifically, the server uses the text "I'm being bullied at school" as training data and updates the AI ​​model.

[1388] Step 9:

[1389] The emotion engine analyzes the user's text message and evaluates its emotional state. The input is the user's text message, and the output is the evaluation result of the emotional state. Specifically, the emotion engine analyzes the message "very sad" and evaluates the danger level.

[1390] Step 10:

[1391] The server receives the emotion engine's evaluation results and sends notifications as necessary. The input is the evaluation result of the emotional state, and the output is the notification sent to the parent or administrator. Specifically, the server receives the emotion engine's evaluation results and sends a notification only if it determines that the risk is high.

[1392] Step 11:

[1393] The emotion engine tracks changes in the user's emotional state and uses that data to improve the system. The input is data on changes in the user's emotional state, and the output is an improved system. Specifically, the emotion engine analyzes messages like "I'm happy today" and "I'm sad now" and records the changes in emotional state.

[1394] Step 12:

[1395] The server receives data from the emotion engine and updates the AI ​​model. The input is data on changes in emotional state, and the output is an updated AI model. Specifically, the server updates the AI ​​model based on the analysis results to more accurately understand the user's emotional state.

[1396] (Application example 3)

[1397] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1398] Today's children access a great deal of information and communicate through social networking sites. However, this also increases the risk of children coming into contact with dangerous texts and inappropriate content. It is difficult for parents and administrators to grasp these risks in real time and respond appropriately. Excessive notifications can also confuse parents and administrators. Furthermore, to improve the accuracy of the system, it is necessary to accurately understand the user's emotional state and incorporate their feedback. Effective methods to solve these issues are needed.

[1399] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1400] In this invention, the server includes: means for monitoring a child's use of social media; means for using an AI model to detect dangerous texts; means for monitoring social media interactions using the social media platform's API; means for notifying parents or administrators in real time when a risk is detected; means for improving the system based on feedback; means for tracking the user's emotional state using an emotion engine and using that information to improve the system; and means for sending a notification only when the risk level exceeds a certain threshold. This allows parents and administrators to grasp the risks of their children's use of social media in real time and respond appropriately. It also prevents excessive notifications and improves the accuracy of the system.

[1401] "Means for monitoring children's use of social media" refers to technology for tracking and recording social media activity used by children in real time.

[1402] "Means using AI models to detect dangerous text" refers to techniques that use natural language processing techniques to analyze text and utilize artificial intelligence models to automatically identify dangerous content.

[1403] "Methods for monitoring social media interactions using the API of a social media platform" refers to technology that monitors messages and posts on social media using the application programming interface (API) provided by the social media platform.

[1404] "Means for notifying parents or administrators in real time when danger is detected" refers to technology that sends immediate notifications to parents or administrators when dangerous text is detected.

[1405] "Means for improving the system based on feedback" refers to techniques for improving the accuracy and performance of the system using feedback from parents and administrators or data collected automatically by the system.

[1406] "Means of tracking the user's emotional state using an emotion engine and using that information to improve the system" is a technology that analyzes the user's emotional state and tracks changes in it to improve the accuracy of the system.

[1407] The "means for sending a notification only when the risk level exceeds a certain threshold" is a technology that sends a notification to parents or administrators only when the risk level of the detected dangerous text exceeds a preset threshold.

[1408] A system for implementing this invention uses an AI model to monitor children's use of social media and detect dangerous texts. The system monitors social media interactions using the social media platform's API and sends real-time notifications to parents and administrators when dangerous texts are detected. The system also improves the system based on feedback, using an emotion engine to track the user's emotional state and using that information to improve the system's accuracy. Furthermore, the system has the ability to send notifications only when the level of danger exceeds a certain threshold.

[1409] Hardware and software used

[1410] Hardware: Smartphones, servers

[1411] Software: Python, smtplib (sending emails), twilio (sending SMS), AI model (emotion engine)

[1412] Data processing and calculation

[1413] 1. Text sentiment analysis: The server uses an emotion engine to analyze text on SNS in real time and evaluate the risk level and emotional state.

[1414] 2. Sending notifications: If the risk exceeds a certain threshold, the server will send notifications to parents or administrators via email and SMS.

[1415] 3. Collecting feedback and improving the system: Feedback from parents and administrators will be collected and used to improve the accuracy of the AI ​​model.

[1416] Specific examples

[1417] For example, if a child sends the text "This is a dangerous message" in a chat app, the server analyzes the text using an emotion engine and sends a real-time notification to the parent if it determines that the message is highly dangerous.

[1418] Prompt Sentence Examples

[1419] Analyze the text below and assess the level of danger and emotional state.

[1420] Text: "This is a dangerous message"

[1421] In this way, the system allows parents and administrators to understand the risks of their children's use of social media in real time and take appropriate action, while also preventing excessive notifications and improving the accuracy of the system.

[1422] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1423] Step 1:

[1424] The server monitors the child's social media usage using the social media platform's API. As input, it receives message data obtained from the social media platform and generates monitored text data as output. This text data is used for sentiment analysis in the next step.

[1425] Step 2:

[1426] The server uses an emotion engine to analyze the text data of the monitored object. It receives the text data generated in step 1 as input and generates the risk and emotional state assessment results as output. Specifically, the emotion engine analyzes the content of the text and calculates the risk score and emotional state (e.g., anger, sadness, etc.).

[1427] Step 3:

[1428] The server determines whether the risk level exceeds a certain threshold. As input, it receives the risk level score generated in step 2, and as output, it generates a flag indicating whether a notification is required. Specifically, if the risk level score exceeds a preset threshold, it sets the notification flag.

[1429] Step 4:

[1430] The server sends real-time notifications to parents and administrators when a notification flag is set. It receives the notification flag, text data, risk score, and emotional state as input, and generates email and SMS notification messages as output. Specifically, it uses smtplib to send emails and twilio to send SMS.

[1431] Step 5:

[1432] The server collects feedback from parents and administrators and improves the system. It receives feedback data from parents and administrators as input and generates updated data for the AI ​​model as output. Specifically, it analyzes the feedback content and reflects it as learning data for the AI ​​model.

[1433] Step 6:

[1434] The server uses an emotion engine to track the user's emotional state and uses that information to improve the system. It receives the user's emotional state data as input and generates system improvement data as output. Specifically, it analyzes changes in the user's emotional state and uses the results as training data for the AI ​​model.

[1435] In this way, the system allows parents and administrators to understand the risks of their children's use of social media in real time and take appropriate action, while also preventing excessive notifications and improving the accuracy of the system.

[1436] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1437] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1438] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1439] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1440] [Third embodiment]

[1441] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1442] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1443] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1444] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1445] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1446] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1447] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1448] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1449] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1450] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1451] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1452] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1453] "Example 1"

[1454] One embodiment of the present invention is a system for monitoring children's use of social networking sites. This system monitors children's social networking site interactions via the API of the social networking site platform used by the children. Specifically, it obtains texts posted by the children and messages received by them.

[1455] "Example 2"

[1456] Furthermore, the system of the present invention inputs the acquired text into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing technology. Specifically, it uses a natural language generation model, such as generative AI, to understand the content of the text and determine whether the text contains dangerous content.

[1457] "Example 3"

[1458] Additionally, the system of the present invention notifies parents or administrators in real time when dangerous text is detected. This notification can be sent via email, SMS, or push notification. Specifically, a notification containing information about the dangerous text, including the time and location of the text, is sent to contacts registered by the parent or administrator.

[1459] "Example 4"

[1460] Furthermore, the system of the present invention improves the system based on feedback. Specifically, it uses feedback from parents or administrators, as well as data automatically collected by the system, to improve the accuracy of the AI ​​model. For example, if a parent or administrator provides feedback such as "this text is not dangerous," the AI ​​model is updated to determine that the text is not dangerous.

[1461] The processing flow of each embodiment will be described below.

[1462] "Example 1"

[1463] Step 1: Monitor the child's social media interactions via the API of the social media platform they use, specifically by capturing text messages posted and received by the child.

[1464] Step 2: The acquired text is fed into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing techniques.

[1465] Step 3: If the AI ​​model detects dangerous text, it notifies parents and administrators in real time via email, SMS, or push notification.

[1466] Step 4: Improve the accuracy of the AI ​​model using feedback from parents and administrators, as well as data collected automatically by the system.

[1467] "Example 2"

[1468] Step 1: Monitor the child's social media interactions via the API of the social media platform they use, specifically by capturing text messages posted and received by the child.

[1469] Step 2: The acquired text is fed into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing techniques.

[1470] Step 3: If the AI ​​model detects dangerous text, it notifies parents and administrators in real time via email, SMS, or push notification.

[1471] Step 4: Improve the accuracy of the AI ​​model using feedback from parents and administrators, as well as data collected automatically by the system.

[1472] "Example 3"

[1473] Step 1: Monitor the child's social media interactions via the API of the social media platform they use, specifically by capturing text messages posted and received by the child.

[1474] Step 2: The acquired text is fed into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing techniques.

[1475] Step 3: If the AI ​​model detects dangerous text, it notifies parents and administrators in real time via email, SMS, or push notification.

[1476] Step 4: Improve the accuracy of the AI ​​model using feedback from parents and administrators, as well as data collected automatically by the system.

[1477] "Example 4"

[1478] Step 1: Monitor the child's social media interactions via the API of the social media platform they use, specifically by capturing text messages posted and received by the child.

[1479] Step 2: The acquired text is fed into an AI model to determine whether the text is dangerous. This AI model analyzes the text using natural language processing techniques.

[1480] Step 3: If the AI ​​model detects dangerous text, it notifies parents and administrators in real time via email, SMS, or push notification.

[1481] Step 4: Improve the accuracy of the AI ​​model using feedback from parents and administrators, as well as data collected automatically by the system.

[1482] Example 1

[1483] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1484] Today's children frequently use social media, increasing the risk of being exposed to inappropriate content and dangerous interactions. However, it is difficult for parents and administrators to monitor these interactions in real time and respond appropriately. Conventional monitoring systems have low accuracy in detecting dangerous content and often delay notifications, so a rapid response is required.

[1485] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for monitoring children's use of SNS, a means for using a generative AI model to detect dangerous text, a means for monitoring SNS interactions using the API of the SNS platform, a means for notifying parents or administrators in real time when a risk is detected, a means for improving the system based on feedback, a means for analyzing text using natural language processing technology, a means for analyzing acquired data and detecting inappropriate content, and a means for sending a notification to parents based on the analysis results. This makes it possible to effectively monitor children's use of SNS, quickly detect dangerous interactions, and notify parents.

[1486] "Means for monitoring children's use of social media" refers to technologies for monitoring activities on social media platforms used by children and obtaining the content of posts and messages.

[1487] A "generative AI model for detecting dangerous text" is an artificial intelligence model used to analyze text using natural language processing techniques to detect inappropriate content or dangerous interactions.

[1488] "Means for monitoring social media interactions using the API of a social media platform" refers to technology that utilizes the application programming interface (API) provided by a social media platform to obtain and monitor user posts and messages.

[1489] "Means for notifying parents or administrators in real time when danger is detected" refers to technology that immediately sends a notification to parents or administrators when dangerous content is detected.

[1490] "Means for improving the system based on feedback" refers to techniques for collecting system usage results and feedback from users, and using that information to improve the system's performance and accuracy.

[1491] "Means for analyzing text using natural language processing technology" refers to natural language processing (NLP) technology used to analyze text data and understand its content.

[1492] "Means for analyzing acquired data and detecting inappropriate content" refers to technology for analyzing data acquired from social media platforms and detecting inappropriate content contained therein.

[1493] The "means for sending a notification to the guardian based on the analysis result" is a technology for sending a notification to the guardian as necessary based on the results of the data analysis.

[1494] This invention is a system for monitoring children's use of social networking sites and detecting dangerous texts. This system includes elements of a server, a terminal, and a user, and operates as follows.

[1495] The server first obtains data related to the child's social media account using the social media platform's API. Specifically, the server authenticates using the API key and secret key provided by the social media platform and then obtains the child's posts and messages. For example, the server can use the Twitter API or Facebook Graph API.

[1496] The server then uses natural language processing (NLP) techniques to analyze the retrieved data. Specifically, it uses generative AI models to analyze the text and detect inappropriate content or dangerous interactions. NLP libraries used include NLTK and spaCy.

[1497] Based on the analysis results, the server sends real-time notifications to parents and administrators as needed via email, SMS, or push notifications, allowing parents and administrators to immediately understand their children's social media usage and take appropriate action.

[1498] Furthermore, the server improves the system based on feedback. Specifically, it collects system usage results and user feedback, and uses this information to improve the performance and accuracy of the generative AI model.

[1499] As a concrete example, consider a child using Twitter. The server retrieves the child's latest posts using the Twitter API and analyzes the text using a generative AI model. If inappropriate content is detected based on the analysis results, a notification is sent to the parent via email.

[1500] Examples of prompt sentences include the following:

[1501] "Please create a system to monitor children's social media usage. Use the Twitter API to retrieve children's posts and use a natural language processing library to analyze them for inappropriate content. Send notifications to parents based on the analysis results."

[1502] In this way, the server can effectively monitor children's use of social networking sites, quickly detect dangerous interactions, and notify parents.

[1503] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1504] Step 1: API Authentication

[1505] The server performs authentication using the API key and secret key of the SNS platform. The API key and secret key are required as input. This gives the server permission to obtain data from the SNS platform. Specifically, the server sends an authentication request using the API key and secret key and obtains an authentication token.

[1506] Step 2: Data Acquisition

[1507] After successful authentication, the server retrieves posts and messages related to the child's account. The input requires an authentication token and the child's account information. The output is the retrieved post and message data. Specifically, it sends a GET request using the social media platform's API to retrieve the child's latest posts and messages.

[1508] Step 3: Data analysis

[1509] The server uses natural language processing (NLP) techniques to analyze the retrieved data. The input requires retrieved post and message data. The output is the analysis results. Specifically, it uses a generative AI model to analyze the text and detect inappropriate content or dangerous interactions. NLP libraries used include NLTK and spaCy.

[1510] Step 4: Hazard detection

[1511] The server determines whether dangerous content has been detected based on the results of the data analysis. The input required is the results of the data analysis. The output is a flag indicating a danger detection. Specifically, the server evaluates the analysis results and checks whether they contain inappropriate words or phrases.

[1512] Step 5: Notification

[1513] The server sends real-time notifications to parents and administrators when a risk is detected. The inputs required are the risk detection flag and analysis results. The output is confirmation that a notification has been sent. Specific actions include sending a notification to parents and administrators via email, SMS, or push notification.

[1514] Step 6: Gather feedback

[1515] The server collects system usage results and user feedback. The inputs required are system usage results and user feedback. The output is the collected feedback data. Specifically, the server collects feedback through the user interface and stores it in a database.

[1516] Step 7: System Improvement

[1517] The server improves the system based on the collected feedback. The input requires the collected feedback data. The output is an improved system. Specifically, it makes adjustments to improve the performance and accuracy of the generative AI model, thereby improving the overall system.

[1518] (Application example 1)

[1519] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1520] Today's children frequently use social networking sites, increasing their risk of being exposed to inappropriate content and dangerous interactions. However, it is difficult for parents and administrators to monitor these risks in real time and respond appropriately. Conventional monitoring systems have low accuracy in detecting dangerous content and often delay notification to parents and administrators. Therefore, there is a need for an effective monitoring system to ensure children's safety.

[1521] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1522] In this invention, the server includes means for monitoring children's use of SNS, means for using a generative AI model to detect dangerous text, means for monitoring SNS interactions using the API of the SNS platform, means for notifying parents or administrators in real time when danger is detected, means for improving the system based on feedback, means for generating and providing regular reports to parents or administrators, and means for inputting prompt sentences into the generative AI model to analyze the content of the text, thereby making it possible to effectively monitor children's use of SNS and quickly detect dangerous content and notify parents or administrators.

[1523] "Means for monitoring children's use of social media" refers to technologies for tracking activities on social media platforms used by children and obtaining the content of posts and messages.

[1524] "Means using generative AI models to detect dangerous text" refers to technology that uses generative AI models to analyze text on social media and automatically detect inappropriate or dangerous content.

[1525] "Methods of monitoring social media interactions using the API of a social media platform" refers to technology that utilizes the API provided by a social media platform to obtain and monitor user posts and messages.

[1526] "Means for notifying parents or administrators in real time when danger is detected" refers to technology that sends immediate notifications to parents or administrators when dangerous content is detected.

[1527] "Means for improving the system based on feedback" refers to techniques for collecting feedback from users and improving the performance and accuracy of the system based on that feedback.

[1528] The "means for generating periodic reports and providing them to parents and administrators" refers to a technology for generating reports summarizing the monitoring results at regular intervals and providing them to parents and administrators.

[1529] "Means for inputting prompt sentences into a generative AI model to analyze the content of text" refers to a technology in which a specific prompt sentence is input into a generative AI model and the resulting analysis results are used to evaluate the content of the text.

[1530] As an embodiment of the present invention, the following system can be constructed.

[1531] First, to monitor a child's use of social media, the server uses the social media platform's API to retrieve the child's posts and messages. Specifically, the server accesses the API endpoint provided by the social media platform and uses authentication information to retrieve data related to the child's account. This data includes posted text and received messages.

[1532] The server then inputs the acquired text data into a generative AI model to detect dangerous text. The generative AI model uses natural language processing techniques to analyze the content of the text and automatically detect inappropriate or dangerous content. This analysis can use natural language processing libraries such as Hugging Face's transformers library.

[1533] Examples of prompts to input to a generative AI model include:

[1534] "Analyze the following message for potential risks: 'I hate you. Don't talk to me anymore.'"

[1535] If dangerous text is detected, the server sends real-time notifications to parents and administrators via email, SMS, or push notification, allowing them to take immediate action.

[1536] In addition, the server periodically generates a report summarizing the monitoring results and provides it to parents or administrators. This report includes a list of detected dangerous text and analysis results, making it easier for parents and administrators to understand their children's social media usage.

[1537] Finally, the server collects user feedback and uses it to improve the system's performance and accuracy. Feedback is based on ratings and comments provided by parents and administrators. This allows the system to continually improve and become more effective at monitoring.

[1538] In this way, a system can be realized that can effectively monitor children's use of social networking sites, quickly detect dangerous content, and notify parents or administrators.

[1539] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1540] Step 1:

[1541] The server retrieves the child's posts and messages using the SNS platform's API. The SNS platform's API endpoint and authentication information are used as input. The retrieved text data (posts and messages) is obtained as output. Specifically, the server sends an HTTP request to the API endpoint and receives the text data as a response.

[1542] Step 2:

[1543] The server inputs the acquired text data into the generative AI model to detect dangerous text. The acquired text data and prompt text are used as input. The output is the analysis result of the text (risk assessment). Specifically, the prompt text is input into the generative AI model, and the model analyzes the content of the text and assesses the risk level.

[1544] Step 3:

[1545] The server sends real-time notifications to parents and administrators when dangerous text is detected. It uses the analysis results and notification destination information as inputs. It outputs the notification. Specific behaviors include sending a notification to the parent or administrator via email, SMS, or push notification.

[1546] Step 4:

[1547] The server periodically generates a report summarizing the monitoring results and provides it to the parent or administrator. The monitoring result data is used as input. As output, a report is generated and provided to the parent or administrator. Specifically, the monitoring results are compiled, compiled into a report format, and sent via email or other means.

[1548] Step 5:

[1549] The server collects user feedback and uses it to improve the system's performance and accuracy. It uses feedback from parents and administrators as input. It outputs improvements to the system. Specifically, it stores the feedback in a database, analyzes it, and adjusts the system's algorithms and settings.

[1550] Example 2

[1551] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1552] Modern social networking sites (SNS) pose a growing risk of children posting dangerous texts. These dangerous texts may contain violent, discriminatory, or threatening content, and may pose serious danger to children themselves or others. Conventional monitoring systems face the challenge of detecting these dangerous texts in real time and taking appropriate action.

[1553] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for monitoring children's use of the information exchange service, means for using a generative AI model for detecting dangerous text, means for monitoring interactions on the information exchange service using an application program interface of the information exchange service platform, means for notifying parents or administrators in real time when a risk is detected, means for improving the system based on the feedback, means for acquiring text entered by a user, means for inputting the acquired text into the generative AI model, means for the generative AI model to analyze the text and determine whether the text is dangerous, and means for notifying the user of the analysis result. This makes it possible to detect dangerous text before children post it and take appropriate measures.

[1554] "Children" refers to minors, particularly those who require supervision when using information exchange services.

[1555] "Information Exchange Service" refers to an online platform for users to post and share messages and content.

[1556] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to analyze text and understand and judge its content.

[1557] "Application Program Interface" refers to an interface for exchanging data and functions between different software systems.

[1558] "Real-time notification" refers to a function that notifies you immediately when a specific event occurs.

[1559] "Feedback" refers to evaluations and opinions provided based on system performance and results.

[1560] "Text" refers to written data posted by users to information exchange services.

[1561] "Analysis" refers to the process by which a generative AI model understands and evaluates the content of a text.

[1562] "Notification" refers to a message used to communicate analysis results to users and administrators.

[1563] MODE FOR CARRYING OUT THE INVENTION

[1564] This invention is a system that detects dangerous text in real time when a child uses a social networking service (SNS) and takes appropriate action. A specific embodiment of this system will be described below.

[1565] Hardware and software used

[1566] The system uses the following hardware and software:

[1567] Server: A central processing unit for receiving, analyzing, and notifying text data.

[1568] Device: A device (smartphone, computer, etc.) that allows a user to enter text.

[1569] Generative AI model: An artificial intelligence model that uses natural language processing techniques to analyze text (e.g., ChatGPT).

[1570] Data processing and calculation

[1571] 1. Get the text:

[1572] A user uses a terminal to input text to be posted to the information exchange service, and the terminal transmits this text to the server.

[1573] 2. Text Analysis:

[1574] The server inputs the received text into a generative AI model, which uses natural language processing techniques to analyze the content of the text and determine whether it contains dangerous content.

[1575] 3. Notification of analysis results:

[1576] The server receives the analysis results from the generative AI model and notifies the user based on the results via email, short message service, or push notification.

[1577] Specific examples

[1578] For example, suppose a user is trying to post a message on social media that reads, "I'm going to plant a bomb at school tomorrow." The server retrieves this text and inputs it into a generative AI model.

[1579] Example prompt sentence:

[1580] Please determine whether the following text contains dangerous content:

[1581] "Tomorrow I'm going to plant a bomb in the school."

[1582] The generative AI model analyzes this prompt and determines that the text contains the dangerous content of "planting a bomb." As a result, the server determines that the text is dangerous and issues a warning to the user.

[1583] Example of a warning message:

[1584] WARNING: The text you entered contains potentially dangerous content. Please stop posting.

[1585] In this way, the server can use the generative AI model to determine the risk of the text provided by the user and take appropriate action.This system can detect dangerous texts before children post them and notify parents or administrators in real time.

[1586] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1587] Step 1:

[1588] The user enters text

[1589] A user uses a terminal (such as a smartphone or PC) to input text to be posted to an information exchange service. The input text is temporarily stored in the terminal's memory. The input data is the text entered by the user, and the output data is the text stored in the terminal.

[1590] Step 2:

[1591] The device sends the text to the server

[1592] The terminal sends the text entered by the user to the server, where the text data is securely transmitted over the Internet. The input data is the text stored on the terminal, and the output data is the text sent to the server.

[1593] Step 3:

[1594] The server receives the text

[1595] The server receives the text sent from the terminal. The received text is temporarily stored for the next processing step. The input data is the text sent to the server, and the output data is the text stored on the server.

[1596] Step 4:

[1597] The server inputs the text into the generative AI model

[1598] The server inputs the received text into the generative AI model. At this time, it generates a prompt sentence and sends it to the generative AI model along with the text. The input data is the text stored on the server, and the output data is the prompt sentence and text sent to the generative AI model.

[1599] Specific prompt examples:

[1600] Please determine whether the following text contains dangerous content:

[1601] "Tomorrow I'm going to plant a bomb in the school."

[1602] Step 5:

[1603] Generative AI models parse text

[1604] The generative AI model analyzes the prompts and text sent from the server. It uses natural language processing technology to understand the context and keywords of the text and evaluate whether it contains dangerous content. The input data are the prompts and text sent to the generative AI model, and the output data are the analysis results.

[1605] Step 6:

[1606] The server receives the analysis results

[1607] The server receives the analysis results from the generative AI model, which include a judgment of whether the text is safe or unsafe. The input data is the analysis results sent from the generative AI model, and the output data is the analysis results stored on the server.

[1608] Step 7:

[1609] The server notifies the user of the analysis results

[1610] The server notifies the user based on the analysis results. For example, if the text is deemed dangerous, it displays a warning message to the user. The notification can be via email, a short message service, or a push notification. The input data is the analysis results stored on the server, and the output data is the notification message sent to the user.

[1611] Specific warning message example:

[1612] WARNING: The text you entered contains potentially dangerous content. Please stop posting.

[1613] (Application example 2)

[1614] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1615] Modern information and communication services are increasing the risk that children will receive dangerous texts and messages. In particular, phishing emails and messages containing inappropriate content are becoming a problem. This not only threatens children's safety, but also makes it difficult for parents and administrators to respond appropriately. Conventional systems have had difficulty detecting dangerous texts in real time and suggesting appropriate countermeasures.

[1616] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1617] In this invention, the server includes means for monitoring a child's use of information and communication services, means for using a generative AI model to detect dangerous text, means for monitoring interactions with information and communication services using an application programming interface of an information and communication service platform, means for notifying a parent or administrator in real time when a danger is detected, means for improving the system based on the feedback, means for analyzing the text of messages and emails received by a user in real time to determine whether the messages or emails contain dangerous content, and means for displaying a warning to the user and suggesting an appropriate response method when dangerous content is detected. This allows a child to receive an immediate warning when they receive a dangerous text, enabling parents or administrators to respond quickly.

[1618] "Information and communication services" are means of communication provided via the Internet, such as messaging, email, and social networking.

[1619] A "generative AI model" is an artificial intelligence model that uses natural language processing techniques to analyze text and generate output tailored to a specific task.

[1620] An "application programming interface" is a standardized interface that allows different software systems to communicate with each other.

[1621] "Real-time notification" is a function that immediately notifies users of information when a specific event occurs.

[1622] "Feedback" is information for evaluation and improvement based on the performance and results of a system.

[1623] A "message" is information such as text, images, and audio sent and received between users.

[1624] "Email" is a digital form of letter sent and received over the Internet.

[1625] A "warning" is a notification that notifies the user that a danger or problem has occurred.

[1626] A "solution" is an action or measure taken in response to a particular problem or situation.

[1627] The following system configuration will be described as an embodiment of the present invention.

[1628] System Configuration

[1629] The system includes a server, a user terminal, and a generative AI model. The server receives the text of messages and emails sent from the user terminal and analyzes the text using the generative AI model. If the text contains dangerous content based on the analysis results, the server notifies the user in real time.

[1630] Hardware and software used

[1631] Hardware: Smartphones, servers

[1632] Software: Python, OpenAI API, smtplib (Python standard library)

[1633] Data processing and calculation

[1634] 1. Text acquisition: The user terminal sends the text of the received message or email to the server.

[1635] 2. Text analysis: The server inputs the received text into a generative AI model and analyzes the text using natural language processing techniques.

[1636] 3. Risk determination: The generative AI model determines whether the text contains risky content.

[1637] 4. Warning Notification: If dangerous content is detected, the server will notify the user in real time via email, short message service, or push notification.

[1638] Specific examples

[1639] For example, if a user receives a phishing email, the content of the email will be analyzed, and if it is deemed dangerous, a warning will be displayed. Specific examples of prompt text are as follows:

[1640] Prompt Sentence Examples

[1641] Determine if the following text is dangerous: "Your account has been hacked. Click this link immediately to change your password."

[1642] In this way, users can be immediately alerted when they receive a dangerous text, allowing parents or administrators to respond quickly.

[1643] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1644] Step 1:

[1645] Obtain the text of messages and emails received by the user terminal.

[1646] Input: Text of a received message or email

[1647] Output: The retrieved text data

[1648] Specific operation: The user device scans the inbox for new messages and emails and extracts their contents in text format.

[1649] Step 2:

[1650] The user terminal transmits the acquired text data to the server.

[1651] Input: Obtained text data

[1652] Output: Text data sent to the server

[1653] Specific operation: The user terminal sends text data to the server using a communication method such as an HTTP request.

[1654] Step 3:

[1655] The server inputs the received text data into the generative AI model.

[1656] Input: Text data sent to the server

[1657] Output: Text data fed into the generative AI model

[1658] Specific operation: The server converts the received text data into an appropriate format for sending to the generative AI model's API.

[1659] Step 4:

[1660] A generative AI model analyzes text data to determine whether it contains dangerous content.

[1661] Input: Text data fed into the generative AI model

[1662] Output: Risk assessment result

[1663] How it works: The generative AI model uses natural language processing techniques to analyze text data and determine whether it contains dangerous content, such as phishing scams or inappropriate content.

[1664] Step 5:

[1665] The server receives the judgment results of the generated AI model and, if dangerous content is detected, notifies the user of a warning.

[1666] Input: Risk assessment result

[1667] Output: A warning notification sent to the user

[1668] Specific operation: The server analyzes the judgment results from the generative AI model and, if they contain dangerous content, sends a warning to the user via email, short message service, or push notification.

[1669] Step 6:

[1670] The user receives a warning notification and confirms the appropriate action to be taken.

[1671] Input: The warning notification sent to the user

[1672] Output: User's response behavior

[1673] Specific actions: The user checks the warning notification they received and takes appropriate action according to the instructions in the notification, such as not clicking on links in phishing emails or changing their passwords.

[1674] Example 3

[1675] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1676] The use of modern information and communication networks, especially among children, poses a growing risk of sending and receiving dangerous text messages. Because these dangerous text messages can lead to bullying, threats, and other harmful behavior, parents and administrators must monitor them in real time and take appropriate action. However, current systems lack the means to effectively detect and promptly notify users of dangerous text messages. Furthermore, their ability to improve the system based on feedback is limited. This makes it difficult to prevent dangerous situations before they occur.

[1677] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1678] In this invention, the server includes means for monitoring a child's use of the information and communications network, means for using a generative AI model to detect dangerous text, means for monitoring interactions on the information and communications network using an application programming interface of the information and communications network platform, means for notifying a parent or administrator in real time when a danger is detected, means for improving the system based on the feedback, means for receiving text messages sent by the user from the terminal, means for analyzing the text messages using the generative AI model, means for detecting dangerous text based on the analysis results, and means for updating the generative AI model based on feedback from the parent or administrator. This makes it possible to detect dangerous text messages in real time and quickly notify parents or administrators. Furthermore, by continuously improving the system based on feedback, dangerous text can be detected with greater accuracy.

[1679] "Means for monitoring children's use of information and communications networks" refers to technologies and devices for monitoring activities on information and communications networks used by children and detecting abnormal or dangerous behavior.

[1680] "Means using generative AI models to detect dangerous text" refers to technologies or devices that use generative AI models to analyze the content of text messages and assess their risk.

[1681] "Means for monitoring communications on an information and communications network using the application programming interface of an information and communications network platform" refers to technology or devices for monitoring messages and data communications on the network using the API provided by the information and communications network platform.

[1682] "Means for notifying parents or administrators in real time when danger is detected" refers to technology or devices for sending immediate notifications to parents or administrators when dangerous text is detected.

[1683] "Means for improving the system based on feedback" refers to technologies or devices that improve the performance or accuracy of the system based on feedback from parents or administrators or automatically collected data.

[1684] The "means for receiving a text message sent from a terminal by a user" refers to a technique or device that allows a server to receive a text message sent from a terminal used by a user.

[1685] "Means for analyzing text messages using a generative AI model" refers to technology or devices that use a generative AI model to analyze the content of received text messages and understand their meaning and intent.

[1686] "Means for detecting dangerous text based on analysis results" refers to technologies or devices for determining whether a text message is dangerous based on the analysis results of a generative AI model.

[1687] "Means for updating the generative AI model based on feedback from parents or administrators" refers to technologies or devices that update the learning data of the generative AI model based on feedback information provided by parents or administrators, thereby improving the accuracy of the model.

[1688] MODE FOR CARRYING OUT THE INVENTION

[1689] The present invention is a system for monitoring children's use of information and communication networks and detecting dangerous text messages. A specific embodiment of this system will be described below.

[1690] System configuration

[1691] This system consists of three main components: a server, a device, and a user. The server analyzes text messages using a generative AI model and detects dangerous text. The device is the device used by the user to send text messages, such as a smartphone or PC. The user is the person who actually sends the text messages.

[1692] Hardware and software used

[1693] The server is a computer system equipped with a high-performance processor and a large amount of memory. Specifically, a server equipped with an Intel Xeon processor or an AMD EPYC processor is suitable. The software running on the server includes the Python libraries spaCy and Transformers. These libraries are used to analyze text messages using natural language processing techniques.

[1694] A terminal is a device that allows a user to input and send text messages. This can be a smartphone or a computer, and these devices must be connected to the Internet.

[1695] System Operation

[1696] When a user sends a text message from their device, the device sends the message to a server. The server inputs the received text message into a generative AI model for analysis. The generative AI model evaluates the content of the text based on pre-trained data and determines whether it is dangerous.

[1697] For example, if a user sends a message saying, "I'm waiting for you at school. You can't escape," the server receives this message and inputs it into the generative AI model, which analyzes it and determines that it contains violent content.

[1698] Notifications and Feedback

[1699] If dangerous text is detected, the server sends a real-time notification to parents or administrators via email, short message service (SMS), or push notification, including information such as the content of the dangerous text, the time it occurred, and its location.

[1700] Parents or administrators can provide feedback based on the received notification, for example, "This text is not dangerous," and the server will use this feedback to update the generative AI model and reflect it in future decisions.

[1701] Specific examples

[1702] As a concrete example, the following prompt sentence is input to the generative AI model:

[1703] Example prompt sentence:

[1704] "It analyzes text messages sent by users to determine if they contain dangerous content, such as messages like: 'I'm waiting for you at school. You can't escape.'"

[1705] In this way, the system can detect dangerous text messages in real time and quickly notify parents or administrators. Furthermore, by continuously improving the system based on feedback, it is possible to detect dangerous text with higher accuracy. The flow of the identification process in Example 3 will be described with reference to FIG. 15.

[1706] Step 1:

[1707] A user sends a text message.

[1708] A user uses a device such as a smartphone or PC to input a text message and press the send button. The input is the text message entered by the user. The output is a command to send the message to the device.

[1709] Step 2:

[1710] The device sends a text message to the server.

[1711] The terminal sends the text message entered by the user to a server over the Internet, encrypting the data using the HTTPS protocol. The input is the text message entered by the user. The output is the text message sent to the server.

[1712] Step 3:

[1713] The server receives the text message.

[1714] The server receives text messages sent from the device. The received data is temporarily stored in a database. The input is the text message sent from the device. The output is the text message stored in the database.

[1715] Step 4:

[1716] The server analyzes the text message using the generative AI model.

[1717] The server uses the Python libraries spaCy and Transformers to analyze text messages. The generative AI model evaluates the content of the text based on pre-trained data. The input is the text message stored in the database. The output is the analysis result.

[1718] Step 5:

[1719] The server detects dangerous text.

[1720] The server determines whether the text is dangerous based on the analysis results of the generative AI model. For example, if the text contains "violent language" or "threatening content," it is deemed dangerous. The input is the analysis results of the generative AI model. The output is the detection result of dangerous text.

[1721] Step 6:

[1722] The server sends a notification to the parent or administrator.

[1723] The server sends real-time notifications to parents and administrators when dangerous text is detected. Notifications can be sent via email, SMS, or push notification. The notification includes information such as the content of the dangerous text, the time it occurred, and its location. The input is the detection result of the dangerous text. The output is the notification sent to the parent or administrator.

[1724] Step 7:

[1725] Parents and administrators provide feedback.

[1726] The parent / administrator provides feedback based on the received notification, for example, "This text is not dangerous" and sends the feedback to the server. The input is the feedback from the parent / administrator. The output is the feedback sent to the server.

[1727] Step 8:

[1728] The server updates the generative AI model based on the feedback.

[1729] The server updates the generative AI model based on feedback from parents and administrators. Specifically, it uses the feedback as training data to improve the accuracy of the model. The input is the feedback from parents and administrators. The output is the updated generative AI model.

[1730] (Application example 3)

[1731] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1732] Today's children frequently use online platforms and social media, increasing their risk of coming into contact with dangerous text or inappropriate content. However, it is difficult for parents and administrators to monitor these dangers in real time and respond appropriately. Furthermore, existing systems have low accuracy in detecting dangerous text, resulting in false positives and missed results. Furthermore, there is a lack of a mechanism for utilizing feedback to improve the system's accuracy, making continuous improvement difficult.

[1733] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1734] In this invention, the server includes means for monitoring a child's online activity, means for using a generative AI model to detect dangerous texts, means for monitoring communications using an online platform's API, means for notifying parents or administrators in real time when a risk is detected, means for improving the system based on feedback, means for sending the notification by email, SMS, or push notification, and means for collecting and learning from feedback to improve the accuracy of the AI ​​model. This makes it possible to detect dangerous texts in real time and quickly notify parents or administrators. The accuracy of the system can also be continuously improved by utilizing feedback.

[1735] "Children's online activities" refers to all of a child's actions and communications that occur using the Internet.

[1736] A "generative AI model" is an artificial intelligence model that has been trained using machine learning techniques and is generated to perform a specific task.

[1737] An "online platform" is a general term for services and applications provided over the Internet, and is a place where users can share information and communicate.

[1738] "API" stands for Application Programming Interface, an interface for exchanging data and functions between different software systems.

[1739] "Communication" refers to the act or process of sending or receiving data or information.

[1740] "Real-time notification" is a function that sends a notification immediately the moment a specific event occurs.

[1741] "Feedback" refers to the act of providing evaluations and opinions on the performance and results of a system, and refers to information that is useful for improving the system.

[1742] "Email" is a form of digital message sent and received over the Internet.

[1743] "SMS" stands for Short Message Service, a service that allows you to send and receive short text messages over a mobile phone network.

[1744] "Push notifications" are notifications sent directly to a user's device from a mobile application or web service.

[1745] "Accuracy" refers to the ability of a system or model to perform a task accurately.

[1746] "Learning" is the process by which a machine learning model finds patterns and rules based on data and improves its performance.

[1747] A system for implementing this invention uses a generative AI model to monitor children's online activity and detect dangerous texts. The system uses the API of an online platform to monitor communications, notifies parents or administrators in real time when a risk is detected, and has the ability to improve the system based on feedback.

[1748] Hardware and software used

[1749] Hardware: Smartphones, servers

[1750] Software: Python, smtplib (sending email), twilio (sending SMS), requests (sending push notifications)

[1751] Data processing and calculation

[1752] 1. Loading the generative AI model: The server loads a generative AI model for detecting dangerous text. This model analyzes text using natural language processing techniques.

[1753] 2. Text evaluation: The server evaluates the text data obtained through the online platform's API using a generative AI model to determine whether the text is dangerous.

[1754] 3. Sending notifications: If dangerous text is detected, the server will send real-time notifications to parents or administrators via email, SMS, or push notification.

[1755] 4. Feedback collection and learning: Feedback is collected from parents and administrators, and the server uses this information to update the generative AI model, thereby improving the accuracy of the system.

[1756] Specific examples

[1757] For example, if a child receives a dangerous message on their smartphone, the content of the message will be notified to the parent in real time. If the parent gives feedback that the text is not dangerous, the server will learn from that information and will not judge similar texts as dangerous in the future.

[1758] Prompt Sentence Examples

[1759] Develop a system to notify parents and administrators of dangerous texts in real time via email, SMS, or push notification, and implement a feedback mechanism to improve the accuracy of the AI ​​model.

[1760] The above is an embodiment of the present invention.

[1761] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1762] Step 1:

[1763] The server loads the generative AI model.

[1764] Input: File path of the generated AI model

[1765] Data processing: Read the model file and expand it in memory

[1766] Output: The loaded generative AI model

[1767] Specific operation: The server loads the generative AI model from the specified file path and extracts it in memory. This model is used to analyze text using natural language processing techniques.

[1768] Step 2:

[1769] The server obtains the text data through the API of the online platform.

[1770] Input: API endpoint, authentication information

[1771] Data processing: Send an API request and retrieve text data

[1772] Output: The retrieved text data

[1773] What it does: The server sends a request to the online platform's API endpoint using the authentication information to retrieve text data about the child's online activities.

[1774] ...

Claims

1. A means for monitoring the SNS usage of a user's child; A method for detecting dangerous text using an AI model; A means for monitoring SNS interactions using an API of an SNS platform; A means for notifying the user or an administrator of the SNS in real time when a danger is detected; a means of improving the system based on feedback; and means for inferring an emotional state of the user's child from the text using an emotion engine; A means to detect posts or messages containing specific keywords; means for sending a notification based on the detected emotional state or keyword; A system including:

2. 10. The system of claim 1, wherein the AI ​​model analyzes text using natural language processing techniques.

3. The system of claim 1 , wherein the notification is via email, SMS, or push notification.

Images