system

Abstract

We provide the system. [Solution] A means for parents to upload videos or diary photos of their children, An AI-based analysis method for analyzing uploaded videos or photos, The analysis revealed a means to determine the characteristics of a child's development, Means of providing analysis results to parents, A means to clearly show the reasoning behind the decision based on the analysis, A method for compiling the data necessary for the consultation in chronological order, A system that includes this.

Description

Technical Field

[0005]

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to the description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present invention relates to a system for realizing early detection and appropriate support for children's developmental disorders. In particular, it aims to provide an opportunity for quick consultation when a guardian has doubts and to provide data for smoothly promoting consultation with medical institutions and administrative bodies. Current problems include that it is difficult for guardians to notice the symptoms of developmental disorders, it is difficult to make reservations at specialized institutions, and the appropriate consultation destinations are unknown.

Means for Solving the Problems

[0006] A "guardian" is a parent or a person in a similar position who is responsible for raising a child.

[0007] "Children" refers to minors who are the subject of developmental analysis.

[0008] "Video" refers to video data recorded by parents or guardians that document the actions and behaviors of their children.

[0009] "Diary photos" refer to data that records a child's daily activities and behaviors as photographs, documented by a parent or childcare facility.

[0010] "Uploading" refers to the act of a parent or guardian sending videos or diary photos to the system for storage.

[0011] "Analysis" refers to the act of using AI technology to analyze uploaded videos and photos and identify characteristics of developmental disorders.

[0012] "AI" refers to artificial intelligence technology, and in this context, it specifically includes methods for analyzing videos and photos and identifying their characteristics.

[0013] "Analysis results" refer to the output of AI analysis, which indicates whether or not characteristics of developmental disorders are present.

[0014] "Characteristics" refers to features such as behaviors, movements, expressions, handwriting, etc. related to children's developmental disorders.

[0015] "Reasons for judgment" is information that explains the criteria and basis used by AI when deriving analysis results.

[0016] "Organize in chronological order" refers to organizing and displaying or storing the uploaded and analysis results in chronological order.

[0017] "Medical institution" refers to facilities or organizations that provide medical services such as hospitals, clinics, specialized clinics, etc.

[0018] "Administration" refers to local governments such as municipalities, prefectures, etc. and related agencies, including organizations that provide services related to welfare and education support.

Brief Explanation of Drawings

[0019] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9]Shows an emotion map to which a plurality of emotions are mapped. [Figure 10] Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0020] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be described.

[0022] In the following embodiments, a processor with a reference number (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0023] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0024] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0025] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0027] [First Embodiment]

[0028] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0029] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0032] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0035] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0039] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0040] One embodiment of the present invention is the provision of a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos and photos from their child's diary, and based on the results of AI analysis, it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0041] User registration and login

[0042] Users first access the system and register by entering information such as username, password, and email address. Existing users log in using their username and password. The server receives this information and compares it with existing user information in the database. In the case of new registration, it verifies that the entered information does not exist in the database and adds the new user information to the database. In the case of login, it verifies that the entered username and password are correct and authentication is successful.

[0043] Uploading videos and diaries

[0044] After a user logs in, they upload videos of their child or photos from their diary. When uploading, the user selects a video file or a diary photo and presses the upload button. The server receives the file and saves it to a temporary storage location. Furthermore, it verifies the file type (video or photo) and saves it to the appropriate folder. During saving, the server logs the file's save date, time, and other metadata.

[0045] AI analysis

[0046] The server sends saved videos and photos to an AI analysis module. In the case of videos, the server uses AI to extract features such as motion and gaze direction. In the case of diary photos, it analyzes facial expressions and handwriting. The AI ​​analysis module analyzes these features to determine whether characteristics of a developmental disorder are present. The server receives these analysis results and stores them in a database.

[0047] Providing analysis results and reasons for judgment

[0048] If a user wants to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. Specifically, it highlights key points such as particular actions, facial expressions, and handwriting, and explains which characteristics were indicators of developmental disorders.

[0049] Data compilation and consultation preparation

[0050] The server compiles user basic information, upload history, and analysis results in chronological order. This allows users to consult with medical institutions and government agencies more smoothly. The server generates the compiled data in formats such as PDF and provides it to the user. Users can download this data and use it for consultations.

[0051] Specific example

[0052] This example shows a user registering with the system and uploading a video of their child. First, the parent accesses the system and registers. Then, they log in and upload a video file of their child's daily activities. The server receives this video and begins AI analysis. After a few minutes, the analysis results are available, and if certain movements or gaze patterns are considered indicators of developmental disorders, detailed analysis results are provided along with the reasons why. Based on these results, the user considers the next steps and, if necessary, downloads data to prepare for consultations with medical institutions or government agencies.

[0053] This series of steps allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures.

[0054] The following describes the processing flow.

[0055] Step 1:

[0056] Users access the system and either register or log in. For new registrations, they enter a username, password, and email address. For logins, they enter their already registered username and password.

[0057] Step 2:

[0058] The server receives the entered information and checks if the username exists in the existing database. For new registrations, it verifies that the username does not exist and then adds the registration information to the database. For logins, it verifies that the username and password match and performs authentication.

[0059] Step 3:

[0060] After logging in, the user uploads video files or photos from their diary. They select the file to upload and press the upload button.

[0061] Step 4:

[0062] The server receives the uploaded file and checks its type (video or photo). It saves the file to a temporary storage location. Then, it moves the file to the appropriate folder and logs metadata including the save date and time and user information.

[0063] Step 5:

[0064] The server sends stored videos and photos to an AI analysis module. For videos, features such as motion and gaze direction are extracted. For photos, facial expressions and handwriting are analyzed.

[0065] Step 6:

[0066] The server receives the analysis results from the AI ​​analysis module and determines whether characteristics of developmental disorders are present. These analysis results are then stored in a database. The analysis results include the date and time of analysis, information about the uploaded media, the features identified by the AI, and an overall assessment.

[0067] Step 7:

[0068] The user accesses the results page to view the analysis results. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The detailed explanation of the results clearly indicates specific points such as particular actions, facial expressions, and handwriting.

[0069] Step 8:

[0070] The server compiles the user's basic information, upload history, and analysis results in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides the user with a download link.

[0071] (Example 1)

[0072] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0073] Modern parents are seeking ways to detect potential developmental disorders in their children early and take appropriate measures. However, current systems and methods make it difficult for parents to easily check their child's developmental characteristics and consult with medical or administrative agencies based on accurate data. In particular, there is a lack of systems that can efficiently analyze various media files such as videos and photos and provide reliable results. Furthermore, clearly explaining the reasoning behind the judgments based on the analysis results and organizing the data necessary for consultations in chronological order are also important issues.

[0074] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0075] In this invention, the server includes means for parents to upload media files of their children, means for analyzing the uploaded media files using artificial intelligence, means for determining the characteristics of the child's developmental disorder based on the analysis results, means for providing the analysis results to the parents, means for clearly showing the reasons for the judgment based on the analysis, means for compiling the data necessary for consultation in chronological order, and means for users to access the system by registering and logging in. This makes it possible for parents to easily check for the possibility of their child's developmental disorder and take quick and appropriate measures based on reliable data.

[0076] "Guardian" refers to an adult who has the responsibility for raising and supervising a child.

[0077] "Media files" refer to digital data that includes visual information, such as video files and image files.

[0078] "Means of uploading" refers to the interface or function that allows users to transfer files from their devices to a server.

[0079] "Analytical artificial intelligence" refers to algorithms and software that automatically extract specific patterns and features from input data and make decisions based on the results.

[0080] "Analysis means" refers to a function that uses artificial intelligence to process uploaded media files and extract and analyze specific features.

[0081] "Means of judgment" refers to algorithms or logic used to derive specific conclusions based on analysis results.

[0082] "Means of providing" refers to functions for displaying analysis results and other information to users or distributing them in a downloadable format.

[0083] "Means of clearly demonstrating the reasoning behind the decision" refers to functions that explain the basis and causes behind the analysis results in a way that is easy for the user to understand.

[0084] "Methods for organizing data chronologically" refers to functions that organize and display data such as analysis results and upload history in chronological order.

[0085] "Method of registering and logging in" refers to the process by which a user registers their information in the system and obtains access rights after authentication.

[0086] One embodiment of the present invention is the provision of a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos and photos from their child's diary, and based on the analysis results by artificial intelligence (AI), it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0087] User registration and login

[0088] Users first access the system and register by entering information such as their username, password, and email address. They then log in using this registered information. The server stores this information in a database and authenticates the user. For new registrations, the server verifies that the entered information does not already exist in the database and adds the new user information. For existing users, the server authenticates their username and password to ensure successful login.

[0089] Uploading videos and diaries

[0090] After logging in, users upload videos of their children or photos from their diaries. This involves selecting a video file or diary photo and pressing the upload button. The server receives the file and saves it to a temporary storage location. It then stores it in the appropriate folder based on the file type and records metadata at the time of saving.

[0091] AI analysis

[0092] The server sends the saved videos and photos to the AI ​​analysis module. For videos, it extracts features such as movement and gaze direction; for photos, it analyzes facial expressions and handwriting. This analysis is performed using a generative AI model. The AI ​​analysis module analyzes the acquired features to determine if characteristics of a developmental disorder are present. The analysis results are stored in a database by the server.

[0093] Providing analysis results and reasons for judgment

[0094] When a user wishes to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database and provides them in a visually easy-to-understand format. This includes a function that explains in detail why certain actions, facial expressions, or handwriting patterns are indicators of developmental disorders.

[0095] Data compilation and consultation preparation

[0096] The server compiles user basic information, upload history, and analysis results in chronological order, thereby supporting users in easily consulting with medical and administrative agencies. The compiled data is generated in formats such as PDF and provided to users in a downloadable format.

[0097] Examples of specific prompt messages include the following:

[0098] "Please analyze the video of the child and let me know if it shows any characteristics of a developmental disorder. This video shows a 5-year-old child playing."

[0099] "I've uploaded a photo of the diary. Please analyze the writing and drawings of this child and let me know the results."

[0100] This system allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures promptly.

[0101] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0102] Step 1:

[0103] Users access the system and register by entering their username, password, and email address on the registration screen. For example, they might enter "username," "password," and "email@example.com." The server receives this information and checks it against the database to ensure there are no duplicates in existing user information. Based on the results, the server saves the new user information to the database.

[0104] Input: Username, Password, Email Address

[0105] Output: User information database registration confirmation message

[0106] Step 2:

[0107] The user logs in by entering their existing username and password on the login screen. For example, they might enter "username" and "password". The server receives this information, accesses the database, and verifies that the username and password combination is correct. If authentication is successful, the user interface is displayed.

[0108] Input: Username, Password

[0109] Output: Login success or failure message

[0110] Step 3:

[0111] After logging in, the user clicks a button to upload videos or diary photos, opening a file selection dialog. For example, they select "child_video.mp4" or "diary_image.jpg" and click the upload button. The server receives the file and saves it to a temporary storage location. Next, it verifies the file type (video or photo) and moves it to the appropriate folder. In addition, it logs the file's save date and metadata.

[0112] Input: Selected file

[0113] Output: Message indicating file save complete.

[0114] Step 4:

[0115] The server sends stored videos and photos to an artificial intelligence (AI) analysis module. For example, it sends "child_video.mp4". The server extracts motion and gaze features from the video file and analyzes facial expressions and handwriting from the photos. It uses a generative AI model to recognize specific actions and patterns.

[0116] Input: Video or photo file

[0117] Output: Extracted features and analysis results

[0118] Step 5:

[0119] The server saves the analysis results received from the AI ​​analysis module to a database. For example, if "frequent eye movement" or "waving motion" is detected, the server saves the analysis results, including these features, to the database.

[0120] Input: AI analysis results

[0121] Output: Status of saving analysis results to the database

[0122] Step 6:

[0123] The user accesses a dedicated results page to view the analysis results. The server retrieves the analysis results for the user from the database and provides them in an easy-to-understand format. This includes a detailed explanation of any specific behaviors, facial expressions, or handwriting detected, and why they may be indicators of developmental disorders.

[0124] Input: User Request

[0125] Output: Formatted analysis results

[0126] Step 7:

[0127] The server compiles user basic information, upload history, and analysis results chronologically, generating them in file formats such as PDF. Users can download and use this data. For example, it can be used as necessary documentation when consulting with medical institutions or government agencies.

[0128] Input: User information, upload history, analysis results

[0129] Output: Time-series data summary PDF file

[0130] Through the above processing steps, the system helps parents check for potential developmental disorders in their children and take prompt action based on reliable data.

[0131] (Application Example 1)

[0132] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0133] Ensuring the safety of passengers in autonomous vehicles is crucial, and it is especially necessary to enable rapid response by identifying abnormal behavior or facial expressions in real time, particularly among minors and children. Furthermore, it is essential to provide an environment where passengers with developmental disabilities can receive appropriate support.

[0134] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0135] In this invention, the server includes means for parents to upload videos or diary photos of their children, means for analyzing the uploaded videos or photos using AI, means for determining the characteristics of the child's developmental disability based on the analysis results, means for providing the analysis results to the parents, means for clearly indicating the reasons for the judgment based on the analysis, means for compiling data necessary for consultation in chronological order, means for recording the movements and facial expressions of passengers with cameras inside the vehicle, means for analyzing the recorded data in real time to detect abnormal movements or facial expressions, and means for notifying the parents' mobile devices when an abnormality is detected. This enables real-time assurance of passenger safety in autonomous vehicles and allows for appropriate support, especially for passengers with characteristics of developmental disabilities.

[0136] "Guardian" refers to an adult who is responsible for raising and protecting a child or minor.

[0137] "Video" refers to digital data that expresses movement by displaying a series of still images at regular intervals.

[0138] "Diary photos" refers to digital image files that are photographs taken of the contents of a diary recorded on paper or digital media.

[0139] "Uploading" refers to the act of transferring data from a local device to a remote system such as a server.

[0140] "Analysis means" refers to methods and devices for extracting characteristics and patterns from data and converting them into meaningful information.

[0141] "AI" stands for Artificial Intelligence, and refers to machine learning algorithms and models that automatically process specific tasks.

[0142] "Developmental disorder" refers to a condition in which a person has certain characteristics or behavioral patterns that are observed during the developmental process, and experiences specific difficulties in learning, socializing, and behavior.

[0143] A "server" refers to a computer system used for storing, processing, and providing data.

[0144] A "camera" is a device that captures light and records still images and videos.

[0145] "Dynamics" is a concept that refers to the movement and changes of people and objects.

[0146] "Facial expression" refers to the expression of emotions and intentions through the movement of facial muscles.

[0147] "Real-time" refers to a state where data collection, processing, and feedback occur instantly in accordance with real-world time.

[0148] A "mobile device" is a portable electronic device, including smartphones and tablets.

[0149] A "notification" refers to a message or warning that informs someone of a specific event or condition.

[0150] A "system" refers to a collection of interconnected components that work together to achieve a specific purpose.

[0151] The system that implements this application consists of the following steps and is carried out using various hardware and software.

[0152] User registration and login

[0153] The user first accesses the system and registers as a parent or guardian. They enter necessary information such as username, password, and email address. Existing users log in with their username and password. The server receives this information, compares it with existing user information in the database, and adds new user information to the database if necessary. During login, the system verifies that the entered information is correct and performs authentication. This allows the user to access the system.

[0154] Uploading videos and diaries

[0155] After a user logs in, parents upload videos and diary photos of their children to the system. The user selects a video file or diary photo and presses the upload button. The server receives the file and saves it to a temporary storage location. At this time, it determines the file type (video or photo) and saves it to the appropriate folder. The save date and other metadata are also recorded.

[0156] AI analysis

[0157] The server sends saved videos and photos to an AI analysis module. In the case of videos, AI is used to extract features such as motion and gaze direction. In the case of diary photos, facial expressions and handwriting are analyzed. The AI ​​analysis module analyzes these features to determine whether or not there are characteristics of a developmental disorder. The analysis results are received by the server and stored in a database.

[0158] Providing analysis results and reasons for judgment

[0159] The user accesses a dedicated results page to view the analysis results. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. It clearly indicates specific points such as certain actions, facial expressions, and handwriting, and provides a detailed explanation of why they were indicators of developmental disorders.

[0160] Data compilation and consultation preparation

[0161] The server compiles user basic information, upload history, and analysis results in chronological order. This allows users to smoothly consult with medical institutions and government agencies. The server generates the compiled data in PDF format and other formats, making it available for users to download.

[0162] Applications in autonomous vehicles

[0163] Cameras are installed inside the vehicle to record the movements and expressions of passengers, especially children, in real time. The recorded data is analyzed in real time by an AI analysis module. If abnormal behavior or expressions are detected, the server immediately sends a notification to the parent's mobile device. This ensures the safety of passengers in autonomous vehicles.

[0164] Specific example

[0165] For example, a parent can register with the system and upload videos of their child's daily life. These videos are stored on a server and analyzed by AI. Parents are notified of any specific abnormal behaviors or gaze directions indicated by the analysis results. Also, when a child is in an autonomous vehicle, the AI ​​analyzes videos taken inside the vehicle, and if it detects abnormal behavior or facial expressions, a warning notification is sent to the parent's smartphone in real time.

[0166] Example of a prompt

[0167] "Please explain how to use video and image data captured by in-car cameras to analyze passengers' facial expressions and movements in real time, thereby supporting safe autonomous driving."

[0168] Thus, by using this system, parents can not only check for characteristics of developmental disorders in their children at an early stage, but also enhance safety inside autonomous vehicles.

[0169] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0170] Step 1:

[0171] A user accesses the system to register or log in. They enter information such as their username, password, and email address, and submit it to the server. The server verifies the input data against the database; in the case of new registration, it is saved as new user information. In the case of login, authentication is performed through verification. The input is the username and password, and the output is the authentication result.

[0172] Step 2:

[0173] After the user logs in, they upload a video of their child or a photo from their diary. The user selects a video file or a photo from their diary and presses the upload button. The server receives this file and saves it to a temporary storage location. It determines the file type and saves it to the appropriate folder. The input is the video file or photo from the diary, and the output is the path to the saved file.

[0174] Step 3:

[0175] The server sends saved videos and photos to an AI analysis module. For videos, it extracts motion and gaze direction frame by frame. For diary photos, it analyzes facial expressions and handwriting features. The analysis results are received by the server and stored in a database. The input is a saved video file or diary photo, and the output is the analysis results.

[0176] Step 4:

[0177] The server formats the analysis results into a dedicated results display page and provides them to the user. It provides detailed explanations of specific actions, facial expressions, handwriting, and other points, clearly indicating the reasons why they were identified as indicators of developmental disorders. Users can view the analysis results through a browser or app. The input is the analysis result data, and the output is the formatted results display page.

[0178] Step 5:

[0179] The server compiles user basic information, upload history, and analysis results in chronological order. This allows users to generate data that facilitates consultations with medical institutions and government agencies. The time-series data is generated in formats such as PDF, making it available for users to download. The input consists of user basic information and analysis results, and the output is a data file compiled in chronological order.

[0180] Step 6:

[0181] Cameras inside the autonomous vehicle record passengers' movements and facial expressions in real time. The recorded data is sent to an AI analysis module to detect abnormalities in movement or facial expressions. If an abnormality is detected, the server sends a notification to the parent's mobile device. The input is real-time recorded video data, and the output is an abnormality detection notification.

[0182] Step 7:

[0183] The parent's mobile device receives notifications sent from the server and displays the details of the anomaly. The user can review the notification and take necessary actions quickly. The input is the notification data from the server, and the output is the displayed notification content.

[0184] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0185] One embodiment of the present invention is the provision of a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos and photos from their child's diary, and based on the analysis results by AI and an emotion engine, it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0186] User registration and login

[0187] Users first access the system and register by entering information such as username, password, and email address. Existing users log in using their username and password. The server receives this information and compares it with existing user information in the database. In the case of new registration, it verifies that the entered information does not exist in the database and adds the new user information to the database. In the case of login, it verifies that the entered username and password are correct and authentication is successful.

[0188] Uploading videos and diaries

[0189] After logging in, users upload videos of their children or photos from their diaries. When uploading, users select a video file or a photo from their diary and press the upload button. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data at the time of upload.

[0190] Collection of emotional data

[0191] The server stores emotional data acquired during the upload process. The emotion engine recognizes emotions such as anxiety, relief, and impatience from the user's facial expressions and tone of voice. This data is also included in the analysis results.

[0192] AI analysis

[0193] The server sends saved videos and photos to an AI analysis module. In the case of videos, the server uses AI to extract features such as motion and gaze direction. In the case of diary photos, it analyzes facial expressions and handwriting. The AI ​​analysis module analyzes these features and determines whether characteristics of developmental disorders are present. The server receives these analysis results and stores them in a database. The analysis results include the date and time of analysis, information about the uploaded media, features found by the AI, and an overall judgment.

[0194] Providing analysis results and reasons for judgment

[0195] When a user wishes to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The analysis results clearly indicate specific points such as actions, facial expressions, and handwriting, and explain which characteristics were indicators of developmental disorders. The server also displays the user's emotional data recognized by the emotion engine, demonstrating consideration for the reliability of the analysis results and the user's feelings.

[0196] Data compilation and consultation preparation

[0197] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links.

[0198] Specific example

[0199] This example illustrates how a user registers with the system and uploads videos of their child. First, the parent accesses the system and registers. After logging in, they upload video files of their child's daily activities. During the upload, the emotion engine analyzes the parent's emotions and detects feelings such as anxiety and impatience. The server sends this video and emotion data to an AI analysis module, and the analysis results are provided after a few minutes. If specific behaviors or gaze patterns are identified as indicators of developmental disorders, detailed analysis results and the parent's emotion data are provided along with the reasons why. Based on these results, the user can consider the next steps and prepare to consult with medical institutions or government agencies if necessary.

[0200] This series of steps allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures. Furthermore, the parents' emotional state is taken into consideration, leading to more reliable decisions.

[0201] The following describes the processing flow.

[0202] Step 1:

[0203] Users access the system and either register or log in. For new registrations, they enter a username, password, and email address. For logins, they enter their already registered username and password.

[0204] Step 2:

[0205] The server receives the entered information and checks if the username exists in the existing database. For new registrations, it verifies that the username does not exist and then adds the registration information to the database. For logins, it verifies that the username and password match and performs authentication.

[0206] Step 3:

[0207] After logging in, the user selects an option to upload a video or diary photo from the system's main screen. During the upload process, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data.

[0208] Step 4:

[0209] The device sends the selected video file or diary photo to the server. At the same time, the emotion engine also sends the user's emotion data (e.g., anxiety, relief, impatience).

[0210] Step 5:

[0211] The server saves received videos, photos, and sentiment data to a temporary storage location. Metadata (e.g., save date and time, type of uploaded file) is logged for the saved files and sentiment data.

[0212] Step 6:

[0213] The server sends stored videos and photos to an AI analysis module. In the case of videos, the server extracts features such as movement and gaze direction. In the case of photos, it performs analysis of facial expressions and handwriting.

[0214] Step 7:

[0215] The server receives the analysis results from the AI ​​analysis module and determines whether characteristics of developmental disorders are present. The analysis results are saved to a database, recording the date and time of analysis, information about the uploaded media, the characteristics found by the AI, and the overall judgment.

[0216] Step 8:

[0217] The user accesses the results page to view the analysis results. The server retrieves the analysis results for that user from the database and formats them, including sentiment data, into an easy-to-read format.

[0218] Step 9:

[0219] The server provides the user with analysis results and emotional data. The results clearly indicate specific points such as actions, facial expressions, and handwriting, and explain which characteristics were indicators of developmental disorders. The display method of the analysis results is also adjusted based on the emotional data.

[0220] Step 10:

[0221] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links.

[0222] This allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures. Furthermore, the parents' emotional state is also taken into consideration, leading to more reliable assessments.

[0223] (Example 2)

[0224] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0225] In recent years, the early detection and support of developmental disorders in children have become increasingly important, but parents have limited means to regularly check for these conditions. Furthermore, parents' own emotional states can make accurate judgments difficult. In particular, feelings of anxiety and impatience can influence parents' judgments, hindering appropriate responses. To address this challenge, a system is needed that allows parents to easily check their child's condition and receive reliable results. Additionally, there is a need for a system that takes parents' emotional states into account to enable more objective and reliable judgments.

[0226] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0227] In this invention, the server includes means for parents to transfer images of their child or articles; means for analyzing the transferred images or videos using artificial intelligence; and means for detecting the parent's emotional state and including that data in the analysis. This makes it possible for parents to routinely check for characteristics of their child's developmental disorder and to obtain analysis results that also take into account the parent's emotional state.

[0228] A "guardian" is a person who has the responsibility to support the healthy growth and development of a child and who takes care of the child's daily life and education.

[0229] "Video" refers to moving image data that records a child's actions or situation. Examples include video clips and movies.

[0230] "Article images" refer to still image files of diaries, drawings, or other documents created by children.

[0231] "Transferring" refers to the act of sending images of videos or articles collected by a parent or guardian to the system.

[0232] Artificial intelligence refers to software programs that include algorithms and models used to analyze images in videos and articles. This allows for the recognition of specific patterns and features.

[0233] "Analysis methods" refer to methods that use artificial intelligence to extract characteristics from transmitted video and article images and determine the possibility of developmental disorders.

[0234] "Emotional state" refers to the psychological state recognized by analyzing the caregiver's facial expressions and tone of voice. Examples include anxiety and reassurance.

[0235] "Electronic file format" refers to data stored in a digital format, and includes, for example, PDFs and electronic spreadsheet files.

[0236] One embodiment of the present invention is to provide a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos or images from articles about their child, and based on the analysis results by artificial intelligence (AI) and an emotion engine, it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0237] This system involves multiple steps, including user registration and login, uploading videos and articles, collecting sentiment data, AI analysis, providing analysis results and reasoning, and summarizing data and preparing for consultations.

[0238] System Overview

[0239] The user first accesses the system and registers by entering information such as username, password, and email address. The terminal displays an interface for entering this information, and the server receives this information and compares it with existing user information in the database. In the case of new registration, the system verifies that the entered information does not exist in the existing database and adds the new user information to the database. In the case of login, the system verifies that the entered username and password are correct and authentication is successful.

[0240] After logging in, users upload videos of their children or images from articles. The device displays a file selection dialog, and the user selects the file to upload and presses the upload button. At this time, the emotion engine analyzes the user's emotions from their facial expressions and voice, and obtains emotion data at the time of upload. The server saves the emotion data simultaneously with the upload.

[0241] The server then sends the saved videos and images to an AI analysis module. This AI analysis module extracts features such as motion and gaze direction in the case of video data, and analyzes facial expressions and handwriting in the case of images in articles. Based on these features, the AI ​​analysis module determines whether or not characteristics of a developmental disorder are present. The server receives the analysis results and stores them in a database.

[0242] Users who wish to view their analysis results can access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The analysis results clearly indicate specific analysis points such as actions, facial expressions, and handwriting, along with the reasons why they were identified as indicators of developmental disorders. The user's emotional data, as recognized by the emotion engine, is also displayed.

[0243] Finally, the server compiles the user's basic information, upload history, analysis results, and sentiment data in chronological order. This allows the user to obtain the data necessary when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides the user with a download link.

[0244] Specific example

[0245] As a concrete example, the procedure for a user to register with the system and upload a video of their child is described. First, the parent accesses the system and registers. Then, they log in and upload a video file of their child's daily activities. During the upload, the emotion engine analyzes the parent's emotions and detects feelings such as anxiety and impatience. The server sends this video and emotion data to an AI analysis module, and the analysis results are provided after a few minutes. If specific behaviors or gaze patterns are considered indicators of developmental disorders, detailed analysis results and the parent's emotion data are provided along with the reasons why. Based on these results, the user considers the next steps and prepares to consult with medical institutions or government agencies if necessary.

[0246] This system allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures. Furthermore, the system takes into account the parents' emotional state, enabling more reliable assessments.

[0247] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0248] Step 1:

[0249] The user accesses the system and enters their username, password, and email address into the user registration form. The terminal collects this information and sends it to the server after the user presses the submit button. The server compares this input information with the existing database and, if it confirms that the user is not already registered, adds the new user information to the database. The input for this step is the username, password, and email address, and the output is the addition of the new user information to the database.

[0250] Step 2:

[0251] The user enters their username and password on the login screen. The device sends this information to the server. The server compares the entered information with the user information in the database, and if the information is correct, authentication is successful. The input for this step is the username and password, and the output is the user's authentication status. If authentication is successful, the device displays the dashboard screen to the user.

[0252] Step 3:

[0253] After the user logs in, they select an option to upload a video of their child or an image of an article. The device displays a file selection dialog, and the user selects the file to upload. After selection, the user presses the upload button, and the device sends the selected file along with the user's current facial expression and voice data to the server. The input for this step is the video or image file and the user's emotion data, and the output is the transmission of the input data to the server.

[0254] Step 4:

[0255] The server stores the image data and emotional data of the received video or article. Simultaneously, the emotion engine analyzes the user's facial expressions and tone of voice, generating and storing emotional data such as anxiety or reassurance. The input for this step is the video or image data and emotional data, and the output is the storage of this data into the database.

[0256] Step 5:

[0257] The server sends the saved video or images to an AI analysis module. The AI ​​analysis module analyzes this data and extracts features such as the child's movements, gaze direction, facial expressions, and handwriting. Based on the analysis results, it determines whether characteristics of a developmental disorder are present and sends the results to the server. The server saves these analysis results to a database. The input for this step is video or image data, and the output is the saving of the analysis results to the database.

[0258] Step 6:

[0259] The user accesses the results page to view the analysis results. The server retrieves the user's analysis results from the database and displays them on the device in a readable format. The results include analysis points such as specific actions, facial expressions, and handwriting, and the reasons why they were indicators of developmental disorders. The user's emotional data, as recognized by the emotion engine, is also displayed. The input for this step is the user's access request, and the output is the display of the analysis results and emotional data.

[0260] Step 7:

[0261] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links. Users download the generated files and submit them to the necessary institutions. The input for this step is the user's data compilation request, and the output is the generation of the compiled data in PDF format and the provision of a download link.

[0262] (Application Example 2)

[0263] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0264] Conventional systems fail to consider the emotional state of parents when assessing the characteristics of a child's developmental disorder, resulting in a lack of consideration for parental anxiety and stress. Furthermore, insufficient detailed analysis of the child's behavior and facial expressions can make accurate assessment difficult. There is a need to address these issues, enabling a more accurate and comprehensive assessment of a child's developmental disorder, and strengthening consideration for parents.

[0265] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0266] In this invention, the server includes means for parents to upload videos or diary photos of their children, means for analyzing the uploaded videos or photos using AI, means for collecting emotional data using an emotion engine that analyzes the parents' facial expressions and tone of voice, means for determining the characteristics of the child's developmental disorder based on the analysis results, means for providing the analysis results and emotional data to the parents, means for clearly indicating the reasons for the judgment based on the analysis, and means for compiling the data necessary for consultation in chronological order. This makes it possible to more accurately determine the characteristics of the child's developmental disorder and to provide results in a way that takes into account the parents' emotional state.

[0267] A "guardian" is someone who is responsible for caring for and supervising a child, and for checking for characteristics of developmental disabilities.

[0268] "Children" in this system refer to minors who may exhibit characteristics of developmental disorders.

[0269] A "video" is a video file used to record a child's actions, gaze direction, and movements.

[0270] A "diary" is a document or photograph that records a child's daily life and activities.

[0271] The "means of uploading" refer to an interface that allows parents to send videos and photos from their diaries to the system.

[0272] "Analyzing AI" refers to artificial intelligence that automatically analyzes uploaded videos and photos and extracts specific features.

[0273] "Analysis method" refers to the process by which AI analyzes videos and photos from diaries and extracts features based on that analysis.

[0274] The "Emotion Engine" is software for analyzing the expressions and voice tones of guardians to collect emotion data.

[0275] "Emotion data" refers to information regarding the emotional state of the guardian collected by the Emotion Engine.

[0276] The "means of judgment" is a process of identifying and evaluating the characteristics of a child's developmental disorder based on data analyzed by AI.

[0277] The "means of providing" is an interface or function for clearly presenting and providing the analysis results and emotion data to the guardian.

[0278] The "means of clearly indicating the reason for judgment" is a process of specifically explaining the characteristics that may indicate a child's developmental disorder based on the analysis data of AI. <000第00882>

[0279] 第0000885 The "means of organizing data necessary for consultation in chronological order" is a means for organizing the data required by the guardian in chronological order and providing it to medical institutions and administrative agencies.

[0280] The "means of extracting the direction of movement and line of sight as feature quantities" is a process of analyzing the direction of a child's movement and line of sight in a video to extract important feature quantities.

[0281] The "means of analyzing expressions and handwriting" is a process of analyzing a child's expressions and handwriting from a photo of a diary and extracting their characteristics and changes.

[0282] Modes for Carrying Out the Invention

[0283] As a mode for carrying out this invention, a system for guardians to check the characteristics of a child's developmental disorder is provided. The following describes the specific implementation method.

[0284] User Registration and Login

[0285] <第 The user first accesses the system and inputs information such as username, password, and email address for new registration. In the case of an existing user, they log in using the username and password. The server receives this information and checks it against the existing user information in the database. In the case of new registration, it is confirmed that the input information does not exist in the existing database, and new user information is added to the database. In the case of login, it is confirmed that the input username and password are correct, and the authentication is successful.

[0286] Upload of Videos and Diaries

[0287] After the user logs in, they upload videos of their children or diary photos. When uploading, the user selects a video file or diary photo and presses the upload button. At this time, the emotion engine analyzes the emotion from the user's expression and voice and obtains the emotion data at the time of upload.

[0288] Collection of Emotion Data

[0289] The server saves the emotion data obtained at the time of upload. The emotion engine recognizes emotions such as anxiety, relief, and impatience from the user's expression and tone of voice. This data is also included in the analysis results.

[0290] Analysis by AI

[0291] The server sends the saved videos and photos to the AI analysis module. In the case of a video, the server uses AI to extract feature quantities such as actions and directions of gaze. In the case of a diary photo, expressions and handwriting are analyzed. The AI analysis module analyzes these feature quantities and determines whether there are characteristics of developmental disorders. The server receives this analysis result and saves it in the database. The analysis results include the analysis date and time, information on the uploaded media, features found by the AI, and the comprehensive judgment.

[0292] Provision of Analysis Results and Reasons for Judgment

[0293] When a user wishes to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The analysis results clearly indicate specific points such as actions, facial expressions, and handwriting, and explain which characteristics were indicators of developmental disorders. The server also displays the user's emotional data recognized by the emotion engine, demonstrating consideration for the reliability of the analysis results and the user's feelings.

[0294] Data compilation and consultation preparation

[0295] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links.

[0296] Specific example

[0297] This example illustrates how a user registers with the system and uploads videos of their child. First, the parent accesses the system and registers. After logging in, they upload video files of their child's daily activities. During the upload, the emotion engine analyzes the parent's emotions and detects feelings such as anxiety and impatience. The server sends this video and emotion data to an AI analysis module, and the analysis results are provided after a few minutes. If specific behaviors or gaze patterns are identified as indicators of developmental disorders, detailed analysis results and the parent's emotion data are provided along with the reasons why. Based on these results, the user can consider the next steps and prepare to consult with medical institutions or government agencies if necessary.

[0298] Examples of prompts for generative AI models

[0299] "Please send videos of children, including emotional data analyzed by EmotionEngine, to AIAnalyzer to analyze the children's behavior and risk factors. Then, please tell me how to notify the parents based on the analysis results."

[0300] The flow of the specific process in Application Example 2 will be described using FIG. 14.

[0301] Step 1:

[0302] The user accesses the system and inputs a username, password, and email address for new registration. The server receives the input data (username, password, email address) and checks it against the existing user information in the database. After confirming that the new information does not exist in the existing database, the new user information is added to the database. The input for this step is user information, and the output is a confirmation message for new user registration.

[0303] Step 2:

[0304] In the case of an existing user, log in using the username and password. The server receives the login information (username, password) entered by the user and checks it against the user information stored in the database. If the check is successful, the server returns a login success message. The input is login information, and the output is a message indicating login success or failure.

[0305] Step 3:

[0306] After the user logs in, upload a video of a child or a photo of a diary from the terminal. At the time of upload, the file path and file name are entered by selecting a file and pressing the upload button. The server uses an emotion engine to analyze the user's expression and voice tone, obtains emotion data, and saves it together with the file. The input for this step is the uploaded file and emotion data, and the output is a confirmation message for saving.

[0307] Step 4:

[0308] The server sends uploaded videos and photos to an AI analysis module. The AI ​​analysis module extracts motion and gaze direction from videos and analyzes facial expressions and handwriting from photos. The input is the uploaded media file, and the output is the extracted features. The server receives the analyzed features and stores them in a database. The input and output for this step are the media file and the analysis results.

[0309] Step 5:

[0310] Based on the analysis results, the server uses AI to make a comprehensive judgment and determine the characteristics of the child's developmental disorder. The input is the features analyzed by the AI, and the output is the judgment result regarding the characteristics of the developmental disorder. The server stores this result in a database.

[0311] Step 6:

[0312] The user accesses a dedicated results display page to view the analysis results. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format along with sentiment data, and displays them. The input for this step is the user ID, and the output is the formatted analysis results.

[0313] Step 7:

[0314] Once the user obtains the analysis results, the server displays the diagnosis reasons and discovered emotional data. Furthermore, if necessary, the system compiles the data required for consultation in chronological order, generates it in a format such as PDF, and provides the user with a download link. The input for this step is the user request, and the output is the chronologically compiled information and a PDF download link.

[0315] Step 8:

[0316] In terms of specific actions, the user checks the analysis results on the terminal and prepares to consult with medical institutions or government agencies as needed. Based on this, the user decides on the next action and takes appropriate action. The input for this step is the analysis results, and the output is instructions for the appropriate next step.

[0317] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0318] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0319] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0320] [Second Embodiment]

[0321] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0322] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0323] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0324] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0325] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0326] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0327] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0328] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0329] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0330] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0331] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0332] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0333] One embodiment of the present invention is the provision of a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos and photos from their child's diary, and based on the results of AI analysis, it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0334] User registration and login

[0335] Users first access the system and register by entering information such as username, password, and email address. Existing users log in using their username and password. The server receives this information and compares it with existing user information in the database. In the case of new registration, it verifies that the entered information does not exist in the database and adds the new user information to the database. In the case of login, it verifies that the entered username and password are correct and authentication is successful.

[0336] Uploading videos and diaries

[0337] After a user logs in, they upload videos of their child or photos from their diary. When uploading, the user selects a video file or a diary photo and presses the upload button. The server receives the file and saves it to a temporary storage location. Furthermore, it verifies the file type (video or photo) and saves it to the appropriate folder. During saving, the server logs the file's save date, time, and other metadata.

[0338] AI analysis

[0339] The server sends saved videos and photos to an AI analysis module. In the case of videos, the server uses AI to extract features such as motion and gaze direction. In the case of diary photos, it analyzes facial expressions and handwriting. The AI ​​analysis module analyzes these features to determine whether characteristics of a developmental disorder are present. The server receives these analysis results and stores them in a database.

[0340] Providing analysis results and reasons for judgment

[0341] If a user wants to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. Specifically, it highlights key points such as particular actions, facial expressions, and handwriting, and explains which characteristics were indicators of developmental disorders.

[0342] Data compilation and consultation preparation

[0343] The server compiles user basic information, upload history, and analysis results in chronological order. This allows users to consult with medical institutions and government agencies more smoothly. The server generates the compiled data in formats such as PDF and provides it to the user. Users can download this data and use it for consultations.

[0344] Specific example

[0345] This example shows a user registering with the system and uploading a video of their child. First, the parent accesses the system and registers. Then, they log in and upload a video file of their child's daily activities. The server receives this video and begins AI analysis. After a few minutes, the analysis results are available, and if certain movements or gaze patterns are considered indicators of developmental disorders, detailed analysis results are provided along with the reasons why. Based on these results, the user considers the next steps and, if necessary, downloads data to prepare for consultations with medical institutions or government agencies.

[0346] This series of steps allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures.

[0347] The following describes the processing flow.

[0348] Step 1:

[0349] Users access the system and either register or log in. For new registrations, they enter a username, password, and email address. For logins, they enter their already registered username and password.

[0350] Step 2:

[0351] The server receives the entered information and checks if the username exists in the existing database. For new registrations, it verifies that the username does not exist and then adds the registration information to the database. For logins, it verifies that the username and password match and performs authentication.

[0352] Step 3:

[0353] After logging in, the user uploads video files or photos from their diary. They select the file to upload and press the upload button.

[0354] Step 4:

[0355] The server receives the uploaded file and checks its type (video or photo). It saves the file to a temporary storage location. Then, it moves the file to the appropriate folder and logs metadata including the save date and time and user information.

[0356] Step 5:

[0357] The server sends stored videos and photos to an AI analysis module. For videos, features such as motion and gaze direction are extracted. For photos, facial expressions and handwriting are analyzed.

[0358] Step 6:

[0359] The server receives the analysis results from the AI ​​analysis module and determines whether characteristics of developmental disorders are present. These analysis results are then stored in a database. The analysis results include the date and time of analysis, information about the uploaded media, the features identified by the AI, and an overall assessment.

[0360] Step 7:

[0361] The user accesses the results page to view the analysis results. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The detailed explanation of the results clearly indicates specific points such as particular actions, facial expressions, and handwriting.

[0362] Step 8:

[0363] The server compiles the user's basic information, upload history, and analysis results in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides the user with a download link.

[0364] (Example 1)

[0365] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0366] Modern parents are seeking ways to detect potential developmental disorders in their children early and take appropriate measures. However, current systems and methods make it difficult for parents to easily check their child's developmental characteristics and consult with medical or administrative agencies based on accurate data. In particular, there is a lack of systems that can efficiently analyze various media files such as videos and photos and provide reliable results. Furthermore, clearly explaining the reasoning behind the judgments based on the analysis results and organizing the data necessary for consultations in chronological order are also important issues.

[0367] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0368] In this invention, the server includes means for parents to upload media files of their children, means for analyzing the uploaded media files using artificial intelligence, means for determining the characteristics of the child's developmental disorder based on the analysis results, means for providing the analysis results to the parents, means for clearly showing the reasons for the judgment based on the analysis, means for compiling the data necessary for consultation in chronological order, and means for users to access the system by registering and logging in. This makes it possible for parents to easily check for the possibility of their child's developmental disorder and take quick and appropriate measures based on reliable data.

[0369] "Guardian" refers to an adult who has the responsibility for raising and supervising a child.

[0370] "Media files" refer to digital data that includes visual information, such as video files and image files.

[0371] "Means of uploading" refers to the interface or function that allows users to transfer files from their devices to a server.

[0372] "Analytical artificial intelligence" refers to algorithms and software that automatically extract specific patterns and features from input data and make decisions based on the results.

[0373] "Analysis means" refers to a function that uses artificial intelligence to process uploaded media files and extract and analyze specific features.

[0374] "Means of judgment" refers to algorithms or logic used to derive specific conclusions based on analysis results.

[0375] "Means of providing" refers to functions for displaying analysis results and other information to users or distributing them in a downloadable format.

[0376] "Means of clearly demonstrating the reasoning behind the decision" refers to functions that explain the basis and causes behind the analysis results in a way that is easy for the user to understand.

[0377] "Methods for organizing data chronologically" refers to functions that organize and display data such as analysis results and upload history in chronological order.

[0378] "Method of registering and logging in" refers to the process by which a user registers their information in the system and obtains access rights after authentication.

[0379] One embodiment of the present invention is the provision of a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos and photos from their child's diary, and based on the analysis results by artificial intelligence (AI), it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0380] User registration and login

[0381] Users first access the system and register by entering information such as their username, password, and email address. They then log in using this registered information. The server stores this information in a database and authenticates the user. For new registrations, the server verifies that the entered information does not already exist in the database and adds the new user information. For existing users, the server authenticates their username and password to ensure successful login.

[0382] Uploading videos and diaries

[0383] After logging in, users upload videos of their children or photos from their diaries. This involves selecting a video file or diary photo and pressing the upload button. The server receives the file and saves it to a temporary storage location. It then stores it in the appropriate folder based on the file type and records metadata at the time of saving.

[0384] AI analysis

[0385] The server sends the saved videos and photos to the AI ​​analysis module. For videos, it extracts features such as movement and gaze direction; for photos, it analyzes facial expressions and handwriting. This analysis is performed using a generative AI model. The AI ​​analysis module analyzes the acquired features to determine if characteristics of a developmental disorder are present. The analysis results are stored in a database by the server.

[0386] Providing analysis results and reasons for judgment

[0387] When a user wishes to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database and provides them in a visually easy-to-understand format. This includes a function that explains in detail why certain actions, facial expressions, or handwriting patterns are indicators of developmental disorders.

[0388] Data compilation and consultation preparation

[0389] The server compiles user basic information, upload history, and analysis results in chronological order, thereby supporting users in easily consulting with medical and administrative agencies. The compiled data is generated in formats such as PDF and provided to users in a downloadable format.

[0390] Examples of specific prompt messages include the following:

[0391] "Please analyze the video of the child and let me know if it shows any characteristics of a developmental disorder. This video shows a 5-year-old child playing."

[0392] "I've uploaded a photo of the diary. Please analyze the writing and drawings of this child and let me know the results."

[0393] This system allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures promptly.

[0394] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0395] Step 1:

[0396] Users access the system and register by entering their username, password, and email address on the registration screen. For example, they might enter "username," "password," and "email@example.com." The server receives this information and checks it against the database to ensure there are no duplicates in existing user information. Based on the results, the server saves the new user information to the database.

[0397] Input: Username, Password, Email Address

[0398] Output: User information database registration confirmation message

[0399] Step 2:

[0400] The user logs in by entering their existing username and password on the login screen. For example, they might enter "username" and "password". The server receives this information, accesses the database, and verifies that the username and password combination is correct. If authentication is successful, the user interface is displayed.

[0401] Input: Username, Password

[0402] Output: Login success or failure message

[0403] Step 3:

[0404] After logging in, the user clicks a button to upload videos or diary photos, opening a file selection dialog. For example, they select "child_video.mp4" or "diary_image.jpg" and click the upload button. The server receives the file and saves it to a temporary storage location. Next, it verifies the file type (video or photo) and moves it to the appropriate folder. In addition, it logs the file's save date and metadata.

[0405] Input: Selected file

[0406] Output: Message indicating file save complete.

[0407] Step 4:

[0408] The server sends stored videos and photos to an artificial intelligence (AI) analysis module. For example, it sends "child_video.mp4". The server extracts motion and gaze features from the video file and analyzes facial expressions and handwriting from the photos. It uses a generative AI model to recognize specific actions and patterns.

[0409] Input: Video or photo file

[0410] Output: Extracted features and analysis results

[0411] Step 5:

[0412] The server saves the analysis results received from the AI ​​analysis module to a database. For example, if "frequent eye movement" or "waving motion" is detected, the server saves the analysis results, including these features, to the database.

[0413] Input: AI analysis results

[0414] Output: Status of saving analysis results to the database

[0415] Step 6:

[0416] The user accesses a dedicated results page to view the analysis results. The server retrieves the analysis results for the user from the database and provides them in an easy-to-understand format. This includes a detailed explanation of any specific behaviors, facial expressions, or handwriting detected, and why they may be indicators of developmental disorders.

[0417] Input: User Request

[0418] Output: Formatted analysis results

[0419] Step 7:

[0420] The server compiles user basic information, upload history, and analysis results chronologically, generating them in file formats such as PDF. Users can download and use this data. For example, it can be used as necessary documentation when consulting with medical institutions or government agencies.

[0421] Input: User information, upload history, analysis results

[0422] Output: Time-series data summary PDF file

[0423] Through the above processing steps, the system helps parents check for potential developmental disorders in their children and take prompt action based on reliable data.

[0424] (Application Example 1)

[0425] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0426] Ensuring the safety of passengers in autonomous vehicles is crucial, and it is especially necessary to enable rapid response by identifying abnormal behavior or facial expressions in real time, particularly among minors and children. Furthermore, it is essential to provide an environment where passengers with developmental disabilities can receive appropriate support.

[0427] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0428] In this invention, the server includes means for parents to upload videos or diary photos of their children, means for analyzing the uploaded videos or photos using AI, means for determining the characteristics of the child's developmental disability based on the analysis results, means for providing the analysis results to the parents, means for clearly indicating the reasons for the judgment based on the analysis, means for compiling data necessary for consultation in chronological order, means for recording the movements and facial expressions of passengers with cameras inside the vehicle, means for analyzing the recorded data in real time to detect abnormal movements or facial expressions, and means for notifying the parents' mobile devices when an abnormality is detected. This enables real-time assurance of passenger safety in autonomous vehicles and allows for appropriate support, especially for passengers with characteristics of developmental disabilities.

[0429] "Guardian" refers to an adult who is responsible for raising and protecting a child or minor.

[0430] "Video" refers to digital data that expresses movement by displaying a series of still images at regular intervals.

[0431] "Diary photos" refers to digital image files that are photographs taken of the contents of a diary recorded on paper or digital media.

[0432] "Uploading" refers to the act of transferring data from a local device to a remote system such as a server.

[0433] "Analysis means" refers to methods and devices for extracting characteristics and patterns from data and converting them into meaningful information.

[0434] "AI" stands for Artificial Intelligence, and refers to machine learning algorithms and models that automatically process specific tasks.

[0435] "Developmental disorder" refers to a condition in which a person has certain characteristics or behavioral patterns that are observed during the developmental process, and experiences specific difficulties in learning, socializing, and behavior.

[0436] A "server" refers to a computer system used for storing, processing, and providing data.

[0437] A "camera" is a device that captures light and records still images and videos.

[0438] "Dynamics" is a concept that refers to the movement and changes of people and objects.

[0439] "Facial expression" refers to the expression of emotions and intentions through the movement of facial muscles.

[0440] "Real-time" refers to a state where data collection, processing, and feedback occur instantly in accordance with real-world time.

[0441] A "mobile device" is a portable electronic device, including smartphones and tablets.

[0442] A "notification" refers to a message or warning that informs someone of a specific event or condition.

[0443] A "system" refers to a collection of interconnected components that work together to achieve a specific purpose.

[0444] The system that implements this application consists of the following steps and is carried out using various hardware and software.

[0445] User registration and login

[0446] The user first accesses the system and registers as a parent or guardian. They enter necessary information such as username, password, and email address. Existing users log in with their username and password. The server receives this information, compares it with existing user information in the database, and adds new user information to the database if necessary. During login, the system verifies that the entered information is correct and performs authentication. This allows the user to access the system.

[0447] Uploading videos and diaries

[0448] After a user logs in, parents upload videos and diary photos of their children to the system. The user selects a video file or diary photo and presses the upload button. The server receives the file and saves it to a temporary storage location. At this time, it determines the file type (video or photo) and saves it to the appropriate folder. The save date and other metadata are also recorded.

[0449] AI analysis

[0450] The server sends saved videos and photos to an AI analysis module. In the case of videos, AI is used to extract features such as motion and gaze direction. In the case of diary photos, facial expressions and handwriting are analyzed. The AI ​​analysis module analyzes these features to determine whether or not there are characteristics of a developmental disorder. The analysis results are received by the server and stored in a database.

[0451] Providing analysis results and reasons for judgment

[0452] The user accesses a dedicated results page to view the analysis results. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. It clearly indicates specific points such as certain actions, facial expressions, and handwriting, and provides a detailed explanation of why they were indicators of developmental disorders.

[0453] Data compilation and consultation preparation

[0454] The server compiles user basic information, upload history, and analysis results in chronological order. This allows users to smoothly consult with medical institutions and government agencies. The server generates the compiled data in PDF format and other formats, making it available for users to download.

[0455] Applications in autonomous vehicles

[0456] Cameras are installed inside the vehicle to record the movements and expressions of passengers, especially children, in real time. The recorded data is analyzed in real time by an AI analysis module. If abnormal behavior or expressions are detected, the server immediately sends a notification to the parent's mobile device. This ensures the safety of passengers in autonomous vehicles.

[0457] Specific example

[0458] For example, a parent can register with the system and upload videos of their child's daily life. These videos are stored on a server and analyzed by AI. Parents are notified of any specific abnormal behaviors or gaze directions indicated by the analysis results. Also, when a child is in an autonomous vehicle, the AI ​​analyzes videos taken inside the vehicle, and if it detects abnormal behavior or facial expressions, a warning notification is sent to the parent's smartphone in real time.

[0459] Example of a prompt

[0460] "Please explain how to use video and image data captured by in-car cameras to analyze passengers' facial expressions and movements in real time, thereby supporting safe autonomous driving."

[0461] Thus, by using this system, parents can not only check for characteristics of developmental disorders in their children at an early stage, but also enhance safety inside autonomous vehicles.

[0462] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0463] Step 1:

[0464] A user accesses the system to register or log in. They enter information such as their username, password, and email address, and submit it to the server. The server verifies the input data against the database; in the case of new registration, it is saved as new user information. In the case of login, authentication is performed through verification. The input is the username and password, and the output is the authentication result.

[0465] Step 2:

[0466] After the user logs in, they upload a video of their child or a photo from their diary. The user selects a video file or a photo from their diary and presses the upload button. The server receives this file and saves it to a temporary storage location. It determines the file type and saves it to the appropriate folder. The input is the video file or photo from the diary, and the output is the path to the saved file.

[0467] Step 3:

[0468] The server sends saved videos and photos to an AI analysis module. For videos, it extracts motion and gaze direction frame by frame. For diary photos, it analyzes facial expressions and handwriting features. The analysis results are received by the server and stored in a database. The input is a saved video file or diary photo, and the output is the analysis results.

[0469] Step 4:

[0470] The server formats the analysis results into a dedicated results display page and provides them to the user. It provides detailed explanations of specific actions, facial expressions, handwriting, and other points, clearly indicating the reasons why they were identified as indicators of developmental disorders. Users can view the analysis results through a browser or app. The input is the analysis result data, and the output is the formatted results display page.

[0471] Step 5:

[0472] The server compiles user basic information, upload history, and analysis results in chronological order. This allows users to generate data that facilitates consultations with medical institutions and government agencies. The time-series data is generated in formats such as PDF, making it available for users to download. The input consists of user basic information and analysis results, and the output is a data file compiled in chronological order.

[0473] Step 6:

[0474] Cameras inside the autonomous vehicle record passengers' movements and facial expressions in real time. The recorded data is sent to an AI analysis module to detect abnormalities in movement or facial expressions. If an abnormality is detected, the server sends a notification to the parent's mobile device. The input is real-time recorded video data, and the output is an abnormality detection notification.

[0475] Step 7:

[0476] The parent's mobile device receives notifications sent from the server and displays the details of the anomaly. The user can review the notification and take necessary actions quickly. The input is the notification data from the server, and the output is the displayed notification content.

[0477] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0478] One embodiment of the present invention is the provision of a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos and photos from their child's diary, and based on the analysis results by AI and an emotion engine, it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0479] User registration and login

[0480] Users first access the system and register by entering information such as username, password, and email address. Existing users log in using their username and password. The server receives this information and compares it with existing user information in the database. In the case of new registration, it verifies that the entered information does not exist in the database and adds the new user information to the database. In the case of login, it verifies that the entered username and password are correct and authentication is successful.

[0481] Uploading videos and diaries

[0482] After logging in, users upload videos of their children or photos from their diaries. When uploading, users select a video file or a photo from their diary and press the upload button. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data at the time of upload.

[0483] Collection of emotional data

[0484] The server stores emotional data acquired during the upload process. The emotion engine recognizes emotions such as anxiety, relief, and impatience from the user's facial expressions and tone of voice. This data is also included in the analysis results.

[0485] AI analysis

[0486] The server sends saved videos and photos to an AI analysis module. In the case of videos, the server uses AI to extract features such as motion and gaze direction. In the case of diary photos, it analyzes facial expressions and handwriting. The AI ​​analysis module analyzes these features and determines whether characteristics of developmental disorders are present. The server receives these analysis results and stores them in a database. The analysis results include the date and time of analysis, information about the uploaded media, features found by the AI, and an overall judgment.

[0487] Providing analysis results and reasons for judgment

[0488] When a user wishes to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The analysis results clearly indicate specific points such as actions, facial expressions, and handwriting, and explain which characteristics were indicators of developmental disorders. The server also displays the user's emotional data recognized by the emotion engine, demonstrating consideration for the reliability of the analysis results and the user's feelings.

[0489] Data compilation and consultation preparation

[0490] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links.

[0491] Specific example

[0492] This example illustrates how a user registers with the system and uploads videos of their child. First, the parent accesses the system and registers. After logging in, they upload video files of their child's daily activities. During the upload, the emotion engine analyzes the parent's emotions and detects feelings such as anxiety and impatience. The server sends this video and emotion data to an AI analysis module, and the analysis results are provided after a few minutes. If specific behaviors or gaze patterns are identified as indicators of developmental disorders, detailed analysis results and the parent's emotion data are provided along with the reasons why. Based on these results, the user can consider the next steps and prepare to consult with medical institutions or government agencies if necessary.

[0493] This series of steps allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures. Furthermore, the parents' emotional state is taken into consideration, leading to more reliable decisions.

[0494] The following describes the processing flow.

[0495] Step 1:

[0496] Users access the system and either register or log in. For new registrations, they enter a username, password, and email address. For logins, they enter their already registered username and password.

[0497] Step 2:

[0498] The server receives the entered information and checks if the username exists in the existing database. For new registrations, it verifies that the username does not exist and then adds the registration information to the database. For logins, it verifies that the username and password match and performs authentication.

[0499] Step 3:

[0500] After logging in, the user selects an option to upload a video or diary photo from the system's main screen. During the upload process, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data.

[0501] Step 4:

[0502] The device sends the selected video file or diary photo to the server. At the same time, the emotion engine also sends the user's emotion data (e.g., anxiety, relief, impatience).

[0503] Step 5:

[0504] The server saves received videos, photos, and sentiment data to a temporary storage location. Metadata (e.g., save date and time, type of uploaded file) is logged for the saved files and sentiment data.

[0505] Step 6:

[0506] The server sends stored videos and photos to an AI analysis module. In the case of videos, the server extracts features such as movement and gaze direction. In the case of photos, it performs analysis of facial expressions and handwriting.

[0507] Step 7:

[0508] The server receives the analysis results from the AI ​​analysis module and determines whether characteristics of developmental disorders are present. The analysis results are saved to a database, recording the date and time of analysis, information about the uploaded media, the characteristics found by the AI, and the overall judgment.

[0509] Step 8:

[0510] The user accesses the results page to view the analysis results. The server retrieves the analysis results for that user from the database and formats them, including sentiment data, into an easy-to-read format.

[0511] Step 9:

[0512] The server provides the user with analysis results and emotional data. The results clearly indicate specific points such as actions, facial expressions, and handwriting, and explain which characteristics were indicators of developmental disorders. The display method of the analysis results is also adjusted based on the emotional data.

[0513] Step 10:

[0514] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links.

[0515] This allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures. Furthermore, the parents' emotional state is also taken into consideration, leading to more reliable assessments.

[0516] (Example 2)

[0517] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0518] In recent years, the early detection and support of developmental disorders in children have become increasingly important, but parents have limited means to regularly check for these conditions. Furthermore, parents' own emotional states can make accurate judgments difficult. In particular, feelings of anxiety and impatience can influence parents' judgments, hindering appropriate responses. To address this challenge, a system is needed that allows parents to easily check their child's condition and receive reliable results. Additionally, there is a need for a system that takes parents' emotional states into account to enable more objective and reliable judgments.

[0519] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0520] In this invention, the server includes means for parents to transfer images of their child or articles; means for analyzing the transferred images or videos using artificial intelligence; and means for detecting the parent's emotional state and including that data in the analysis. This makes it possible for parents to routinely check for characteristics of their child's developmental disorder and to obtain analysis results that also take into account the parent's emotional state.

[0521] A "guardian" is a person who has the responsibility to support the healthy growth and development of a child and who takes care of the child's daily life and education.

[0522] "Video" refers to moving image data that records a child's actions or situation. Examples include video clips and movies.

[0523] "Article images" refer to still image files of diaries, drawings, or other documents created by children.

[0524] "Transferring" refers to the act of sending images of videos or articles collected by a parent or guardian to the system.

[0525] Artificial intelligence refers to software programs that include algorithms and models used to analyze images in videos and articles. This allows for the recognition of specific patterns and features.

[0526] "Analysis methods" refer to methods that use artificial intelligence to extract characteristics from transmitted video and article images and determine the possibility of developmental disorders.

[0527] "Emotional state" refers to the psychological state recognized by analyzing the caregiver's facial expressions and tone of voice. Examples include anxiety and reassurance.

[0528] "Electronic file format" refers to data stored in a digital format, and includes, for example, PDFs and electronic spreadsheet files.

[0529] One embodiment of the present invention is to provide a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos or images from articles about their child, and based on the analysis results by artificial intelligence (AI) and an emotion engine, it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0530] This system involves multiple steps, including user registration and login, uploading videos and articles, collecting sentiment data, AI analysis, providing analysis results and reasoning, and summarizing data and preparing for consultations.

[0531] System Overview

[0532] The user first accesses the system and registers by entering information such as username, password, and email address. The terminal displays an interface for entering this information, and the server receives this information and compares it with existing user information in the database. In the case of new registration, the system verifies that the entered information does not exist in the existing database and adds the new user information to the database. In the case of login, the system verifies that the entered username and password are correct and authentication is successful.

[0533] After logging in, users upload videos of their children or images from articles. The device displays a file selection dialog, and the user selects the file to upload and presses the upload button. At this time, the emotion engine analyzes the user's emotions from their facial expressions and voice, and obtains emotion data at the time of upload. The server saves the emotion data simultaneously with the upload.

[0534] The server then sends the saved videos and images to an AI analysis module. This AI analysis module extracts features such as motion and gaze direction in the case of video data, and analyzes facial expressions and handwriting in the case of images in articles. Based on these features, the AI ​​analysis module determines whether or not characteristics of a developmental disorder are present. The server receives the analysis results and stores them in a database.

[0535] Users who wish to view their analysis results can access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The analysis results clearly indicate specific analysis points such as actions, facial expressions, and handwriting, along with the reasons why they were identified as indicators of developmental disorders. The user's emotional data, as recognized by the emotion engine, is also displayed.

[0536] Finally, the server compiles the user's basic information, upload history, analysis results, and sentiment data in chronological order. This allows the user to obtain the data necessary when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides the user with a download link.

[0537] Specific example

[0538] As a concrete example, the procedure for a user to register with the system and upload a video of their child is described. First, the parent accesses the system and registers. Then, they log in and upload a video file of their child's daily activities. During the upload, the emotion engine analyzes the parent's emotions and detects feelings such as anxiety and impatience. The server sends this video and emotion data to an AI analysis module, and the analysis results are provided after a few minutes. If specific behaviors or gaze patterns are considered indicators of developmental disorders, detailed analysis results and the parent's emotion data are provided along with the reasons why. Based on these results, the user considers the next steps and prepares to consult with medical institutions or government agencies if necessary.

[0539] This system allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures. Furthermore, the system takes into account the parents' emotional state, enabling more reliable assessments.

[0540] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0541] Step 1:

[0542] The user accesses the system and enters their username, password, and email address into the user registration form. The terminal collects this information and sends it to the server after the user presses the submit button. The server compares this input information with the existing database and, if it confirms that the user is not already registered, adds the new user information to the database. The input for this step is the username, password, and email address, and the output is the addition of the new user information to the database.

[0543] Step 2:

[0544] The user enters their username and password on the login screen. The device sends this information to the server. The server compares the entered information with the user information in the database, and if the information is correct, authentication is successful. The input for this step is the username and password, and the output is the user's authentication status. If authentication is successful, the device displays the dashboard screen to the user.

[0545] Step 3:

[0546] After the user logs in, they select an option to upload a video of their child or an image of an article. The device displays a file selection dialog, and the user selects the file to upload. After selection, the user presses the upload button, and the device sends the selected file along with the user's current facial expression and voice data to the server. The input for this step is the video or image file and the user's emotion data, and the output is the transmission of the input data to the server.

[0547] Step 4:

[0548] The server stores the image data and emotional data of the received video or article. Simultaneously, the emotion engine analyzes the user's facial expressions and tone of voice, generating and storing emotional data such as anxiety or reassurance. The input for this step is the video or image data and emotional data, and the output is the storage of this data into the database.

[0549] Step 5:

[0550] The server sends the saved video or images to an AI analysis module. The AI ​​analysis module analyzes this data and extracts features such as the child's movements, gaze direction, facial expressions, and handwriting. Based on the analysis results, it determines whether characteristics of a developmental disorder are present and sends the results to the server. The server saves these analysis results to a database. The input for this step is video or image data, and the output is the saving of the analysis results to the database.

[0551] Step 6:

[0552] The user accesses the results page to view the analysis results. The server retrieves the user's analysis results from the database and displays them on the device in a readable format. The results include analysis points such as specific actions, facial expressions, and handwriting, and the reasons why they were indicators of developmental disorders. The user's emotional data, as recognized by the emotion engine, is also displayed. The input for this step is the user's access request, and the output is the display of the analysis results and emotional data.

[0553] Step 7:

[0554] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links. Users download the generated files and submit them to the necessary institutions. The input for this step is the user's data compilation request, and the output is the generation of the compiled data in PDF format and the provision of a download link.

[0555] (Application Example 2)

[0556] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0557] Conventional systems fail to consider the emotional state of parents when assessing the characteristics of a child's developmental disorder, resulting in a lack of consideration for parental anxiety and stress. Furthermore, insufficient detailed analysis of the child's behavior and facial expressions can make accurate assessment difficult. There is a need to address these issues, enabling a more accurate and comprehensive assessment of a child's developmental disorder, and strengthening consideration for parents.

[0558] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0559] In this invention, the server includes means for parents to upload videos or diary photos of their children, means for analyzing the uploaded videos or photos using AI, means for collecting emotional data using an emotion engine that analyzes the parents' facial expressions and tone of voice, means for determining the characteristics of the child's developmental disorder based on the analysis results, means for providing the analysis results and emotional data to the parents, means for clearly indicating the reasons for the judgment based on the analysis, and means for compiling the data necessary for consultation in chronological order. This makes it possible to more accurately determine the characteristics of the child's developmental disorder and to provide results in a way that takes into account the parents' emotional state.

[0560] A "guardian" is someone who is responsible for caring for and supervising a child, and for checking for characteristics of developmental disabilities.

[0561] "Children" in this system refer to minors who may exhibit characteristics of developmental disorders.

[0562] A "video" is a video file used to record a child's actions, gaze direction, and movements.

[0563] A "diary" is a document or photograph that records a child's daily life and activities.

[0564] The "means of uploading" refer to an interface that allows parents to send videos and photos from their diaries to the system.

[0565] "Analyzing AI" refers to artificial intelligence that automatically analyzes uploaded videos and photos and extracts specific features.

[0566] "Analysis method" refers to the process by which AI analyzes videos and photos from diaries and extracts features based on that analysis.

[0567] The "emotion engine" is software that analyzes the facial expressions and tone of voice of parents to collect emotional data.

[0568] "Emotional data" refers to information about the emotional state of parents collected by the emotion engine.

[0569] The "means of judgment" refer to the process of identifying and evaluating the characteristics of a child's developmental disorder based on data analyzed by AI.

[0570] "Means of provision" refers to interfaces and functions that display and provide analysis results and emotional data to parents in an easy-to-understand manner.

[0571] "Means of clearly demonstrating the reasoning behind the judgment" refers to a process of specifically explaining the characteristics that may indicate a child has a developmental disorder, based on AI analysis data.

[0572] "Methods for compiling data necessary for consultation in chronological order" refers to methods for parents to organize the necessary data chronologically and provide it to medical institutions and administrative agencies.

[0573] "Methods for extracting movement and gaze direction as features" refers to the process of analyzing a child's movement and gaze direction from a video to extract important features.

[0574] "Methods for analyzing facial expressions and handwriting" refers to the process of analyzing a child's facial expressions and handwriting from photographs in their diary, and extracting their characteristics and changes.

[0575] Modes for carrying out the invention

[0576] One embodiment of this invention is a system for parents to check for characteristics of developmental disorders in their children. The specific implementation method is described below.

[0577] User registration and login

[0578] Users first access the system and register by entering information such as username, password, and email address. Existing users log in using their username and password. The server receives this information and compares it with existing user information in the database. In the case of new registration, it verifies that the entered information does not exist in the database and adds the new user information to the database. In the case of login, it verifies that the entered username and password are correct and authentication is successful.

[0579] Uploading videos and diaries

[0580] After logging in, users upload videos of their children or photos from their diaries. When uploading, users select a video file or a photo from their diary and press the upload button. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data at the time of upload.

[0581] Collection of emotional data

[0582] The server stores emotional data acquired during the upload. The emotion engine recognizes emotions such as anxiety, relief, and impatience from the user's facial expressions and tone of voice. This data is also included in the analysis results.

[0583] AI analysis

[0584] The server sends saved videos and photos to an AI analysis module. In the case of videos, the server uses AI to extract features such as motion and gaze direction. In the case of diary photos, it analyzes facial expressions and handwriting. The AI ​​analysis module analyzes these features and determines whether characteristics of developmental disorders are present. The server receives these analysis results and stores them in a database. The analysis results include the date and time of analysis, information about the uploaded media, features found by the AI, and an overall judgment.

[0585] Providing analysis results and reasons for judgment

[0586] When a user wishes to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The analysis results clearly indicate specific points such as actions, facial expressions, and handwriting, and explain which characteristics were indicators of developmental disorders. The server also displays the user's emotional data recognized by the emotion engine, demonstrating consideration for the reliability of the analysis results and the user's feelings.

[0587] Data compilation and consultation preparation

[0588] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links.

[0589] Specific example

[0590] This example illustrates how a user registers with the system and uploads videos of their child. First, the parent accesses the system and registers. After logging in, they upload video files of their child's daily activities. During the upload, the emotion engine analyzes the parent's emotions and detects feelings such as anxiety and impatience. The server sends this video and emotion data to an AI analysis module, and the analysis results are provided after a few minutes. If specific behaviors or gaze patterns are identified as indicators of developmental disorders, detailed analysis results and the parent's emotion data are provided along with the reasons why. Based on these results, the user can consider the next steps and prepare to consult with medical institutions or government agencies if necessary.

[0591] Examples of prompts for generative AI models

[0592] "Please send videos of children, including emotional data analyzed by EmotionEngine, to AIAnalyzer to analyze the children's behavior and risk factors. Then, please tell me how to notify the parents based on the analysis results."

[0593] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0594] Step 1:

[0595] A user accesses the system and registers by entering their username, password, and email address. The server receives the input data (username, password, email address) and compares it with existing user information in the database. After confirming that the new information does not exist in the existing database, the server adds the new user information to the database. The input for this step is user information, and the output is a confirmation message for the new user registration.

[0596] Step 2:

[0597] For existing users, login is performed using a username and password. The server receives the login information (username, password) entered by the user and compares it with the user information stored in the database. If the comparison is successful, the server returns a login success message. The input is the login information, and the output is a login success or failure message.

[0598] Step 3:

[0599] After the user logs in, they upload videos of their child or photos from their diary from their device. During upload, the file path and filename are entered by selecting the file and pressing the upload button. The server uses an emotion engine to analyze the user's facial expressions and tone of voice, and retrieves emotion data, which is saved along with the file. The input for this step is the uploaded file and emotion data, and the output is a confirmation message for saving.

[0600] Step 4:

[0601] The server sends uploaded videos and photos to an AI analysis module. The AI ​​analysis module extracts motion and gaze direction from videos and analyzes facial expressions and handwriting from photos. The input is the uploaded media file, and the output is the extracted features. The server receives the analyzed features and stores them in a database. The input and output for this step are the media file and the analysis results.

[0602] Step 5:

[0603] Based on the analysis results, the server uses AI to make a comprehensive judgment and determine the characteristics of the child's developmental disorder. The input is the features analyzed by the AI, and the output is the judgment result regarding the characteristics of the developmental disorder. The server stores this result in a database.

[0604] Step 6:

[0605] The user accesses a dedicated results display page to view the analysis results. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format along with sentiment data, and displays them. The input for this step is the user ID, and the output is the formatted analysis results.

[0606] Step 7:

[0607] Once the user obtains the analysis results, the server displays the diagnosis reasons and discovered emotional data. Furthermore, if necessary, the system compiles the data required for consultation in chronological order, generates it in a format such as PDF, and provides the user with a download link. The input for this step is the user request, and the output is the chronologically compiled information and a PDF download link.

[0608] Step 8:

[0609] In terms of specific actions, the user checks the analysis results on the terminal and prepares to consult with medical institutions or government agencies as needed. Based on this, the user decides on the next action and takes appropriate action. The input for this step is the analysis results, and the output is instructions for the appropriate next step.

[0610] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0611] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0612] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0613] [Third Embodiment]

[0614] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0615] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0616] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0617] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0618] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0619] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0620] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0621] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0622] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0623] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0624] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0625] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0626] One embodiment of the present invention is the provision of a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos and photos from their child's diary, and based on the results of AI analysis, it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0627] User registration and login

[0628] Users first access the system and register by entering information such as username, password, and email address. Existing users log in using their username and password. The server receives this information and compares it with existing user information in the database. In the case of new registration, it verifies that the entered information does not exist in the database and adds the new user information to the database. In the case of login, it verifies that the entered username and password are correct and authentication is successful.

[0629] Uploading videos and diaries

[0630] After a user logs in, they upload videos of their child or photos from their diary. When uploading, the user selects a video file or a diary photo and presses the upload button. The server receives the file and saves it to a temporary storage location. Furthermore, it verifies the file type (video or photo) and saves it to the appropriate folder. During saving, the server logs the file's save date, time, and other metadata.

[0631] AI analysis

[0632] The server sends saved videos and photos to an AI analysis module. In the case of videos, the server uses AI to extract features such as motion and gaze direction. In the case of diary photos, it analyzes facial expressions and handwriting. The AI ​​analysis module analyzes these features to determine whether characteristics of a developmental disorder are present. The server receives these analysis results and stores them in a database.

[0633] Providing analysis results and reasons for judgment

[0634] If a user wants to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. Specifically, it highlights key points such as particular actions, facial expressions, and handwriting, and explains which characteristics were indicators of developmental disorders.

[0635] Data compilation and consultation preparation

[0636] The server compiles user basic information, upload history, and analysis results in chronological order. This allows users to consult with medical institutions and government agencies more smoothly. The server generates the compiled data in formats such as PDF and provides it to the user. Users can download this data and use it for consultations.

[0637] Specific example

[0638] This example shows a user registering with the system and uploading a video of their child. First, the parent accesses the system and registers. Then, they log in and upload a video file of their child's daily activities. The server receives this video and begins AI analysis. After a few minutes, the analysis results are available, and if certain movements or gaze patterns are considered indicators of developmental disorders, detailed analysis results are provided along with the reasons why. Based on these results, the user considers the next steps and, if necessary, downloads data to prepare for consultations with medical institutions or government agencies.

[0639] This series of steps allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures.

[0640] The following describes the processing flow.

[0641] Step 1:

[0642] Users access the system and either register or log in. For new registrations, they enter a username, password, and email address. For logins, they enter their already registered username and password.

[0643] Step 2:

[0644] The server receives the entered information and checks if the username exists in the existing database. For new registrations, it verifies that the username does not exist and then adds the registration information to the database. For logins, it verifies that the username and password match and performs authentication.

[0645] Step 3:

[0646] After logging in, the user uploads video files or photos from their diary. They select the file to upload and press the upload button.

[0647] Step 4:

[0648] The server receives the uploaded file and checks its type (video or photo). It saves the file to a temporary storage location. Then, it moves the file to the appropriate folder and logs metadata including the save date and time and user information.

[0649] Step 5:

[0650] The server sends stored videos and photos to an AI analysis module. For videos, features such as motion and gaze direction are extracted. For photos, facial expressions and handwriting are analyzed.

[0651] Step 6:

[0652] The server receives the analysis results from the AI ​​analysis module and determines whether characteristics of developmental disorders are present. These analysis results are then stored in a database. The analysis results include the date and time of analysis, information about the uploaded media, the features identified by the AI, and an overall assessment.

[0653] Step 7:

[0654] The user accesses the results page to view the analysis results. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The detailed explanation of the results clearly indicates specific points such as particular actions, facial expressions, and handwriting.

[0655] Step 8:

[0656] The server compiles the user's basic information, upload history, and analysis results in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides the user with a download link.

[0657] (Example 1)

[0658] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0659] Modern parents are seeking ways to detect potential developmental disorders in their children early and take appropriate measures. However, current systems and methods make it difficult for parents to easily check their child's developmental characteristics and consult with medical or administrative agencies based on accurate data. In particular, there is a lack of systems that can efficiently analyze various media files such as videos and photos and provide reliable results. Furthermore, clearly explaining the reasoning behind the judgments based on the analysis results and organizing the data necessary for consultations in chronological order are also important issues.

[0660] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0661] In this invention, the server includes means for parents to upload media files of their children, means for analyzing the uploaded media files using artificial intelligence, means for determining the characteristics of the child's developmental disorder based on the analysis results, means for providing the analysis results to the parents, means for clearly showing the reasons for the judgment based on the analysis, means for compiling the data necessary for consultation in chronological order, and means for users to access the system by registering and logging in. This makes it possible for parents to easily check for the possibility of their child's developmental disorder and take quick and appropriate measures based on reliable data.

[0662] "Guardian" refers to an adult who has the responsibility for raising and supervising a child.

[0663] "Media files" refer to digital data that includes visual information, such as video files and image files.

[0664] "Means of uploading" refers to the interface or function that allows users to transfer files from their devices to a server.

[0665] "Analytical artificial intelligence" refers to algorithms and software that automatically extract specific patterns and features from input data and make decisions based on the results.

[0666] "Analysis means" refers to a function that uses artificial intelligence to process uploaded media files and extract and analyze specific features.

[0667] "Means of judgment" refers to algorithms or logic used to derive specific conclusions based on analysis results.

[0668] "Means of providing" refers to functions for displaying analysis results and other information to users or distributing them in a downloadable format.

[0669] "Means of clearly demonstrating the reasoning behind the decision" refers to functions that explain the basis and causes behind the analysis results in a way that is easy for the user to understand.

[0670] "Methods for organizing data chronologically" refers to functions that organize and display data such as analysis results and upload history in chronological order.

[0671] "Method of registering and logging in" refers to the process by which a user registers their information in the system and obtains access rights after authentication.

[0672] One embodiment of the present invention is the provision of a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos and photos from their child's diary, and based on the analysis results by artificial intelligence (AI), it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0673] User registration and login

[0674] Users first access the system and register by entering information such as their username, password, and email address. They then log in using this registered information. The server stores this information in a database and authenticates the user. For new registrations, the server verifies that the entered information does not already exist in the database and adds the new user information. For existing users, the server authenticates their username and password to ensure successful login.

[0675] Uploading videos and diaries

[0676] After logging in, users upload videos of their children or photos from their diaries. This involves selecting a video file or diary photo and pressing the upload button. The server receives the file and saves it to a temporary storage location. It then stores it in the appropriate folder based on the file type and records metadata at the time of saving.

[0677] AI analysis

[0678] The server sends the saved videos and photos to the AI ​​analysis module. For videos, it extracts features such as movement and gaze direction; for photos, it analyzes facial expressions and handwriting. This analysis is performed using a generative AI model. The AI ​​analysis module analyzes the acquired features to determine if characteristics of a developmental disorder are present. The analysis results are stored in a database by the server.

[0679] Providing analysis results and reasons for judgment

[0680] When a user wishes to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database and provides them in a visually easy-to-understand format. This includes a function that explains in detail why certain actions, facial expressions, or handwriting patterns are indicators of developmental disorders.

[0681] Data compilation and consultation preparation

[0682] The server compiles user basic information, upload history, and analysis results in chronological order, thereby supporting users in easily consulting with medical and administrative agencies. The compiled data is generated in formats such as PDF and provided to users in a downloadable format.

[0683] Examples of specific prompt messages include the following:

[0684] "Please analyze the video of the child and let me know if it shows any characteristics of a developmental disorder. This video shows a 5-year-old child playing."

[0685] "I've uploaded a photo of the diary. Please analyze the writing and drawings of this child and let me know the results."

[0686] This system allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures promptly.

[0687] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0688] Step 1:

[0689] Users access the system and register by entering their username, password, and email address on the registration screen. For example, they might enter "username," "password," and "email@example.com." The server receives this information and checks it against the database to ensure there are no duplicates in existing user information. Based on the results, the server saves the new user information to the database.

[0690] Input: Username, Password, Email Address

[0691] Output: User information database registration confirmation message

[0692] Step 2:

[0693] The user logs in by entering their existing username and password on the login screen. For example, they might enter "username" and "password". The server receives this information, accesses the database, and verifies that the username and password combination is correct. If authentication is successful, the user interface is displayed.

[0694] Input: Username, Password

[0695] Output: Login success or failure message

[0696] Step 3:

[0697] After logging in, the user clicks a button to upload videos or diary photos, opening a file selection dialog. For example, they select "child_video.mp4" or "diary_image.jpg" and click the upload button. The server receives the file and saves it to a temporary storage location. Next, it verifies the file type (video or photo) and moves it to the appropriate folder. In addition, it logs the file's save date and metadata.

[0698] Input: Selected file

[0699] Output: Message indicating file save complete.

[0700] Step 4:

[0701] The server sends stored videos and photos to an artificial intelligence (AI) analysis module. For example, it sends "child_video.mp4". The server extracts motion and gaze features from the video file and analyzes facial expressions and handwriting from the photos. It uses a generative AI model to recognize specific actions and patterns.

[0702] Input: Video or photo file

[0703] Output: Extracted features and analysis results

[0704] Step 5:

[0705] The server saves the analysis results received from the AI ​​analysis module to a database. For example, if "frequent eye movement" or "waving motion" is detected, the server saves the analysis results, including these features, to the database.

[0706] Input: AI analysis results

[0707] Output: Status of saving analysis results to the database

[0708] Step 6:

[0709] The user accesses a dedicated results page to view the analysis results. The server retrieves the analysis results for the user from the database and provides them in an easy-to-understand format. This includes a detailed explanation of any specific behaviors, facial expressions, or handwriting detected, and why they may be indicators of developmental disorders.

[0710] Input: User Request

[0711] Output: Formatted analysis results

[0712] Step 7:

[0713] The server compiles user basic information, upload history, and analysis results chronologically, generating them in file formats such as PDF. Users can download and use this data. For example, it can be used as necessary documentation when consulting with medical institutions or government agencies.

[0714] Input: User information, upload history, analysis results

[0715] Output: Time-series data summary PDF file

[0716] Through the above processing steps, the system helps parents check for potential developmental disorders in their children and take prompt action based on reliable data.

[0717] (Application Example 1)

[0718] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0719] Ensuring the safety of passengers in autonomous vehicles is crucial, and it is especially necessary to enable rapid response by identifying abnormal behavior or facial expressions in real time, particularly among minors and children. Furthermore, it is essential to provide an environment where passengers with developmental disabilities can receive appropriate support.

[0720] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0721] In this invention, the server includes means for parents to upload videos or diary photos of their children, means for analyzing the uploaded videos or photos using AI, means for determining the characteristics of the child's developmental disability based on the analysis results, means for providing the analysis results to the parents, means for clearly indicating the reasons for the judgment based on the analysis, means for compiling data necessary for consultation in chronological order, means for recording the movements and facial expressions of passengers with cameras inside the vehicle, means for analyzing the recorded data in real time to detect abnormal movements or facial expressions, and means for notifying the parents' mobile devices when an abnormality is detected. This enables real-time assurance of passenger safety in autonomous vehicles and allows for appropriate support, especially for passengers with characteristics of developmental disabilities.

[0722] "Guardian" refers to an adult who is responsible for raising and protecting a child or minor.

[0723] "Video" refers to digital data that expresses movement by displaying a series of still images at regular intervals.

[0724] "Diary photos" refers to digital image files that are photographs taken of the contents of a diary recorded on paper or digital media.

[0725] "Uploading" refers to the act of transferring data from a local device to a remote system such as a server.

[0726] "Analysis means" refers to methods and devices for extracting characteristics and patterns from data and converting them into meaningful information.

[0727] "AI" stands for Artificial Intelligence, and refers to machine learning algorithms and models that automatically process specific tasks.

[0728] "Developmental disorder" refers to a condition in which a person has certain characteristics or behavioral patterns that are observed during the developmental process, and experiences specific difficulties in learning, socializing, and behavior.

[0729] A "server" refers to a computer system used for storing, processing, and providing data.

[0730] A "camera" is a device that captures light and records still images and videos.

[0731] "Dynamics" is a concept that refers to the movement and changes of people and objects.

[0732] "Facial expression" refers to the expression of emotions and intentions through the movement of facial muscles.

[0733] "Real-time" refers to a state where data collection, processing, and feedback occur instantly in accordance with real-world time.

[0734] A "mobile device" is a portable electronic device, including smartphones and tablets.

[0735] A "notification" refers to a message or warning that informs someone of a specific event or condition.

[0736] A "system" refers to a collection of interconnected components that work together to achieve a specific purpose.

[0737] The system that implements this application consists of the following steps and is carried out using various hardware and software.

[0738] User registration and login

[0739] The user first accesses the system and registers as a parent or guardian. They enter necessary information such as username, password, and email address. Existing users log in with their username and password. The server receives this information, compares it with existing user information in the database, and adds new user information to the database if necessary. During login, the system verifies that the entered information is correct and performs authentication. This allows the user to access the system.

[0740] Uploading videos and diaries

[0741] After a user logs in, parents upload videos and diary photos of their children to the system. The user selects a video file or diary photo and presses the upload button. The server receives the file and saves it to a temporary storage location. At this time, it determines the file type (video or photo) and saves it to the appropriate folder. The save date and other metadata are also recorded.

[0742] AI analysis

[0743] The server sends saved videos and photos to an AI analysis module. In the case of videos, AI is used to extract features such as motion and gaze direction. In the case of diary photos, facial expressions and handwriting are analyzed. The AI ​​analysis module analyzes these features to determine whether or not there are characteristics of a developmental disorder. The analysis results are received by the server and stored in a database.

[0744] Providing analysis results and reasons for judgment

[0745] The user accesses a dedicated results page to view the analysis results. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. It clearly indicates specific points such as certain actions, facial expressions, and handwriting, and provides a detailed explanation of why they were indicators of developmental disorders.

[0746] Data compilation and consultation preparation

[0747] The server compiles user basic information, upload history, and analysis results in chronological order. This allows users to smoothly consult with medical institutions and government agencies. The server generates the compiled data in PDF format and other formats, making it available for users to download.

[0748] Applications in autonomous vehicles

[0749] Cameras are installed inside the vehicle to record the movements and expressions of passengers, especially children, in real time. The recorded data is analyzed in real time by an AI analysis module. If abnormal behavior or expressions are detected, the server immediately sends a notification to the parent's mobile device. This ensures the safety of passengers in autonomous vehicles.

[0750] Specific example

[0751] For example, a parent can register with the system and upload videos of their child's daily life. These videos are stored on a server and analyzed by AI. Parents are notified of any specific abnormal behaviors or gaze directions indicated by the analysis results. Also, when a child is in an autonomous vehicle, the AI ​​analyzes videos taken inside the vehicle, and if it detects abnormal behavior or facial expressions, a warning notification is sent to the parent's smartphone in real time.

[0752] Example of a prompt

[0753] "Please explain how to use video and image data captured by in-car cameras to analyze passengers' facial expressions and movements in real time, thereby supporting safe autonomous driving."

[0754] Thus, by using this system, parents can not only check for characteristics of developmental disorders in their children at an early stage, but also enhance safety inside autonomous vehicles.

[0755] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0756] Step 1:

[0757] A user accesses the system to register or log in. They enter information such as their username, password, and email address, and submit it to the server. The server verifies the input data against the database; in the case of new registration, it is saved as new user information. In the case of login, authentication is performed through verification. The input is the username and password, and the output is the authentication result.

[0758] Step 2:

[0759] After the user logs in, they upload a video of their child or a photo from their diary. The user selects a video file or a photo from their diary and presses the upload button. The server receives this file and saves it to a temporary storage location. It determines the file type and saves it to the appropriate folder. The input is the video file or photo from the diary, and the output is the path to the saved file.

[0760] Step 3:

[0761] The server sends saved videos and photos to an AI analysis module. For videos, it extracts motion and gaze direction frame by frame. For diary photos, it analyzes facial expressions and handwriting features. The analysis results are received by the server and stored in a database. The input is a saved video file or diary photo, and the output is the analysis results.

[0762] Step 4:

[0763] The server formats the analysis results into a dedicated results display page and provides them to the user. It provides detailed explanations of specific actions, facial expressions, handwriting, and other points, clearly indicating the reasons why they were identified as indicators of developmental disorders. Users can view the analysis results through a browser or app. The input is the analysis result data, and the output is the formatted results display page.

[0764] Step 5:

[0765] The server compiles user basic information, upload history, and analysis results in chronological order. This allows users to generate data that facilitates consultations with medical institutions and government agencies. The time-series data is generated in formats such as PDF, making it available for users to download. The input consists of user basic information and analysis results, and the output is a data file compiled in chronological order.

[0766] Step 6:

[0767] Cameras inside the autonomous vehicle record passengers' movements and facial expressions in real time. The recorded data is sent to an AI analysis module to detect abnormalities in movement or facial expressions. If an abnormality is detected, the server sends a notification to the parent's mobile device. The input is real-time recorded video data, and the output is an abnormality detection notification.

[0768] Step 7:

[0769] The parent's mobile device receives notifications sent from the server and displays the details of the anomaly. The user can review the notification and take necessary actions quickly. The input is the notification data from the server, and the output is the displayed notification content.

[0770] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0771] One embodiment of the present invention is the provision of a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos and photos from their child's diary, and based on the analysis results by AI and an emotion engine, it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0772] User registration and login

[0773] Users first access the system and register by entering information such as username, password, and email address. Existing users log in using their username and password. The server receives this information and compares it with existing user information in the database. In the case of new registration, it verifies that the entered information does not exist in the database and adds the new user information to the database. In the case of login, it verifies that the entered username and password are correct and authentication is successful.

[0774] Uploading videos and diaries

[0775] After logging in, users upload videos of their children or photos from their diaries. When uploading, users select a video file or a photo from their diary and press the upload button. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data at the time of upload.

[0776] Collection of emotional data

[0777] The server stores emotional data acquired during the upload process. The emotion engine recognizes emotions such as anxiety, relief, and impatience from the user's facial expressions and tone of voice. This data is also included in the analysis results.

[0778] AI analysis

[0779] The server sends saved videos and photos to an AI analysis module. In the case of videos, the server uses AI to extract features such as motion and gaze direction. In the case of diary photos, it analyzes facial expressions and handwriting. The AI ​​analysis module analyzes these features and determines whether characteristics of developmental disorders are present. The server receives these analysis results and stores them in a database. The analysis results include the date and time of analysis, information about the uploaded media, features found by the AI, and an overall judgment.

[0780] Providing analysis results and reasons for judgment

[0781] When a user wishes to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The analysis results clearly indicate specific points such as actions, facial expressions, and handwriting, and explain which characteristics were indicators of developmental disorders. The server also displays the user's emotional data recognized by the emotion engine, demonstrating consideration for the reliability of the analysis results and the user's feelings.

[0782] Data compilation and consultation preparation

[0783] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links.

[0784] Specific example

[0785] This example illustrates how a user registers with the system and uploads videos of their child. First, the parent accesses the system and registers. After logging in, they upload video files of their child's daily activities. During the upload, the emotion engine analyzes the parent's emotions and detects feelings such as anxiety and impatience. The server sends this video and emotion data to an AI analysis module, and the analysis results are provided after a few minutes. If specific behaviors or gaze patterns are identified as indicators of developmental disorders, detailed analysis results and the parent's emotion data are provided along with the reasons why. Based on these results, the user can consider the next steps and prepare to consult with medical institutions or government agencies if necessary.

[0786] This series of steps allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures. Furthermore, the parents' emotional state is taken into consideration, leading to more reliable decisions.

[0787] The following describes the processing flow.

[0788] Step 1:

[0789] Users access the system and either register or log in. For new registrations, they enter a username, password, and email address. For logins, they enter their already registered username and password.

[0790] Step 2:

[0791] The server receives the entered information and checks if the username exists in the existing database. For new registrations, it verifies that the username does not exist and then adds the registration information to the database. For logins, it verifies that the username and password match and performs authentication.

[0792] Step 3:

[0793] After logging in, the user selects an option to upload a video or diary photo from the system's main screen. During the upload process, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data.

[0794] Step 4:

[0795] The device sends the selected video file or diary photo to the server. At the same time, the emotion engine also sends the user's emotion data (e.g., anxiety, relief, impatience).

[0796] Step 5:

[0797] The server saves received videos, photos, and sentiment data to a temporary storage location. Metadata (e.g., save date and time, type of uploaded file) is logged for the saved files and sentiment data.

[0798] Step 6:

[0799] The server sends stored videos and photos to an AI analysis module. In the case of videos, the server extracts features such as movement and gaze direction. In the case of photos, it performs analysis of facial expressions and handwriting.

[0800] Step 7:

[0801] The server receives the analysis results from the AI ​​analysis module and determines whether characteristics of developmental disorders are present. The analysis results are saved to a database, recording the date and time of analysis, information about the uploaded media, the characteristics found by the AI, and the overall judgment.

[0802] Step 8:

[0803] The user accesses the results page to view the analysis results. The server retrieves the analysis results for that user from the database and formats them, including sentiment data, into an easy-to-read format.

[0804] Step 9:

[0805] The server provides the user with analysis results and emotional data. The results clearly indicate specific points such as actions, facial expressions, and handwriting, and explain which characteristics were indicators of developmental disorders. The display method of the analysis results is also adjusted based on the emotional data.

[0806] Step 10:

[0807] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links.

[0808] This allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures. Furthermore, the parents' emotional state is also taken into consideration, leading to more reliable assessments.

[0809] (Example 2)

[0810] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0811] In recent years, the early detection and support of developmental disorders in children have become increasingly important, but parents have limited means to regularly check for these conditions. Furthermore, parents' own emotional states can make accurate judgments difficult. In particular, feelings of anxiety and impatience can influence parents' judgments, hindering appropriate responses. To address this challenge, a system is needed that allows parents to easily check their child's condition and receive reliable results. Additionally, there is a need for a system that takes parents' emotional states into account to enable more objective and reliable judgments.

[0812] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0813] In this invention, the server includes means for parents to transfer images of their child or articles; means for analyzing the transferred images or videos using artificial intelligence; and means for detecting the parent's emotional state and including that data in the analysis. This makes it possible for parents to routinely check for characteristics of their child's developmental disorder and to obtain analysis results that also take into account the parent's emotional state.

[0814] A "guardian" is a person who has the responsibility to support the healthy growth and development of a child and who takes care of the child's daily life and education.

[0815] "Video" refers to moving image data that records a child's actions or situation. Examples include video clips and movies.

[0816] "Article images" refer to still image files of diaries, drawings, or other documents created by children.

[0817] "Transferring" refers to the act of sending images of videos or articles collected by a parent or guardian to the system.

[0818] Artificial intelligence refers to software programs that include algorithms and models used to analyze images in videos and articles. This allows for the recognition of specific patterns and features.

[0819] "Analysis methods" refer to methods that use artificial intelligence to extract characteristics from transmitted video and article images and determine the possibility of developmental disorders.

[0820] "Emotional state" refers to the psychological state recognized by analyzing the caregiver's facial expressions and tone of voice. Examples include anxiety and reassurance.

[0821] "Electronic file format" refers to data stored in a digital format, and includes, for example, PDFs and electronic spreadsheet files.

[0822] One embodiment of the present invention is to provide a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos or images from articles about their child, and based on the analysis results by artificial intelligence (AI) and an emotion engine, it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0823] This system involves multiple steps, including user registration and login, uploading videos and articles, collecting sentiment data, AI analysis, providing analysis results and reasoning, and summarizing data and preparing for consultations.

[0824] System Overview

[0825] The user first accesses the system and registers by entering information such as username, password, and email address. The terminal displays an interface for entering this information, and the server receives this information and compares it with existing user information in the database. In the case of new registration, the system verifies that the entered information does not exist in the existing database and adds the new user information to the database. In the case of login, the system verifies that the entered username and password are correct and authentication is successful.

[0826] After logging in, users upload videos of their children or images from articles. The device displays a file selection dialog, and the user selects the file to upload and presses the upload button. At this time, the emotion engine analyzes the user's emotions from their facial expressions and voice, and obtains emotion data at the time of upload. The server saves the emotion data simultaneously with the upload.

[0827] The server then sends the saved videos and images to an AI analysis module. This AI analysis module extracts features such as motion and gaze direction in the case of video data, and analyzes facial expressions and handwriting in the case of images in articles. Based on these features, the AI ​​analysis module determines whether or not characteristics of a developmental disorder are present. The server receives the analysis results and stores them in a database.

[0828] Users who wish to view their analysis results can access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The analysis results clearly indicate specific analysis points such as actions, facial expressions, and handwriting, along with the reasons why they were identified as indicators of developmental disorders. The user's emotional data, as recognized by the emotion engine, is also displayed.

[0829] Finally, the server compiles the user's basic information, upload history, analysis results, and sentiment data in chronological order. This allows the user to obtain the data necessary when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides the user with a download link.

[0830] Specific example

[0831] As a concrete example, the procedure for a user to register with the system and upload a video of their child is described. First, the parent accesses the system and registers. Then, they log in and upload a video file of their child's daily activities. During the upload, the emotion engine analyzes the parent's emotions and detects feelings such as anxiety and impatience. The server sends this video and emotion data to an AI analysis module, and the analysis results are provided after a few minutes. If specific behaviors or gaze patterns are considered indicators of developmental disorders, detailed analysis results and the parent's emotion data are provided along with the reasons why. Based on these results, the user considers the next steps and prepares to consult with medical institutions or government agencies if necessary.

[0832] This system allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures. Furthermore, the system takes into account the parents' emotional state, enabling more reliable assessments.

[0833] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0834] Step 1:

[0835] The user accesses the system and enters their username, password, and email address into the user registration form. The terminal collects this information and sends it to the server after the user presses the submit button. The server compares this input information with the existing database and, if it confirms that the user is not already registered, adds the new user information to the database. The input for this step is the username, password, and email address, and the output is the addition of the new user information to the database.

[0836] Step 2:

[0837] The user enters their username and password on the login screen. The device sends this information to the server. The server compares the entered information with the user information in the database, and if the information is correct, authentication is successful. The input for this step is the username and password, and the output is the user's authentication status. If authentication is successful, the device displays the dashboard screen to the user.

[0838] Step 3:

[0839] After the user logs in, they select an option to upload a video of their child or an image of an article. The device displays a file selection dialog, and the user selects the file to upload. After selection, the user presses the upload button, and the device sends the selected file along with the user's current facial expression and voice data to the server. The input for this step is the video or image file and the user's emotion data, and the output is the transmission of the input data to the server.

[0840] Step 4:

[0841] The server stores the image data and emotional data of the received video or article. Simultaneously, the emotion engine analyzes the user's facial expressions and tone of voice, generating and storing emotional data such as anxiety or reassurance. The input for this step is the video or image data and emotional data, and the output is the storage of this data into the database.

[0842] Step 5:

[0843] The server sends the saved video or images to an AI analysis module. The AI ​​analysis module analyzes this data and extracts features such as the child's movements, gaze direction, facial expressions, and handwriting. Based on the analysis results, it determines whether characteristics of a developmental disorder are present and sends the results to the server. The server saves these analysis results to a database. The input for this step is video or image data, and the output is the saving of the analysis results to the database.

[0844] Step 6:

[0845] The user accesses the results page to view the analysis results. The server retrieves the user's analysis results from the database and displays them on the device in a readable format. The results include analysis points such as specific actions, facial expressions, and handwriting, and the reasons why they were indicators of developmental disorders. The user's emotional data, as recognized by the emotion engine, is also displayed. The input for this step is the user's access request, and the output is the display of the analysis results and emotional data.

[0846] Step 7:

[0847] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links. Users download the generated files and submit them to the necessary institutions. The input for this step is the user's data compilation request, and the output is the generation of the compiled data in PDF format and the provision of a download link.

[0848] (Application Example 2)

[0849] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0850] Conventional systems fail to consider the emotional state of parents when assessing the characteristics of a child's developmental disorder, resulting in a lack of consideration for parental anxiety and stress. Furthermore, insufficient detailed analysis of the child's behavior and facial expressions can make accurate assessment difficult. There is a need to address these issues, enabling a more accurate and comprehensive assessment of a child's developmental disorder, and strengthening consideration for parents.

[0851] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0852] In this invention, the server includes means for parents to upload videos or diary photos of their children, means for analyzing the uploaded videos or photos using AI, means for collecting emotional data using an emotion engine that analyzes the parents' facial expressions and tone of voice, means for determining the characteristics of the child's developmental disorder based on the analysis results, means for providing the analysis results and emotional data to the parents, means for clearly indicating the reasons for the judgment based on the analysis, and means for compiling the data necessary for consultation in chronological order. This makes it possible to more accurately determine the characteristics of the child's developmental disorder and to provide results in a way that takes into account the parents' emotional state.

[0853] A "guardian" is someone who is responsible for caring for and supervising a child, and for checking for characteristics of developmental disabilities.

[0854] "Children" in this system refer to minors who may exhibit characteristics of developmental disorders.

[0855] A "video" is a video file used to record a child's actions, gaze direction, and movements.

[0856] A "diary" is a document or photograph that records a child's daily life and activities.

[0857] The "means of uploading" refer to an interface that allows parents to send videos and photos from their diaries to the system.

[0858] "Analyzing AI" refers to artificial intelligence that automatically analyzes uploaded videos and photos and extracts specific features.

[0859] "Analysis method" refers to the process by which AI analyzes videos and photos from diaries and extracts features based on that analysis.

[0860] The "emotion engine" is software that analyzes the facial expressions and tone of voice of parents to collect emotional data.

[0861] "Emotional data" refers to information about the emotional state of parents collected by the emotion engine.

[0862] The "means of judgment" refer to the process of identifying and evaluating the characteristics of a child's developmental disorder based on data analyzed by AI.

[0863] "Means of provision" refers to interfaces and functions that display and provide analysis results and emotional data to parents in an easy-to-understand manner.

[0864] "Means of clearly demonstrating the reasoning behind the judgment" refers to a process of specifically explaining the characteristics that may indicate a child has a developmental disorder, based on AI analysis data.

[0865] "Methods for compiling data necessary for consultation in chronological order" refers to methods for parents to organize the necessary data chronologically and provide it to medical institutions and administrative agencies.

[0866] "Methods for extracting movement and gaze direction as features" refers to the process of analyzing a child's movement and gaze direction from a video to extract important features.

[0867] "Methods for analyzing facial expressions and handwriting" refers to the process of analyzing a child's facial expressions and handwriting from photographs in their diary, and extracting their characteristics and changes.

[0868] Modes for carrying out the invention

[0869] One embodiment of this invention is a system for parents to check for characteristics of developmental disorders in their children. The specific implementation method is described below.

[0870] User registration and login

[0871] Users first access the system and register by entering information such as username, password, and email address. Existing users log in using their username and password. The server receives this information and compares it with existing user information in the database. In the case of new registration, it verifies that the entered information does not exist in the database and adds the new user information to the database. In the case of login, it verifies that the entered username and password are correct and authentication is successful.

[0872] Uploading videos and diaries

[0873] After logging in, users upload videos of their children or photos from their diaries. When uploading, users select a video file or a photo from their diary and press the upload button. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data at the time of upload.

[0874] Collection of emotional data

[0875] The server stores emotional data acquired during the upload. The emotion engine recognizes emotions such as anxiety, relief, and impatience from the user's facial expressions and tone of voice. This data is also included in the analysis results.

[0876] AI analysis

[0877] The server sends saved videos and photos to an AI analysis module. In the case of videos, the server uses AI to extract features such as motion and gaze direction. In the case of diary photos, it analyzes facial expressions and handwriting. The AI ​​analysis module analyzes these features and determines whether characteristics of developmental disorders are present. The server receives these analysis results and stores them in a database. The analysis results include the date and time of analysis, information about the uploaded media, features found by the AI, and an overall judgment.

[0878] Providing analysis results and reasons for judgment

[0879] When a user wishes to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The analysis results clearly indicate specific points such as actions, facial expressions, and handwriting, and explain which characteristics were indicators of developmental disorders. The server also displays the user's emotional data recognized by the emotion engine, demonstrating consideration for the reliability of the analysis results and the user's feelings.

[0880] Data compilation and consultation preparation

[0881] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links.

[0882] Specific example

[0883] This example illustrates how a user registers with the system and uploads videos of their child. First, the parent accesses the system and registers. After logging in, they upload video files of their child's daily activities. During the upload, the emotion engine analyzes the parent's emotions and detects feelings such as anxiety and impatience. The server sends this video and emotion data to an AI analysis module, and the analysis results are provided after a few minutes. If specific behaviors or gaze patterns are identified as indicators of developmental disorders, detailed analysis results and the parent's emotion data are provided along with the reasons why. Based on these results, the user can consider the next steps and prepare to consult with medical institutions or government agencies if necessary.

[0884] Examples of prompts for generative AI models

[0885] "Please send videos of children, including emotional data analyzed by EmotionEngine, to AIAnalyzer to analyze the children's behavior and risk factors. Then, please tell me how to notify the parents based on the analysis results."

[0886] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0887] Step 1:

[0888] A user accesses the system and registers by entering their username, password, and email address. The server receives the input data (username, password, email address) and compares it with existing user information in the database. After confirming that the new information does not exist in the existing database, the server adds the new user information to the database. The input for this step is user information, and the output is a confirmation message for the new user registration.

[0889] Step 2:

[0890] For existing users, login is performed using a username and password. The server receives the login information (username, password) entered by the user and compares it with the user information stored in the database. If the comparison is successful, the server returns a login success message. The input is the login information, and the output is a login success or failure message.

[0891] Step 3:

[0892] After the user logs in, they upload videos of their child or photos from their diary from their device. During upload, the file path and filename are entered by selecting the file and pressing the upload button. The server uses an emotion engine to analyze the user's facial expressions and tone of voice, and retrieves emotion data, which is saved along with the file. The input for this step is the uploaded file and emotion data, and the output is a confirmation message for saving.

[0893] Step 4:

[0894] The server sends uploaded videos and photos to an AI analysis module. The AI ​​analysis module extracts motion and gaze direction from videos and analyzes facial expressions and handwriting from photos. The input is the uploaded media file, and the output is the extracted features. The server receives the analyzed features and stores them in a database. The input and output for this step are the media file and the analysis results.

[0895] Step 5:

[0896] Based on the analysis results, the server uses AI to make a comprehensive judgment and determine the characteristics of the child's developmental disorder. The input is the features analyzed by the AI, and the output is the judgment result regarding the characteristics of the developmental disorder. The server stores this result in a database.

[0897] Step 6:

[0898] The user accesses a dedicated results display page to view the analysis results. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format along with sentiment data, and displays them. The input for this step is the user ID, and the output is the formatted analysis results.

[0899] Step 7:

[0900] Once the user obtains the analysis results, the server displays the diagnosis reasons and discovered emotional data. Furthermore, if necessary, the system compiles the data required for consultation in chronological order, generates it in a format such as PDF, and provides the user with a download link. The input for this step is the user request, and the output is the chronologically compiled information and a PDF download link.

[0901] Step 8:

[0902] In terms of specific actions, the user checks the analysis results on the terminal and prepares to consult with medical institutions or government agencies as needed. Based on this, the user decides on the next action and takes appropriate action. The input for this step is the analysis results, and the output is instructions for the appropriate next step.

[0903] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0904] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0905] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0906] [Fourth Embodiment]

[0907] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0908] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0909] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0910] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0911] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0912] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0913] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0914] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0915] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0916] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0917] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0918] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0919] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0920] One embodiment of the present invention is the provision of a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos and photos from their child's diary, and based on the results of AI analysis, it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0921] User registration and login

[0922] Users first access the system and register by entering information such as username, password, and email address. Existing users log in using their username and password. The server receives this information and compares it with existing user information in the database. In the case of new registration, it verifies that the entered information does not exist in the database and adds the new user information to the database. In the case of login, it verifies that the entered username and password are correct and authentication is successful.

[0923] Uploading videos and diaries

[0924] After a user logs in, they upload videos of their child or photos from their diary. When uploading, the user selects a video file or a diary photo and presses the upload button. The server receives the file and saves it to a temporary storage location. Furthermore, it verifies the file type (video or photo) and saves it to the appropriate folder. During saving, the server logs the file's save date, time, and other metadata.

[0925] AI analysis

[0926] The server sends saved videos and photos to an AI analysis module. In the case of videos, the server uses AI to extract features such as motion and gaze direction. In the case of diary photos, it analyzes facial expressions and handwriting. The AI ​​analysis module analyzes these features to determine whether characteristics of a developmental disorder are present. The server receives these analysis results and stores them in a database.

[0927] Providing analysis results and reasons for judgment

[0928] If a user wants to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. Specifically, it highlights key points such as particular actions, facial expressions, and handwriting, and explains which characteristics were indicators of developmental disorders.

[0929] Data compilation and consultation preparation

[0930] The server compiles user basic information, upload history, and analysis results in chronological order. This allows users to consult with medical institutions and government agencies more smoothly. The server generates the compiled data in formats such as PDF and provides it to the user. Users can download this data and use it for consultations.

[0931] Specific example

[0932] This example shows a user registering with the system and uploading a video of their child. First, the parent accesses the system and registers. Then, they log in and upload a video file of their child's daily activities. The server receives this video and begins AI analysis. After a few minutes, the analysis results are available, and if certain movements or gaze patterns are considered indicators of developmental disorders, detailed analysis results are provided along with the reasons why. Based on these results, the user considers the next steps and, if necessary, downloads data to prepare for consultations with medical institutions or government agencies.

[0933] This series of steps allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures.

[0934] The following describes the processing flow.

[0935] Step 1:

[0936] Users access the system and either register or log in. For new registrations, they enter a username, password, and email address. For logins, they enter their already registered username and password.

[0937] Step 2:

[0938] The server receives the entered information and checks if the username exists in the existing database. For new registrations, it verifies that the username does not exist and then adds the registration information to the database. For logins, it verifies that the username and password match and performs authentication.

[0939] Step 3:

[0940] After logging in, the user uploads video files or photos from their diary. They select the file to upload and press the upload button.

[0941] Step 4:

[0942] The server receives the uploaded file and checks its type (video or photo). It saves the file to a temporary storage location. Then, it moves the file to the appropriate folder and logs metadata including the save date and time and user information.

[0943] Step 5:

[0944] The server sends stored videos and photos to an AI analysis module. For videos, features such as motion and gaze direction are extracted. For photos, facial expressions and handwriting are analyzed.

[0945] Step 6:

[0946] The server receives the analysis results from the AI ​​analysis module and determines whether characteristics of developmental disorders are present. These analysis results are then stored in a database. The analysis results include the date and time of analysis, information about the uploaded media, the features identified by the AI, and an overall assessment.

[0947] Step 7:

[0948] The user accesses the results page to view the analysis results. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The detailed explanation of the results clearly indicates specific points such as particular actions, facial expressions, and handwriting.

[0949] Step 8:

[0950] The server compiles the user's basic information, upload history, and analysis results in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides the user with a download link.

[0951] (Example 1)

[0952] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0953] Modern parents are seeking ways to detect potential developmental disorders in their children early and take appropriate measures. However, current systems and methods make it difficult for parents to easily check their child's developmental characteristics and consult with medical or administrative agencies based on accurate data. In particular, there is a lack of systems that can efficiently analyze various media files such as videos and photos and provide reliable results. Furthermore, clearly explaining the reasoning behind the judgments based on the analysis results and organizing the data necessary for consultations in chronological order are also important issues.

[0954] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0955] In this invention, the server includes means for parents to upload media files of their children, means for analyzing the uploaded media files using artificial intelligence, means for determining the characteristics of the child's developmental disorder based on the analysis results, means for providing the analysis results to the parents, means for clearly showing the reasons for the judgment based on the analysis, means for compiling the data necessary for consultation in chronological order, and means for users to access the system by registering and logging in. This makes it possible for parents to easily check for the possibility of their child's developmental disorder and take quick and appropriate measures based on reliable data.

[0956] "Guardian" refers to an adult who has the responsibility for raising and supervising a child.

[0957] "Media files" refer to digital data that includes visual information, such as video files and image files.

[0958] "Means of uploading" refers to the interface or function that allows users to transfer files from their devices to a server.

[0959] "Analytical artificial intelligence" refers to algorithms and software that automatically extract specific patterns and features from input data and make decisions based on the results.

[0960] "Analysis means" refers to a function that uses artificial intelligence to process uploaded media files and extract and analyze specific features.

[0961] "Means of judgment" refers to algorithms or logic used to derive specific conclusions based on analysis results.

[0962] "Means of providing" refers to functions for displaying analysis results and other information to users or distributing them in a downloadable format.

[0963] "Means of clearly demonstrating the reasoning behind the decision" refers to functions that explain the basis and causes behind the analysis results in a way that is easy for the user to understand.

[0964] "Methods for organizing data chronologically" refers to functions that organize and display data such as analysis results and upload history in chronological order.

[0965] "Method of registering and logging in" refers to the process by which a user registers their information in the system and obtains access rights after authentication.

[0966] One embodiment of the present invention is the provision of a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos and photos from their child's diary, and based on the analysis results by artificial intelligence (AI), it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[0967] User registration and login

[0968] Users first access the system and register by entering information such as their username, password, and email address. They then log in using this registered information. The server stores this information in a database and authenticates the user. For new registrations, the server verifies that the entered information does not already exist in the database and adds the new user information. For existing users, the server authenticates their username and password to ensure successful login.

[0969] Uploading videos and diaries

[0970] After logging in, users upload videos of their children or photos from their diaries. This involves selecting a video file or diary photo and pressing the upload button. The server receives the file and saves it to a temporary storage location. It then stores it in the appropriate folder based on the file type and records metadata at the time of saving.

[0971] AI analysis

[0972] The server sends the saved videos and photos to the AI ​​analysis module. For videos, it extracts features such as movement and gaze direction; for photos, it analyzes facial expressions and handwriting. This analysis is performed using a generative AI model. The AI ​​analysis module analyzes the acquired features to determine if characteristics of a developmental disorder are present. The analysis results are stored in a database by the server.

[0973] Providing analysis results and reasons for judgment

[0974] When a user wishes to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database and provides them in a visually easy-to-understand format. This includes a function that explains in detail why certain actions, facial expressions, or handwriting patterns are indicators of developmental disorders.

[0975] Data compilation and consultation preparation

[0976] The server compiles user basic information, upload history, and analysis results in chronological order, thereby supporting users in easily consulting with medical and administrative agencies. The compiled data is generated in formats such as PDF and provided to users in a downloadable format.

[0977] Examples of specific prompt messages include the following:

[0978] "Please analyze the video of the child and let me know if it shows any characteristics of a developmental disorder. This video shows a 5-year-old child playing."

[0979] "I've uploaded a photo of the diary. Please analyze the writing and drawings of this child and let me know the results."

[0980] This system allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures promptly.

[0981] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0982] Step 1:

[0983] Users access the system and register by entering their username, password, and email address on the registration screen. For example, they might enter "username," "password," and "email@example.com." The server receives this information and checks it against the database to ensure there are no duplicates in existing user information. Based on the results, the server saves the new user information to the database.

[0984] Input: Username, Password, Email Address

[0985] Output: User information database registration confirmation message

[0986] Step 2:

[0987] The user logs in by entering their existing username and password on the login screen. For example, they might enter "username" and "password". The server receives this information, accesses the database, and verifies that the username and password combination is correct. If authentication is successful, the user interface is displayed.

[0988] Input: Username, Password

[0989] Output: Login success or failure message

[0990] Step 3:

[0991] After logging in, the user clicks a button to upload videos or diary photos, opening a file selection dialog. For example, they select "child_video.mp4" or "diary_image.jpg" and click the upload button. The server receives the file and saves it to a temporary storage location. Next, it verifies the file type (video or photo) and moves it to the appropriate folder. In addition, it logs the file's save date and metadata.

[0992] Input: Selected file

[0993] Output: Message indicating file save complete.

[0994] Step 4:

[0995] The server sends stored videos and photos to an artificial intelligence (AI) analysis module. For example, it sends "child_video.mp4". The server extracts motion and gaze features from the video file and analyzes facial expressions and handwriting from the photos. It uses a generative AI model to recognize specific actions and patterns.

[0996] Input: Video or photo file

[0997] Output: Extracted features and analysis results

[0998] Step 5:

[0999] The server saves the analysis results received from the AI ​​analysis module to a database. For example, if "frequent eye movement" or "waving motion" is detected, the server saves the analysis results, including these features, to the database.

[1000] Input: AI analysis results

[1001] Output: Status of saving analysis results to the database

[1002] Step 6:

[1003] The user accesses a dedicated results page to view the analysis results. The server retrieves the analysis results for the user from the database and provides them in an easy-to-understand format. This includes a detailed explanation of any specific behaviors, facial expressions, or handwriting detected, and why they may be indicators of developmental disorders.

[1004] Input: User Request

[1005] Output: Formatted analysis results

[1006] Step 7:

[1007] The server compiles user basic information, upload history, and analysis results chronologically, generating them in file formats such as PDF. Users can download and use this data. For example, it can be used as necessary documentation when consulting with medical institutions or government agencies.

[1008] Input: User information, upload history, analysis results

[1009] Output: Time-series data summary PDF file

[1010] Through the above processing steps, the system helps parents check for potential developmental disorders in their children and take prompt action based on reliable data.

[1011] (Application Example 1)

[1012] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1013] Ensuring the safety of passengers in autonomous vehicles is crucial, and it is especially necessary to enable rapid response by identifying abnormal behavior or facial expressions in real time, particularly among minors and children. Furthermore, it is essential to provide an environment where passengers with developmental disabilities can receive appropriate support.

[1014] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1015] In this invention, the server includes means for parents to upload videos or diary photos of their children, means for analyzing the uploaded videos or photos using AI, means for determining the characteristics of the child's developmental disability based on the analysis results, means for providing the analysis results to the parents, means for clearly indicating the reasons for the judgment based on the analysis, means for compiling data necessary for consultation in chronological order, means for recording the movements and facial expressions of passengers with cameras inside the vehicle, means for analyzing the recorded data in real time to detect abnormal movements or facial expressions, and means for notifying the parents' mobile devices when an abnormality is detected. This enables real-time assurance of passenger safety in autonomous vehicles and allows for appropriate support, especially for passengers with characteristics of developmental disabilities.

[1016] "Guardian" refers to an adult who is responsible for raising and protecting a child or minor.

[1017] "Video" refers to digital data that expresses movement by displaying a series of still images at regular intervals.

[1018] "Diary photos" refers to digital image files that are photographs taken of the contents of a diary recorded on paper or digital media.

[1019] "Uploading" refers to the act of transferring data from a local device to a remote system such as a server.

[1020] "Analysis means" refers to methods and devices for extracting characteristics and patterns from data and converting them into meaningful information.

[1021] "AI" stands for Artificial Intelligence, and refers to machine learning algorithms and models that automatically process specific tasks.

[1022] "Developmental disorder" refers to a condition in which a person has certain characteristics or behavioral patterns that are observed during the developmental process, and experiences specific difficulties in learning, socializing, and behavior.

[1023] A "server" refers to a computer system used for storing, processing, and providing data.

[1024] A "camera" is a device that captures light and records still images and videos.

[1025] "Dynamics" is a concept that refers to the movement and changes of people and objects.

[1026] "Facial expression" refers to the expression of emotions and intentions through the movement of facial muscles.

[1027] "Real-time" refers to a state where data collection, processing, and feedback occur instantly in accordance with real-world time.

[1028] A "mobile device" is a portable electronic device, including smartphones and tablets.

[1029] A "notification" refers to a message or warning that informs someone of a specific event or condition.

[1030] A "system" refers to a collection of interconnected components that work together to achieve a specific purpose.

[1031] The system that implements this application consists of the following steps and is carried out using various hardware and software.

[1032] User registration and login

[1033] The user first accesses the system and registers as a parent or guardian. They enter necessary information such as username, password, and email address. Existing users log in with their username and password. The server receives this information, compares it with existing user information in the database, and adds new user information to the database if necessary. During login, the system verifies that the entered information is correct and performs authentication. This allows the user to access the system.

[1034] Uploading videos and diaries

[1035] After a user logs in, parents upload videos and diary photos of their children to the system. The user selects a video file or diary photo and presses the upload button. The server receives the file and saves it to a temporary storage location. At this time, it determines the file type (video or photo) and saves it to the appropriate folder. The save date and other metadata are also recorded.

[1036] AI analysis

[1037] The server sends saved videos and photos to an AI analysis module. In the case of videos, AI is used to extract features such as motion and gaze direction. In the case of diary photos, facial expressions and handwriting are analyzed. The AI ​​analysis module analyzes these features to determine whether or not there are characteristics of a developmental disorder. The analysis results are received by the server and stored in a database.

[1038] Providing analysis results and reasons for judgment

[1039] The user accesses a dedicated results page to view the analysis results. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. It clearly indicates specific points such as certain actions, facial expressions, and handwriting, and provides a detailed explanation of why they were indicators of developmental disorders.

[1040] Data compilation and consultation preparation

[1041] The server compiles user basic information, upload history, and analysis results in chronological order. This allows users to smoothly consult with medical institutions and government agencies. The server generates the compiled data in PDF format and other formats, making it available for users to download.

[1042] Applications in autonomous vehicles

[1043] Cameras are installed inside the vehicle to record the movements and expressions of passengers, especially children, in real time. The recorded data is analyzed in real time by an AI analysis module. If abnormal behavior or expressions are detected, the server immediately sends a notification to the parent's mobile device. This ensures the safety of passengers in autonomous vehicles.

[1044] Specific example

[1045] For example, a parent can register with the system and upload videos of their child's daily life. These videos are stored on a server and analyzed by AI. Parents are notified of any specific abnormal behaviors or gaze directions indicated by the analysis results. Also, when a child is in an autonomous vehicle, the AI ​​analyzes videos taken inside the vehicle, and if it detects abnormal behavior or facial expressions, a warning notification is sent to the parent's smartphone in real time.

[1046] Example of a prompt

[1047] "Please explain how to use video and image data captured by in-car cameras to analyze passengers' facial expressions and movements in real time, thereby supporting safe autonomous driving."

[1048] Thus, by using this system, parents can not only check for characteristics of developmental disorders in their children at an early stage, but also enhance safety inside autonomous vehicles.

[1049] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1050] Step 1:

[1051] A user accesses the system to register or log in. They enter information such as their username, password, and email address, and submit it to the server. The server verifies the input data against the database; in the case of new registration, it is saved as new user information. In the case of login, authentication is performed through verification. The input is the username and password, and the output is the authentication result.

[1052] Step 2:

[1053] After the user logs in, they upload a video of their child or a photo from their diary. The user selects a video file or a photo from their diary and presses the upload button. The server receives this file and saves it to a temporary storage location. It determines the file type and saves it to the appropriate folder. The input is the video file or photo from the diary, and the output is the path to the saved file.

[1054] Step 3:

[1055] The server sends saved videos and photos to an AI analysis module. For videos, it extracts motion and gaze direction frame by frame. For diary photos, it analyzes facial expressions and handwriting features. The analysis results are received by the server and stored in a database. The input is a saved video file or diary photo, and the output is the analysis results.

[1056] Step 4:

[1057] The server formats the analysis results into a dedicated results display page and provides them to the user. It provides detailed explanations of specific actions, facial expressions, handwriting, and other points, clearly indicating the reasons why they were identified as indicators of developmental disorders. Users can view the analysis results through a browser or app. The input is the analysis result data, and the output is the formatted results display page.

[1058] Step 5:

[1059] The server compiles user basic information, upload history, and analysis results in chronological order. This allows users to generate data that facilitates consultations with medical institutions and government agencies. The time-series data is generated in formats such as PDF, making it available for users to download. The input consists of user basic information and analysis results, and the output is a data file compiled in chronological order.

[1060] Step 6:

[1061] Cameras inside the autonomous vehicle record passengers' movements and facial expressions in real time. The recorded data is sent to an AI analysis module to detect abnormalities in movement or facial expressions. If an abnormality is detected, the server sends a notification to the parent's mobile device. The input is real-time recorded video data, and the output is an abnormality detection notification.

[1062] Step 7:

[1063] The parent's mobile device receives notifications sent from the server and displays the details of the anomaly. The user can review the notification and take necessary actions quickly. The input is the notification data from the server, and the output is the displayed notification content.

[1064] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1065] One embodiment of the present invention is the provision of a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos and photos from their child's diary, and based on the analysis results by AI and an emotion engine, it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[1066] User registration and login

[1067] Users first access the system and register by entering information such as username, password, and email address. Existing users log in using their username and password. The server receives this information and compares it with existing user information in the database. In the case of new registration, it verifies that the entered information does not exist in the database and adds the new user information to the database. In the case of login, it verifies that the entered username and password are correct and authentication is successful.

[1068] Uploading videos and diaries

[1069] After logging in, users upload videos of their children or photos from their diaries. When uploading, users select a video file or a photo from their diary and press the upload button. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data at the time of upload.

[1070] Collection of emotional data

[1071] The server stores emotional data acquired during the upload process. The emotion engine recognizes emotions such as anxiety, relief, and impatience from the user's facial expressions and tone of voice. This data is also included in the analysis results.

[1072] AI analysis

[1073] The server sends saved videos and photos to an AI analysis module. In the case of videos, the server uses AI to extract features such as motion and gaze direction. In the case of diary photos, it analyzes facial expressions and handwriting. The AI ​​analysis module analyzes these features and determines whether characteristics of developmental disorders are present. The server receives these analysis results and stores them in a database. The analysis results include the date and time of analysis, information about the uploaded media, features found by the AI, and an overall judgment.

[1074] Providing analysis results and reasons for judgment

[1075] When a user wishes to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The analysis results clearly indicate specific points such as actions, facial expressions, and handwriting, and explain which characteristics were indicators of developmental disorders. The server also displays the user's emotional data recognized by the emotion engine, demonstrating consideration for the reliability of the analysis results and the user's feelings.

[1076] Data compilation and consultation preparation

[1077] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links.

[1078] Specific example

[1079] This example illustrates how a user registers with the system and uploads videos of their child. First, the parent accesses the system and registers. After logging in, they upload video files of their child's daily activities. During the upload, the emotion engine analyzes the parent's emotions and detects feelings such as anxiety and impatience. The server sends this video and emotion data to an AI analysis module, and the analysis results are provided after a few minutes. If specific behaviors or gaze patterns are identified as indicators of developmental disorders, detailed analysis results and the parent's emotion data are provided along with the reasons why. Based on these results, the user can consider the next steps and prepare to consult with medical institutions or government agencies if necessary.

[1080] This series of steps allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures. Furthermore, the parents' emotional state is taken into consideration, leading to more reliable decisions.

[1081] The following describes the processing flow.

[1082] Step 1:

[1083] Users access the system and either register or log in. For new registrations, they enter a username, password, and email address. For logins, they enter their already registered username and password.

[1084] Step 2:

[1085] The server receives the entered information and checks if the username exists in the existing database. For new registrations, it verifies that the username does not exist and then adds the registration information to the database. For logins, it verifies that the username and password match and performs authentication.

[1086] Step 3:

[1087] After logging in, the user selects an option to upload a video or diary photo from the system's main screen. During the upload process, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data.

[1088] Step 4:

[1089] The device sends the selected video file or diary photo to the server. At the same time, the emotion engine also sends the user's emotion data (e.g., anxiety, relief, impatience).

[1090] Step 5:

[1091] The server saves received videos, photos, and sentiment data to a temporary storage location. Metadata (e.g., save date and time, type of uploaded file) is logged for the saved files and sentiment data.

[1092] Step 6:

[1093] The server sends stored videos and photos to an AI analysis module. In the case of videos, the server extracts features such as movement and gaze direction. In the case of photos, it performs analysis of facial expressions and handwriting.

[1094] Step 7:

[1095] The server receives the analysis results from the AI ​​analysis module and determines whether characteristics of developmental disorders are present. The analysis results are saved to a database, recording the date and time of analysis, information about the uploaded media, the characteristics found by the AI, and the overall judgment.

[1096] Step 8:

[1097] The user accesses the results page to view the analysis results. The server retrieves the analysis results for that user from the database and formats them, including sentiment data, into an easy-to-read format.

[1098] Step 9:

[1099] The server provides the user with analysis results and emotional data. The results clearly indicate specific points such as actions, facial expressions, and handwriting, and explain which characteristics were indicators of developmental disorders. The display method of the analysis results is also adjusted based on the emotional data.

[1100] Step 10:

[1101] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links.

[1102] This allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures. Furthermore, the parents' emotional state is also taken into consideration, leading to more reliable assessments.

[1103] (Example 2)

[1104] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1105] In recent years, the early detection and support of developmental disorders in children have become increasingly important, but parents have limited means to regularly check for these conditions. Furthermore, parents' own emotional states can make accurate judgments difficult. In particular, feelings of anxiety and impatience can influence parents' judgments, hindering appropriate responses. To address this challenge, a system is needed that allows parents to easily check their child's condition and receive reliable results. Additionally, there is a need for a system that takes parents' emotional states into account to enable more objective and reliable judgments.

[1106] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1107] In this invention, the server includes means for parents to transfer images of their child or articles; means for analyzing the transferred images or videos using artificial intelligence; and means for detecting the parent's emotional state and including that data in the analysis. This makes it possible for parents to routinely check for characteristics of their child's developmental disorder and to obtain analysis results that also take into account the parent's emotional state.

[1108] A "guardian" is a person who has the responsibility to support the healthy growth and development of a child and who takes care of the child's daily life and education.

[1109] "Video" refers to moving image data that records a child's actions or situation. Examples include video clips and movies.

[1110] "Article images" refer to still image files of diaries, drawings, or other documents created by children.

[1111] "Transferring" refers to the act of sending images of videos or articles collected by a parent or guardian to the system.

[1112] Artificial intelligence refers to software programs that include algorithms and models used to analyze images in videos and articles. This allows for the recognition of specific patterns and features.

[1113] "Analysis methods" refer to methods that use artificial intelligence to extract characteristics from transmitted video and article images and determine the possibility of developmental disorders.

[1114] "Emotional state" refers to the psychological state recognized by analyzing the caregiver's facial expressions and tone of voice. Examples include anxiety and reassurance.

[1115] "Electronic file format" refers to data stored in a digital format, and includes, for example, PDFs and electronic spreadsheet files.

[1116] One embodiment of the present invention is to provide a system for parents to check the characteristics of their child's developmental disorder. This system allows parents to upload videos or images from articles about their child, and based on the analysis results by artificial intelligence (AI) and an emotion engine, it determines the characteristics of the developmental disorder and provides the necessary data in a compiled format.

[1117] This system involves multiple steps, including user registration and login, uploading videos and articles, collecting sentiment data, AI analysis, providing analysis results and reasoning, and summarizing data and preparing for consultations.

[1118] System Overview

[1119] The user first accesses the system and registers by entering information such as username, password, and email address. The terminal displays an interface for entering this information, and the server receives this information and compares it with existing user information in the database. In the case of new registration, the system verifies that the entered information does not exist in the existing database and adds the new user information to the database. In the case of login, the system verifies that the entered username and password are correct and authentication is successful.

[1120] After logging in, users upload videos of their children or images from articles. The device displays a file selection dialog, and the user selects the file to upload and presses the upload button. At this time, the emotion engine analyzes the user's emotions from their facial expressions and voice, and obtains emotion data at the time of upload. The server saves the emotion data simultaneously with the upload.

[1121] The server then sends the saved videos and images to an AI analysis module. This AI analysis module extracts features such as motion and gaze direction in the case of video data, and analyzes facial expressions and handwriting in the case of images in articles. Based on these features, the AI ​​analysis module determines whether or not characteristics of a developmental disorder are present. The server receives the analysis results and stores them in a database.

[1122] Users who wish to view their analysis results can access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The analysis results clearly indicate specific analysis points such as actions, facial expressions, and handwriting, along with the reasons why they were identified as indicators of developmental disorders. The user's emotional data, as recognized by the emotion engine, is also displayed.

[1123] Finally, the server compiles the user's basic information, upload history, analysis results, and sentiment data in chronological order. This allows the user to obtain the data necessary when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides the user with a download link.

[1124] Specific example

[1125] As a concrete example, the procedure for a user to register with the system and upload a video of their child is described. First, the parent accesses the system and registers. Then, they log in and upload a video file of their child's daily activities. During the upload, the emotion engine analyzes the parent's emotions and detects feelings such as anxiety and impatience. The server sends this video and emotion data to an AI analysis module, and the analysis results are provided after a few minutes. If specific behaviors or gaze patterns are considered indicators of developmental disorders, detailed analysis results and the parent's emotion data are provided along with the reasons why. Based on these results, the user considers the next steps and prepares to consult with medical institutions or government agencies if necessary.

[1126] This system allows parents to easily and effectively check for potential developmental disorders in their children and take necessary measures. Furthermore, the system takes into account the parents' emotional state, enabling more reliable assessments.

[1127] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1128] Step 1:

[1129] The user accesses the system and enters their username, password, and email address into the user registration form. The terminal collects this information and sends it to the server after the user presses the submit button. The server compares this input information with the existing database and, if it confirms that the user is not already registered, adds the new user information to the database. The input for this step is the username, password, and email address, and the output is the addition of the new user information to the database.

[1130] Step 2:

[1131] The user enters their username and password on the login screen. The device sends this information to the server. The server compares the entered information with the user information in the database, and if the information is correct, authentication is successful. The input for this step is the username and password, and the output is the user's authentication status. If authentication is successful, the device displays the dashboard screen to the user.

[1132] Step 3:

[1133] After the user logs in, they select an option to upload a video of their child or an image of an article. The device displays a file selection dialog, and the user selects the file to upload. After selection, the user presses the upload button, and the device sends the selected file along with the user's current facial expression and voice data to the server. The input for this step is the video or image file and the user's emotion data, and the output is the transmission of the input data to the server.

[1134] Step 4:

[1135] The server stores the image data and emotional data of the received video or article. Simultaneously, the emotion engine analyzes the user's facial expressions and tone of voice, generating and storing emotional data such as anxiety or reassurance. The input for this step is the video or image data and emotional data, and the output is the storage of this data into the database.

[1136] Step 5:

[1137] The server sends the saved video or images to an AI analysis module. The AI ​​analysis module analyzes this data and extracts features such as the child's movements, gaze direction, facial expressions, and handwriting. Based on the analysis results, it determines whether characteristics of a developmental disorder are present and sends the results to the server. The server saves these analysis results to a database. The input for this step is video or image data, and the output is the saving of the analysis results to the database.

[1138] Step 6:

[1139] The user accesses the results page to view the analysis results. The server retrieves the user's analysis results from the database and displays them on the device in a readable format. The results include analysis points such as specific actions, facial expressions, and handwriting, and the reasons why they were indicators of developmental disorders. The user's emotional data, as recognized by the emotion engine, is also displayed. The input for this step is the user's access request, and the output is the display of the analysis results and emotional data.

[1140] Step 7:

[1141] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links. Users download the generated files and submit them to the necessary institutions. The input for this step is the user's data compilation request, and the output is the generation of the compiled data in PDF format and the provision of a download link.

[1142] (Application Example 2)

[1143] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1144] Conventional systems fail to consider the emotional state of parents when assessing the characteristics of a child's developmental disorder, resulting in a lack of consideration for parental anxiety and stress. Furthermore, insufficient detailed analysis of the child's behavior and facial expressions can make accurate assessment difficult. There is a need to address these issues, enabling a more accurate and comprehensive assessment of a child's developmental disorder, and strengthening consideration for parents.

[1145] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1146] In this invention, the server includes means for parents to upload videos or diary photos of their children, means for analyzing the uploaded videos or photos using AI, means for collecting emotional data using an emotion engine that analyzes the parents' facial expressions and tone of voice, means for determining the characteristics of the child's developmental disorder based on the analysis results, means for providing the analysis results and emotional data to the parents, means for clearly indicating the reasons for the judgment based on the analysis, and means for compiling the data necessary for consultation in chronological order. This makes it possible to more accurately determine the characteristics of the child's developmental disorder and to provide results in a way that takes into account the parents' emotional state.

[1147] A "guardian" is someone who is responsible for caring for and supervising a child, and for checking for characteristics of developmental disabilities.

[1148] "Children" in this system refer to minors who may exhibit characteristics of developmental disorders.

[1149] A "video" is a video file used to record a child's actions, gaze direction, and movements.

[1150] A "diary" is a document or photograph that records a child's daily life and activities.

[1151] The "means of uploading" refer to an interface that allows parents to send videos and photos from their diaries to the system.

[1152] "Analyzing AI" refers to artificial intelligence that automatically analyzes uploaded videos and photos and extracts specific features.

[1153] "Analysis method" refers to the process by which AI analyzes videos and photos from diaries and extracts features based on that analysis.

[1154] The "emotion engine" is software that analyzes the facial expressions and tone of voice of parents to collect emotional data.

[1155] "Emotional data" refers to information about the emotional state of parents collected by the emotion engine.

[1156] The "means of judgment" refer to the process of identifying and evaluating the characteristics of a child's developmental disorder based on data analyzed by AI.

[1157] "Means of provision" refers to interfaces and functions that display and provide analysis results and emotional data to parents in an easy-to-understand manner.

[1158] "Means of clearly demonstrating the reasoning behind the judgment" refers to a process of specifically explaining the characteristics that may indicate a child has a developmental disorder, based on AI analysis data.

[1159] "Methods for compiling data necessary for consultation in chronological order" refers to methods for parents to organize the necessary data chronologically and provide it to medical institutions and administrative agencies.

[1160] "Methods for extracting movement and gaze direction as features" refers to the process of analyzing a child's movement and gaze direction from a video to extract important features.

[1161] "Methods for analyzing facial expressions and handwriting" refers to the process of analyzing a child's facial expressions and handwriting from photographs in their diary, and extracting their characteristics and changes.

[1162] Modes for carrying out the invention

[1163] One embodiment of this invention is a system for parents to check for characteristics of developmental disorders in their children. The specific implementation method is described below.

[1164] User registration and login

[1165] Users first access the system and register by entering information such as username, password, and email address. Existing users log in using their username and password. The server receives this information and compares it with existing user information in the database. In the case of new registration, it verifies that the entered information does not exist in the database and adds the new user information to the database. In the case of login, it verifies that the entered username and password are correct and authentication is successful.

[1166] Uploading videos and diaries

[1167] After logging in, users upload videos of their children or photos from their diaries. When uploading, users select a video file or a photo from their diary and press the upload button. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data at the time of upload.

[1168] Collection of emotional data

[1169] The server stores emotional data acquired during the upload. The emotion engine recognizes emotions such as anxiety, relief, and impatience from the user's facial expressions and tone of voice. This data is also included in the analysis results.

[1170] AI analysis

[1171] The server sends saved videos and photos to an AI analysis module. In the case of videos, the server uses AI to extract features such as motion and gaze direction. In the case of diary photos, it analyzes facial expressions and handwriting. The AI ​​analysis module analyzes these features and determines whether characteristics of developmental disorders are present. The server receives these analysis results and stores them in a database. The analysis results include the date and time of analysis, information about the uploaded media, features found by the AI, and an overall judgment.

[1172] Providing analysis results and reasons for judgment

[1173] When a user wishes to view their analysis results, they access a dedicated results display page. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format, and provides them to the user. The analysis results clearly indicate specific points such as actions, facial expressions, and handwriting, and explain which characteristics were indicators of developmental disorders. The server also displays the user's emotional data recognized by the emotion engine, demonstrating consideration for the reliability of the analysis results and the user's feelings.

[1174] Data compilation and consultation preparation

[1175] The server compiles user basic information, upload history, analysis results, and sentiment data in chronological order. This allows users to obtain the data they need when consulting with medical institutions or government agencies. The server generates this data in formats such as PDF and provides users with download links.

[1176] Specific example

[1177] This example illustrates how a user registers with the system and uploads videos of their child. First, the parent accesses the system and registers. After logging in, they upload video files of their child's daily activities. During the upload, the emotion engine analyzes the parent's emotions and detects feelings such as anxiety and impatience. The server sends this video and emotion data to an AI analysis module, and the analysis results are provided after a few minutes. If specific behaviors or gaze patterns are identified as indicators of developmental disorders, detailed analysis results and the parent's emotion data are provided along with the reasons why. Based on these results, the user can consider the next steps and prepare to consult with medical institutions or government agencies if necessary.

[1178] Examples of prompts for generative AI models

[1179] "Please send videos of children, including emotional data analyzed by EmotionEngine, to AIAnalyzer to analyze the children's behavior and risk factors. Then, please tell me how to notify the parents based on the analysis results."

[1180] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1181] Step 1:

[1182] A user accesses the system and registers by entering their username, password, and email address. The server receives the input data (username, password, email address) and compares it with existing user information in the database. After confirming that the new information does not exist in the existing database, the server adds the new user information to the database. The input for this step is user information, and the output is a confirmation message for the new user registration.

[1183] Step 2:

[1184] For existing users, login is performed using a username and password. The server receives the login information (username, password) entered by the user and compares it with the user information stored in the database. If the comparison is successful, the server returns a login success message. The input is the login information, and the output is a login success or failure message.

[1185] Step 3:

[1186] After the user logs in, they upload videos of their child or photos from their diary from their device. During upload, the file path and filename are entered by selecting the file and pressing the upload button. The server uses an emotion engine to analyze the user's facial expressions and tone of voice, and retrieves emotion data, which is saved along with the file. The input for this step is the uploaded file and emotion data, and the output is a confirmation message for saving.

[1187] Step 4:

[1188] The server sends uploaded videos and photos to an AI analysis module. The AI ​​analysis module extracts motion and gaze direction from videos and analyzes facial expressions and handwriting from photos. The input is the uploaded media file, and the output is the extracted features. The server receives the analyzed features and stores them in a database. The input and output for this step are the media file and the analysis results.

[1189] Step 5:

[1190] Based on the analysis results, the server uses AI to make a comprehensive judgment and determine the characteristics of the child's developmental disorder. The input is the features analyzed by the AI, and the output is the judgment result regarding the characteristics of the developmental disorder. The server stores this result in a database.

[1191] Step 6:

[1192] The user accesses a dedicated results display page to view the analysis results. The server retrieves the user's analysis results from the database, formats them in an easy-to-read format along with sentiment data, and displays them. The input for this step is the user ID, and the output is the formatted analysis results.

[1193] Step 7:

[1194] Once the user obtains the analysis results, the server displays the diagnosis reasons and discovered emotional data. Furthermore, if necessary, the system compiles the data required for consultation in chronological order, generates it in a format such as PDF, and provides the user with a download link. The input for this step is the user request, and the output is the chronologically compiled information and a PDF download link.

[1195] Step 8:

[1196] In terms of specific actions, the user checks the analysis results on the terminal and prepares to consult with medical institutions or government agencies as needed. Based on this, the user decides on the next action and takes appropriate action. The input for this step is the analysis results, and the output is instructions for the appropriate next step.

[1197] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1198] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1199] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1200] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1201] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1202] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1203] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1204] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1205] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1206] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1207] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1208] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1209] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1210] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1211] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1212] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1213] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1214] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1215] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1216] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1217] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1218] The following is further disclosed regarding the embodiments described above.

[1219] (Claim 1)

[1220] A means for parents to upload videos or diary photos of their children,

[1221] An AI-based analysis method for analyzing uploaded videos or photos,

[1222] The analysis revealed a means to determine the characteristics of developmental disorders in children,

[1223] Means of providing analysis results to parents,

[1224] A means to clearly show the reasoning behind the decision based on the analysis,

[1225] A method for compiling the data necessary for the consultation in chronological order,

[1226] A system that includes this.

[1227] (Claim 2)

[1228] The system according to claim 1, further comprising means for extracting motion and gaze direction as features from an uploaded video.

[1229] (Claim 3)

[1230] The system according to claim 1, further comprising means for analyzing facial expressions and handwriting from uploaded diary photos.

[1231] "Example 1"

[1232] (Claim 1)

[1233] Methods for parents to upload their children's media files,

[1234] An analysis method using artificial intelligence to analyze uploaded media files,

[1235] The analysis revealed a means to determine the characteristics of developmental disorders in children,

[1236] Means of providing analysis results to parents,

[1237] A means to clearly show the reasoning behind the decision based on the analysis,

[1238] A method for compiling the data necessary for the consultation in chronological order,

[1239] The means by which users can access the system are by registering and logging in,

[1240] A system that includes this.

[1241] (Claim 2)

[1242] A method for extracting motion and gaze direction as features from uploaded videos,

[1243] The system according to claim 1, further comprising means for sorting uploaded media files into folders based on their category.

[1244] (Claim 3)

[1245] A method for analyzing facial expressions and handwriting from uploaded diary images,

[1246] The system according to claim 1, further comprising means for formatting and providing the analysis results in an easily understandable format.

[1247] "Application Example 1"

[1248] (Claim 1)

[1249] A means for parents to upload videos or diary photos of their children,

[1250] An AI-based analysis method for analyzing uploaded videos or photos,

[1251] The analysis revealed a means to determine the characteristics of developmental disorders in children,

[1252] Means of providing analysis results to parents,

[1253] A means to clearly show the reasoning behind the decision based on the analysis,

[1254] A method for compiling the data necessary for the consultation in chronological order,

[1255] A means of recording passengers' movements and expressions using cameras inside the vehicle,

[1256] A means of analyzing recorded data in real time to detect abnormal movements and facial expressions,

[1257] A means of notifying the parent's mobile device if an abnormality is detected,

[1258] A system that includes this.

[1259] (Claim 2)

[1260] The system according to claim 1, further comprising means for extracting motion and gaze direction as features from an uploaded video.

[1261] (Claim 3)

[1262] The system according to claim 1, further comprising means for analyzing facial expressions and handwriting from uploaded diary photos.

[1263] "Example 2 of combining an emotion engine"

[1264] (Claim 1)

[1265] Means for parents to transfer images of their children or articles,

[1266] An analysis method using artificial intelligence to analyze transmitted video or images,

[1267] The analysis revealed a means to determine the characteristics of developmental disorders in children,

[1268] Means of providing analysis results to parents,

[1269] A means to clearly show the reasoning behind the decision based on the analysis,

[1270] A means to detect the emotional state of a parent and include that data in the analysis,

[1271] A method for compiling the data necessary for the consultation in chronological order,

[1272] Means of providing this data in electronic file format,

[1273] A system that includes this.

[1274] (Claim 2)

[1275] The system according to claim 1, comprising means for extracting motion and gaze direction as features from transmitted video.

[1276] (Claim 3)

[1277] The system according to claim 1, comprising means for analyzing facial expressions and handwriting from images of a forwarded article.

[1278] "Application example 2 when combining with an emotional engine"

[1279] (Claim 1)

[1280] A means for parents to upload videos or diary photos of their children,

[1281] An AI-based analysis method for analyzing uploaded videos or photos,

[1282] The analysis revealed a means to determine the characteristics of developmental disorders in children,

[1283] Means of providing analysis results to parents,

[1284] A means to clearly show the reasoning behind the decision based on the analysis,

[1285] A method for collecting emotional data using an emotion engine that analyzes the facial expressions and tone of voice of parents,

[1286] A means of providing parents with analysis results, including emotional data collected by the emotion engine,

[1287] A method for compiling the data necessary for the consultation in chronological order,

[1288] A system that includes this.

[1289] (Claim 2)

[1290] The system according to claim 1, further comprising means for extracting motion and gaze direction as features from an uploaded video.

[1291] (Claim 3)

[1292] The system according to claim 1, further comprising means for analyzing facial expressions and handwriting from uploaded diary photos. [Explanation of Symbols]

[1293] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for parents to upload videos or diary photos of their children, An AI-based analysis method for analyzing uploaded videos or photos, The analysis revealed a means to determine the characteristics of a child's development, Means of providing analysis results to parents, A means to clearly show the reasoning behind the decision based on the analysis, A method for compiling the data necessary for the consultation in chronological order, A system that includes this.

2. The system according to claim 1, further comprising means for extracting motion and gaze direction as features from an uploaded video.

3. The system according to claim 1, further comprising means for analyzing facial expressions and handwriting from uploaded diary photos.

Images