system

Abstract

We provide the system. [Solution] A means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using a camera mounted on the user's device, A means for evaluating the impact on the user's vision using information obtained from the aforementioned imaging device, and for automatically adjusting the display settings of the user device based on the evaluation results, A means to inspect the information users access and prevent the display of harmful information, A means for notifying the user device of usage restrictions using interactive characters or voice guides to prevent problems from occurring, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, with the increasing use of the Internet by children, problems such as vision decline, viewing of harmful information, and even parent-child troubles have become serious. Conventional approaches have not been able to comprehensively solve these problems, and direct intervention by users has often been required. Therefore, there is a need for means to provide an environment in which children can safely use the Internet while preventing parent-child troubles.

Means for Solving the Problems

[0005] This invention provides a means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using a camera equipped on the user device, and for evaluating the impact on visual acuity. Furthermore, it automatically adjusts the display settings of the user device based on the evaluation results to protect the user's eye health. It also includes means for inspecting the information the user accesses and preventing the display of harmful information. In addition, it builds a system that prevents parent-child conflicts by using interactive characters and voice guides to notify users of usage restrictions in an enjoyable way. This system enables users to use the internet safely and healthily.

[0006] A "user's device" is an electronic device that an individual uses to exchange information over the internet.

[0007] "Recording device" refers to a camera or sensor built into or connected to a device to record the user's actions or the surrounding conditions.

[0008] "Eye-gaze distance" refers to the distance from the user's eyes to the device screen and is an important factor in evaluating its impact on visual acuity.

[0009] "Posture" refers to the position and placement of a user's body while using a device, and serves as a criterion for judging the health effects of prolonged use.

[0010] "Blink count" refers to the number of times a user blinks within a certain period of time, and is used to measure the degree of eye strain.

[0011] "Ambient brightness" refers to the intensity of light in the environment in which the user is using the device, and is a factor involved in adjusting visual stimuli.

[0012] "Effects on vision" refers to any changes that may occur in the user's eye function or health as a result of using the device.

[0013] "Evaluation results" refer to information about the user's health and safety, analyzed based on the detected data.

[0014] "Display settings" refer to visual display elements such as screen brightness, contrast, and color temperature of the device, and are used to adjust the user's visual comfort.

[0015] "Harmful information" refers to web content or messages that may negatively impact user safety or privacy.

[0016] An "interactive character" refers to a virtual agent that provides information and guidance by interacting with the user.

[0017] "Audio guidance" refers to a method of conveying information to the user through voice instructions and explanations played by a device.

[0018] "Usage restrictions" refer to rules that limit usage time or access to features, set to promote safe and healthy device use.

[0019] "Parent-child conflicts" refer to family disputes and problems arising from differences of opinion or lack of understanding regarding the use of devices. [Brief explanation of the drawing]

[0020] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0021] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0022] First, the language used in the following description will be explained.

[0023] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0024] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0025] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0026] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0028] [First Embodiment]

[0029] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0030] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0033] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0036] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0040] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0041] This invention is a system for ensuring safe and healthy internet use by children on a user's device. Specifically, this system involves a terminal equipped with a camera that continuously monitors the user's behavior and surrounding environment, and a server that analyzes this data and makes adjustments as needed.

[0042] First, the device uses a camera to detect the user's gaze distance, posture, blinking frequency, and ambient light. This allows the device to understand how the user is using the device. Next, the device sends this data to a server. The server evaluates the impact on the user's eyesight based on the transmitted data. For example, if the user is too close to the device or blinking infrequently, the server may determine that this is negatively impacting their eyesight.

[0043] Based on these evaluation results, the server sends instructions to the terminal to automatically adjust the display settings. For example, if the user's surroundings are dark, the terminal will reduce the screen brightness to prevent vision deterioration. Settings are optimized in this way to protect the user's eye health.

[0044] Furthermore, the server monitors the information users access and blocks harmful content. If a user attempts to access a phishing site deemed dangerous, the server immediately blocks the access and displays a message on the device stating, "This page has been blocked because it is not safe."

[0045] Furthermore, this system uses interactive characters and voice guidance to prevent conflicts between parents and children. For example, if the device detects that the user's usage time is nearing its end, an animated character on the device will inform the user with a message such as, "It's almost break time! Let's save the game and go have a snack together!"

[0046] In this way, this system is designed to allow users to use the internet with peace of mind, promoting healthy usage and also helping with parent-child communication.

[0047] The following describes the processing flow.

[0048] Step 1:

[0049] The device uses a camera to detect the user's gaze distance, posture, blink rate, and ambient light in real time. This allows the user's device usage to be understood.

[0050] Step 2:

[0051] The terminal sends the detected data to the server. Based on the received data, the server determines whether those values ​​exceed a predetermined threshold.

[0052] Step 3:

[0053] The server performs an assessment of the impact on visual acuity and determines what adjustments are needed based on the results. If the assessment results exceed a threshold, the server instructs the terminal to adjust the display settings.

[0054] Step 4:

[0055] The device receives instructions from the server and automatically adjusts screen brightness and contrast to protect the user's eye health. It also displays notifications to encourage the user to maintain an appropriate distance from the screen.

[0056] Step 5:

[0057] The terminal monitors the information accessed by the user and reports it to the server. The server analyzes the received information through a filtering system to determine if it contains harmful information.

[0058] Step 6:

[0059] If the server detects harmful information, it sends a command to the terminal to block it. The terminal blocks access to that information and displays a notification to the user stating, "This information has been blocked for security reasons."

[0060] Step 7:

[0061] The user (in this case, an AI agent) uses interactive characters and voice guides through the device to notify them of usage limits in a fun way. When the user's usage time is nearing its end, the character will notify them with a message such as, "Time for a break!"

[0062] Through this series of processes, the terminal and server work together to support the user's safe and healthy device usage.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] In recent years, concerns have grown regarding the impact of internet use on children's vision and its safety. In particular, vision loss due to prolonged device use and access to harmful information are major issues for parents and educators. Managing screen time and communicating with children are also challenges. There is a need to comprehensively address these issues and provide an environment where children can use the internet safely and healthily.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server includes means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using an information gathering device; means for evaluating the impact on the user's vision using the information obtained from the information gathering device and automatically adjusting the display settings of the user's device based on the evaluation results; and means for monitoring the information used by the user and preventing the display of harmful information. This makes it possible to provide an environment in which children can use the internet safely and healthily.

[0068] An "information gathering device" is a device such as a sensor or camera used to detect the user's gaze distance, posture, blink rate, and ambient brightness.

[0069] "User equipment" refers to electronic devices used by users to access the internet, including smartphones, tablets, and computers.

[0070] "Interactive characters" refer to animated characters or interactive objects that operate on the screen to provide users with information visually and aurally.

[0071] "Voice guidance" is a system that conveys necessary information to users via voice, and can also be used to restrict usage or issue warnings through voice messages.

[0072] "Evaluating the impact on vision" is a process of analyzing data obtained from information gathering devices to assess the impact of device use on the user's eyesight.

[0073] "Automatic adjustment" refers to the system independently changing the settings of the user's device, taking into account the impact on the user's eyesight.

[0074] "Harmful information" refers to content or websites that are inappropriate or dangerous for users and whose display should be blocked by the system.

[0075] This invention is a system designed to provide users with a safe and secure environment for using the internet. Specifically, an information gathering device installed in the terminal detects the user's gaze distance, posture, blink rate, and ambient brightness. Using this data, the server evaluates the impact on the user's vision and automatically adjusts the terminal's display settings as needed.

[0076] The terminal continuously collects this information using hardware equipped with cameras and various sensors. The data is transmitted to a server via a secure and efficient network protocol, and the server analyzes the data using advanced analytical software. The software used here requires the ability to process data in real time.

[0077] For example, if a user's gaze is too close to the device or they are not blinking frequently enough, the server can detect this and send an instruction to the device to automatically adjust the screen brightness. Furthermore, if the ambient light around the device decreases, the screen brightness can be reduced to lessen the strain on the eyes.

[0078] Furthermore, the server monitors the websites users visit and the information they access. If it detects access to phishing sites or harmful content, it immediately blocks it and displays a message on the device saying, "This page was blocked because it is not safe."

[0079] This system manages the time children spend using the device and provides friendly notifications using interactive characters and voice guidance. For example, when usage time is nearing its end, an animated character will inform the user, "It's almost break time! Let's save the game and go have a snack together!"

[0080] A concrete example of a prompt message when using a generative AI model is, "Please describe the details of a system that automatically adjusts the brightness of a device to prevent children from damaging their eyesight." This is an example of input used to describe the functionality and features of a system to the AI ​​model.

[0081] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0082] Step 1:

[0083] The device uses cameras and sensors to detect the user's gaze distance, posture, blink rate, and ambient light. The input for this step is physical information about the user's surroundings. This allows the device to collect data on the user's device usage and format it as digital data. The output is real-time data about the user's usage.

[0084] Step 2:

[0085] The terminal sends the data obtained in step 1 to the server. The input is the usage data provided by the terminal. The server receives this data and converts it into an appropriate format for data processing. The output is user status data in an analyzable format.

[0086] Step 3:

[0087] The server analyzes the received data and evaluates its impact on the user's visual health. The input for this step is data such as gaze distance and posture transmitted from the terminal. Based on this, the server performs visual simulations and statistical analysis to assess the potential impact on the user's vision. The output is the evaluation result regarding the degree of impact on vision.

[0088] Step 4:

[0089] Based on the evaluation results, the server sends instructions to the terminal for automatic adjustment of display settings. The input is the evaluation results obtained in step 3. Based on this, the server determines the necessary changes to display settings (e.g., adjusting screen brightness and contrast) to reduce the strain on the user's eyes. The output is the specific instructions for adjusting the display settings.

[0090] Step 5:

[0091] The terminal adjusts the display settings according to instructions received from the server. The input is the adjustment instructions from the server. The terminal dynamically changes the display settings based on these instructions. The output is the terminal's screen with the adjusted display settings.

[0092] Step 6:

[0093] The server monitors the information users access and immediately blocks any phishing sites or harmful content it detects. The input is the user's access information. The server uses a filtering algorithm to detect harmful information. The output is a notification that harmful information has been blocked.

[0094] Step 7:

[0095] The terminal uses interactive characters and voice to notify the user when their usage time is ending. The input is data from the server regarding the end of usage time. To encourage the user to take a break, the terminal displays a character on the screen and provides voice guidance such as, "It's almost break time! Save your game and let's go get a snack together!" The output is a visual and auditory message for the user.

[0096] (Application Example 1)

[0097] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0098] Concerns exist regarding the safety and health impacts of internet use by children today through information processing devices. Specifically, issues include deterioration of visual health, harmful content, and inappropriate usage time. These challenges need to be addressed comprehensively.

[0099] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0100] In this invention, the server includes means for detecting the user's gaze distance, posture, blink rate, and ambient brightness; means for inspecting the information the user accesses and preventing the display of harmful information; and means for a robot to monitor the user's internet usage time and encourage appropriate breaks. This makes it possible to provide an appropriate usage environment while ensuring the safety and health of children's internet use.

[0101] "User" refers to an individual or their family member who uses this system, and especially to children who use the internet.

[0102] An "information processing device" is an electronic device used by users to obtain information or view content, and it is equipped with a camera and sensors.

[0103] A "shooting device" is a device that has a camera function to detect the user's gaze distance, posture, blink rate, and ambient brightness.

[0104] "Eye-gaze distance" refers to the physical distance between the user's eyes and the screen of the information processing device.

[0105] "Posture" refers to the user's body position and stance when using an information processing device.

[0106] "Blink count" refers to the frequency of a user's blinks within a given period of time.

[0107] "Ambient brightness" refers to the degree of brightness in the environment where the information processing device is placed.

[0108] "Display settings" refers to visual adjustments such as brightness and contrast of the content displayed on the screen of an information processing device.

[0109] An "interactive visual element" is a graphical motif or character that can visually interact with the user.

[0110] "Voice guidance" refers to a function or device that uses voice to convey instructions or information to the user.

[0111] "Usage restrictions" refer to settings that limit the time or content a user can access when using their information processing device.

[0112] "Conflict" refers to disagreements or disagreements between parents and children arising from the use of information processing devices.

[0113] "Internet usage time" refers to the total time a user spends connected to the internet through an information processing device.

[0114] A "robot" is a household machine that works in conjunction with a system installed on an information processing device to monitor and assist the user's activities.

[0115] In implementing this invention, the user's information processing device is equipped with a high-precision imaging device and sensors. This imaging device detects the user's gaze distance, posture, blink rate, and ambient brightness in real time. This data is first temporarily processed at the terminal and then transmitted to a server via the network.

[0116] On the server, advanced algorithms are executed that utilize external AI analysis tools such as Google Cloud to evaluate the impact on the user's eyesight based on the transmitted data. Machine learning models are used to assess the user's health status during data analysis. Based on these evaluation results, display settings are automatically adjusted. For example, if the ambient light is low, the brightness of the device screen can be appropriately reduced to alleviate visual strain.

[0117] Furthermore, the information users access is monitored on the server, and phishing sites and inappropriate content are blocked based on filtering criteria provided by external sources. AI-generated models are also utilized in this process. For example, if a user attempts to access a harmful site, the system immediately responds by displaying a message such as, "This page has been blocked because it is not safe."

[0118] Furthermore, interactive visual elements and audio guidance for usage restrictions are presented in a friendly manner, employing child-friendly animated characters. For example, after prolonged use, the character might prompt, "It's time to take a break!", encouraging rest and helping to maintain eye health and concentration. The robot then engages in natural interaction with the user based on this instruction.

[0119] As a concrete example, the prompt message would look like this:

[0120] "The robot should monitor the user's blinking frequency and prompt appropriate action."

[0121] "If the user is viewing the screen in a dark environment, the system will either brighten the surroundings or adjust the device's brightness."

[0122] Thus, the present invention promotes user health maintenance and safe use of the internet through cooperation between an information processing device and a server.

[0123] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0124] Step 1:

[0125] The terminal uses its built-in camera to sense the user's gaze distance, posture, blink rate, and ambient light. This data is obtained as input. The terminal then performs initial filtering on the input data, converts it to the required format, and prepares it for transmission to the server.

[0126] Step 2:

[0127] The server, upon receiving data transmitted from the terminal, processes the data using external AI analysis tools such as Google Cloud. In this processing step, a machine learning model analyzes the data to assess the user's health status. Specifically, it sets warning signals if the gaze distance is too short or the blinking frequency is insufficient. These analysis results are then output.

[0128] Step 3:

[0129] Based on the analysis results, the server sends necessary adjustment instructions to the user's information processing device. For example, if the ambient light is low, an instruction to lower the screen brightness is sent to the terminal. Based on this prompt, the terminal performs the action.

[0130] Step 4:

[0131] The server monitors the information users are trying to access. It applies filtering criteria provided by external sources to detect phishing sites and harmful content. This process uses an AI model to validate rules for blocking information deemed harmful, ensuring that such information is not output.

[0132] Step 5:

[0133] If a user uses the information processing device for a certain period of time, the server issues a command to send interactive visual elements and audio guidance to the terminal. Specifically, a robot displays an animated character and delivers an audio message saying, "Let's take a break." This guidance is provided in a way that is friendly and approachable to the user.

[0134] Step 6:

[0135] Based on feedback from the device, users adjust their actions, such as taking breaks. This step aims to elicit spontaneous actions from users by providing visual and auditory information. In particular, it is the moment when users hear a prompt like "Let's take a break" and decide whether or not to actually put the device down.

[0136] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0137] This invention aims to further consider the user's mental and physical health by incorporating an emotion engine that recognizes the user's emotions into a system that supports safe and healthy internet use on user devices. This system works in conjunction with the device's camera and emotion engine to evaluate the user's emotional state based on their gaze distance, posture, blinking frequency, ambient light, as well as their facial expressions and tone of voice.

[0138] First, the device not only collects data on the user's behavior and surrounding environment using a camera, but also activates an emotion engine to detect changes in the user's facial expressions and voice. This allows the device to understand the user's emotional state in real time. For example, if the user is nervous, the emotion engine will detect this from their facial expressions and tone of voice.

[0139] Next, the server receives the data sent from the device and analyzes it using multiple evaluation criteria. The server uses algorithms to assess the impact on the user's vision and emotional state, and instructs the user device to adjust display settings and notification methods as needed. For example, if the user is experiencing stress, the screen display may be changed to calmer colors, and notifications may be sent in a way that minimizes stress.

[0140] Furthermore, the server monitors the information users access and automatically blocks any harmful content. The emotion engine can also immediately issue a warning if it detects an abnormal change in the user's emotions. For example, if a user's emotions suddenly become unstable, the device will display a message such as "For your safety, this content will no longer be accessible," demonstrating consideration for the user's feelings.

[0141] Furthermore, the user (AI agent) uses interactive characters and voice guides to notify the user of usage restrictions in a way that is appropriate to their emotional state. If the emotion engine detects "joy," it will notify the user of usage restrictions with a positive message, thereby preventing conflicts between parents and children.

[0142] As described above, this system combines an emotion engine and is optimized to protect both the user's vision and emotions, enabling safe and healthy internet use.

[0143] The following describes the processing flow.

[0144] Step 1:

[0145] The device uses a camera to detect the user's gaze distance, posture, blink rate, and ambient light, while also activating an emotion engine to analyze the user's facial expressions and voice tone. This allows the system to understand the user's device usage and emotional state.

[0146] Step 2:

[0147] The device sends the collected data to the server. The server uses the received information to evaluate the impact on the user's vision and emotional state. It combines the gaze distance data with the results of the facial expression analysis to generate an evaluation result.

[0148] Step 3:

[0149] Based on the evaluation results, the server sends instructions to adjust display settings and notification methods for the user's device. For example, if the user is experiencing stress, the server instructs the device to change the screen to calming colors and play soothing audio guidance.

[0150] Step 4:

[0151] The device automatically adjusts display settings according to server instructions. Furthermore, if the emotion engine detects tension or anxiety, it provides relaxing content and messages to stabilize the user's emotions. For example, it might play an audio message such as, "Take a deep breath and relax."

[0152] Step 5:

[0153] The terminal continues to monitor the information the user accesses and reports it to the server. The server filters the received information and automatically blocks access if harmful information is detected.

[0154] Step 6:

[0155] The server continuously monitors the user's emotional state and, if it detects an unexpected change in emotion, sends a warning to the device. The device then notifies the user that "for safety reasons, viewing of the content has been stopped."

[0156] Step 7:

[0157] Users are notified of usage restrictions in a fun way through interactive characters and voice guides. For example, if an emotion of joy is detected, a positive message such as "Your smile is radiant!" is displayed, which helps to alleviate conflicts between parents and children.

[0158] This allows the terminal and server to work together to protect the user's health and emotions, ensuring safe and secure internet use.

[0159] (Example 2)

[0160] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0161] The widespread use of digital devices in modern society raises concerns about their impact on users' vision and emotional health. In particular, there is a need to mitigate the risks of exposure to harmful information when users access the internet, as well as the health effects of improper posture and eye movements while using devices. Furthermore, optimizing internet use to suit each user's individual emotional state is essential.

[0162] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0163] In this invention, the server includes means for detecting the user's gaze, posture, facial expressions, and voice tone using imaging and audio input devices installed on the user's device; means for analyzing the user's emotional state using the collected information and identifying emotions in real time; and means for adjusting display and notification settings based on the analysis results to protect the user's eyesight and emotions. This promotes the user's physical and mental health and enables safe and appropriate internet use.

[0164] "Photography equipment" refers to devices that include cameras and sensors mounted on digital devices for capturing the user's gaze, posture, and facial expressions.

[0165] A "voice input device" is a device that includes a microphone for collecting the user's voice and processing it as voice data.

[0166] An "emotion engine" is a system that includes algorithms and software to analyze collected visual and auditory data and identify the user's emotional state.

[0167] "Filtering" is the process of restricting information based on criteria to detect and prevent the display of harmful content when users use the internet.

[0168] "Display settings" refer to settings that adjust the brightness, color tone, contrast, etc., of a digital device's screen according to the user's health condition.

[0169] "Notification settings" are settings that allow you to adjust how information and warnings are displayed to the user, taking into account the user's emotional state and health condition.

[0170] An "interactive character" is a digital representation that provides appropriate usage restrictions and information through interaction with the user, and includes those that function as visual or audio guides.

[0171] "Abnormal emotional change" refers to a sudden and unstable state in which the user's emotional condition changes, which is not expected in a normal usage environment.

[0172] This invention is a system designed to make internet use on users' digital devices safe and healthy. The system uses a camera and an audio input device installed on the user's device as hardware. Specifically, the camera captures the user's gaze distance, posture, and facial expressions, and the microphone collects the tone of the user's voice as audio input. As software, an emotion engine analyzes this data to identify the user's emotional state.

[0173] The device acquires real-time user data through its camera and voice input devices. The emotion engine then analyzes this data to determine the user's emotional state, such as whether they are stressed, relaxed, or happy. Based on this analysis, the server instructs the user to configure display and notification settings optimized to protect their visual and emotional health.

[0174] As a concrete example, if the emotion engine detects that a user is experiencing stress, the server instructs the device to change the screen's color tone to a calming blue and to change notifications to a gentler sound. Furthermore, if an abnormal emotional shift is detected, a warning such as "For your safety, please stop viewing this content" can be displayed quickly.

[0175] The user (AI agent) utilizes interactive characters and voice guides to notify them of appropriate usage restrictions in an emotionally responsive manner. For example, if the emotion engine detects the user's happiness, it will send a positive message such as, "You're having a wonderful day. How about taking a break now?"

[0176] Examples of prompts for the generative AI model include: "Please describe in detail the algorithm that the emotion engine uses to determine emotional states," and "Please give specific examples of what display changes the server will instruct if a stressed state is detected."

[0177] As described above, this invention takes the form of close cooperation between hardware and software in order to protect the user's eyesight and emotional state and to support healthy and safe internet use.

[0178] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0179] Step 1:

[0180] The device uses its built-in camera to capture the user's gaze distance, posture, and facial expressions, and uses its voice input device to collect voice tone. In this step, the input is real-time acquired visual and audio information, and the output is sent to the emotion engine. Specifically, the device operates its camera and microphone simultaneously, continuously collecting the necessary data while the user is using the device.

[0181] Step 2:

[0182] The server receives visual and audio data transmitted from the terminal and performs data analysis using an emotion engine. The input is the various captured data, and the output is an evaluation of the user's emotional state. Specifically, the emotion engine processes the data using statistical models and machine learning algorithms to identify emotions such as "tension," "relaxation," and "joy."

[0183] Step 3:

[0184] Based on the analysis results, the server automatically adjusts the display and notification settings, taking into account the user's eyesight and emotional state. This process takes the evaluation results of the emotion engine as input and outputs new settings to the device. Specifically, if the server determines that the user is experiencing stress, it instructs the device to change the screen to a calmer color scheme and soften notification sounds.

[0185] Step 4:

[0186] The server monitors the content users access, filters it as needed, and prevents the display of harmful information. This step involves user access patterns and evaluation criteria as input, and filtered results as output. Based on the filtering criteria, the server prevents users from accessing harmful sites and ensures their safety.

[0187] Step 5:

[0188] The user (AI agent) activates interactive characters and voice guides to notify them of usage restrictions based on their emotional state. This process takes server notification settings as input and output usage restriction notifications via voice or messages. Specifically, when the user is feeling "joy," positive messages such as "You're doing great! Shall we prepare for the next activity?" are displayed.

[0189] (Application Example 2)

[0190] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0191] In modern urban life, while internet use has become commonplace, the amount of information that may negatively affect users' eyesight and mental health is also increasing. Furthermore, if the information users access is harmful, they are more susceptible to its effects, raising concerns about the burden on their individual mental and physical health. Moreover, it is not easy to efficiently provide user-optimized and comfortable services from the vast amount of information available in urban areas. This invention aims to solve these problems.

[0192] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0193] In this invention, the server includes means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using a camera mounted on the user's device; means for evaluating the impact on the user's vision and emotional state based on the information obtained from the camera and adjusting the display settings of the user's device based on the results; means for inspecting the information accessed by the user and preventing the display of harmful information; and interactive display means for suggesting urban services based on the emotional state. This makes it possible to realize safe and comfortable internet use while considering the user's mental health.

[0194] A "shooting device" is a device used to detect the user's gaze distance, posture, blink rate, and ambient brightness, and is installed within the device.

[0195] "Information" refers to data such as the user's eye-tracking distance, posture, blinking frequency, ambient light, and data on the content the user accesses.

[0196] "Display settings" refer to settings on the device's display, including screen brightness, color tone, and contrast, which are automatically adjusted according to the user's eyesight and emotional state.

[0197] "Harmful information" refers to information that may negatively impact a user's mental health or safety, and should be filtered according to appropriate standards.

[0198] An "interactive display system" is a mechanism that allows for the exchange of information with the user and is used to suggest appropriate urban services based on the user's emotional state.

[0199] "Emotional state" refers to a psychological state evaluated based on the user's facial expressions and tone of voice, and is an important element that plays a crucial role in system operation and service delivery.

[0200] In the system implementing this invention, a camera mounted on the user device collects data on the user's gaze distance, posture, blink rate, and ambient brightness. This data is used to evaluate the impact on the user's vision and emotional state in real time.

[0201] The server receives data transferred from the imaging device and performs data analysis. This analysis uses deep learning frameworks such as TENSORFLOW® to execute algorithms that estimate the user's emotional state from their facial expressions and tone of voice. This allows for necessary adjustments to protect the user's vision and mental health.

[0202] The terminal automatically adjusts the display settings based on instructions sent from the server. For example, it adjusts the screen brightness and color tone to reduce eye strain. Furthermore, it uses interactive display methods on the device to suggest urban services and safety information based on the user's emotional state.

[0203] This system constantly monitors the information users access and immediately filters out any harmful content. Furthermore, it can send notifications to prompt appropriate action if a user's emotional state deviates from normal.

[0204] For example, if the system detects that the user is feeling sad, it can suggest nearby relaxation spots or events that could improve their mood. An example of a prompt to input into the generating AI model would be, "Please tell me how to build a recommendation model that shows what services are suggested when the user's emotional state is 'happy'."

[0205] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0206] Step 1:

[0207] The terminal uses a camera to collect environmental data such as the user's gaze distance, posture, blink rate, and ambient light. This data is temporarily stored locally and then sent to a server for analysis. The data input consists of video and sensor data acquired from the camera, while the output is organized data ready for transfer to the server.

[0208] Step 2:

[0209] The server analyzes the received data. First, it applies machine learning algorithms to evaluate the impact on the user's vision and emotional state. The input is the environmental data sent from the terminal in step 1, and the output is the analysis result, which is the assessment of the emotional state and the impact on vision. This allows for necessary adjustments to maintain the user's health. Specific technologies used include tools such as TensorFlow.

[0210] Step 3:

[0211] The terminal adjusts the display settings based on instructions from the server. For example, if it determines that the user's eyes are tired, it will lower the screen brightness and change the color tone to a warmer color. The input is the analysis result from the server in step 2, and the output is the adjusted display settings.

[0212] Step 4:

[0213] The server inspects the information accessed by the user and filters out harmful content. It selects and displays only non-harmful information from the information the user receives. The input is internet data accessed by the user, and filtered, safe data is output.

[0214] Step 5:

[0215] Users receive interactive suggestions. Here, personalized information about city services and events is suggested based on the user's emotional state. Input is the server's analysis results and filtered information, and output is notifications of services and information tailored to the user. Furthermore, new service suggestions can be generated using generative AI models.

[0216] Step 6:

[0217] The device issues a warning notification when it detects a sudden change in the user's emotional state. For example, if it senses that the user is in a state of sadness, it will present information that can provide emotional support. The input is the user's current emotional state, and the output is a warning notification or support information.

[0218] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0219] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0220] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0221] [Second Embodiment]

[0222] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0223] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0224] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0225] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0226] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0227] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0228] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0229] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0230] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0231] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0232] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0233] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0234] This invention is a system for ensuring safe and healthy internet use by children on a user's device. Specifically, this system involves a terminal equipped with a camera that continuously monitors the user's behavior and surrounding environment, and a server that analyzes this data and makes adjustments as needed.

[0235] First, the device uses a camera to detect the user's gaze distance, posture, blinking frequency, and ambient light. This allows the device to understand how the user is using the device. Next, the device sends this data to a server. The server evaluates the impact on the user's eyesight based on the transmitted data. For example, if the user is too close to the device or blinking infrequently, the server may determine that this is negatively impacting their eyesight.

[0236] Based on these evaluation results, the server sends instructions to the terminal to automatically adjust the display settings. For example, if the user's surroundings are dark, the terminal will reduce the screen brightness to prevent vision deterioration. Settings are optimized in this way to protect the user's eye health.

[0237] Furthermore, the server monitors the information users access and blocks harmful content. If a user attempts to access a phishing site deemed dangerous, the server immediately blocks the access and displays a message on the device stating, "This page has been blocked because it is not safe."

[0238] Furthermore, this system uses interactive characters and voice guidance to prevent conflicts between parents and children. For example, if the device detects that the user's usage time is nearing its end, an animated character on the device will inform the user with a message such as, "It's almost break time! Let's save the game and go have a snack together!"

[0239] In this way, this system is designed to allow users to use the internet with peace of mind, promoting healthy usage and also helping with parent-child communication.

[0240] The following describes the processing flow.

[0241] Step 1:

[0242] The device uses a camera to detect the user's gaze distance, posture, blink rate, and ambient light in real time. This allows the user's device usage to be understood.

[0243] Step 2:

[0244] The terminal sends the detected data to the server. Based on the received data, the server determines whether those values ​​exceed a predetermined threshold.

[0245] Step 3:

[0246] The server performs an assessment of the impact on visual acuity and determines what adjustments are needed based on the results. If the assessment results exceed a threshold, the server instructs the terminal to adjust the display settings.

[0247] Step 4:

[0248] The device receives instructions from the server and automatically adjusts screen brightness and contrast to protect the user's eye health. It also displays notifications to encourage the user to maintain an appropriate distance from the screen.

[0249] Step 5:

[0250] The terminal monitors the information accessed by the user and reports it to the server. The server analyzes the received information through a filtering system to determine if it contains harmful information.

[0251] Step 6:

[0252] If the server detects harmful information, it sends a command to the terminal to block it. The terminal blocks access to that information and displays a notification to the user stating, "This information has been blocked for security reasons."

[0253] Step 7:

[0254] The user (in this case, an AI agent) uses interactive characters and voice guides through the device to notify them of usage limits in a fun way. When the user's usage time is nearing its end, the character will notify them with a message such as, "Time for a break!"

[0255] Through this series of processes, the terminal and server work together to support the user's safe and healthy device usage.

[0256] (Example 1)

[0257] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0258] In recent years, concerns have grown regarding the impact of internet use on children's vision and its safety. In particular, vision loss due to prolonged device use and access to harmful information are major issues for parents and educators. Managing screen time and communicating with children are also challenges. There is a need to comprehensively address these issues and provide an environment where children can use the internet safely and healthily.

[0259] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0260] In this invention, the server includes means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using an information gathering device; means for evaluating the impact on the user's vision using the information obtained from the information gathering device and automatically adjusting the display settings of the user's device based on the evaluation results; and means for monitoring the information used by the user and preventing the display of harmful information. This makes it possible to provide an environment in which children can use the internet safely and healthily.

[0261] An "information gathering device" is a device such as a sensor or camera used to detect the user's gaze distance, posture, blink rate, and ambient brightness.

[0262] "User equipment" refers to electronic devices used by users to access the internet, including smartphones, tablets, and computers.

[0263] "Interactive characters" refer to animated characters or interactive objects that operate on the screen to provide users with information visually and aurally.

[0264] "Voice guidance" is a system that conveys necessary information to users via voice, and can also be used to restrict usage or issue warnings through voice messages.

[0265] "Evaluating the impact on vision" is a process of analyzing data obtained from information gathering devices to assess the impact of device use on the user's eyesight.

[0266] "Automatic adjustment" refers to the system independently changing the settings of the user's device, taking into account the impact on the user's eyesight.

[0267] "Harmful information" refers to content or websites that are inappropriate or dangerous for users and whose display should be blocked by the system.

[0268] This invention is a system designed to provide users with a safe and secure environment for using the internet. Specifically, an information gathering device installed in the terminal detects the user's gaze distance, posture, blink rate, and ambient brightness. Using this data, the server evaluates the impact on the user's vision and automatically adjusts the terminal's display settings as needed.

[0269] The terminal continuously collects this information using hardware equipped with cameras and various sensors. The data is transmitted to a server via a secure and efficient network protocol, and the server analyzes the data using advanced analytical software. The software used here requires the ability to process data in real time.

[0270] For example, if a user's gaze is too close to the device or they are not blinking frequently enough, the server can detect this and send an instruction to the device to automatically adjust the screen brightness. Furthermore, if the ambient light around the device decreases, the screen brightness can be reduced to lessen the strain on the eyes.

[0271] Furthermore, the server monitors the websites users visit and the information they access. If it detects access to phishing sites or harmful content, it immediately blocks it and displays a message on the device saying, "This page was blocked because it is not safe."

[0272] This system manages the time children spend using the device and provides friendly notifications using interactive characters and voice guidance. For example, when usage time is nearing its end, an animated character will inform the user, "It's almost break time! Let's save the game and go have a snack together!"

[0273] A concrete example of a prompt message when using a generative AI model is, "Please describe the details of a system that automatically adjusts the brightness of a device to prevent children from damaging their eyesight." This is an example of input used to describe the functionality and features of a system to the AI ​​model.

[0274] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0275] Step 1:

[0276] The device uses cameras and sensors to detect the user's gaze distance, posture, blink rate, and ambient light. The input for this step is physical information about the user's surroundings. This allows the device to collect data on the user's device usage and format it as digital data. The output is real-time data about the user's usage.

[0277] Step 2:

[0278] The terminal sends the data obtained in step 1 to the server. The input is the usage data provided by the terminal. The server receives this data and converts it into an appropriate format for data processing. The output is user status data in an analyzable format.

[0279] Step 3:

[0280] The server analyzes the received data and evaluates its impact on the user's visual health. The input for this step is data such as gaze distance and posture transmitted from the terminal. Based on this, the server performs visual simulations and statistical analysis to assess the potential impact on the user's vision. The output is the evaluation result regarding the degree of impact on vision.

[0281] Step 4:

[0282] Based on the evaluation results, the server sends instructions to the terminal for automatic adjustment of display settings. The input is the evaluation results obtained in step 3. Based on this, the server determines the necessary changes to display settings (e.g., adjusting screen brightness and contrast) to reduce the strain on the user's eyes. The output is the specific instructions for adjusting the display settings.

[0283] Step 5:

[0284] The terminal actually adjusts the display settings according to the instructions received from the server. The input is the adjustment instruction from the server. The terminal dynamically changes the settings of the display based on it. The output is the screen of the terminal with the adjusted display settings.

[0285] Step 6:

[0286] The server monitors the information accessed by the user and immediately blocks the information if it detects a phishing site or harmful content. The input is the user's access information. The server uses a filtering algorithm to detect harmful information. The output is a notification that the harmful information has been blocked.

[0287] Step 7:

[0288] The terminal notifies the user that the usage time is up using interactive characters or voices. The input is the data regarding the end time of usage from the server. The terminal displays a character on the screen and provides a voice guide such as "It's almost break time! Save the game and let's go have a snack together!" to encourage the user to take a break. The output is a visual and auditory message for the user.

[0289] (Application Example 1)

[0290] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0291] When modern children use the Internet through an information processing device, there are concerns about its safety and impact on health. Specifically, the deterioration of visual health, the harmfulness of content, and inappropriate usage time have become problems. It is necessary to comprehensively solve these issues.

[0292] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0293] In this invention, the server includes means for detecting the user's gaze distance, posture, blink rate, and ambient brightness; means for inspecting the information the user accesses and preventing the display of harmful information; and means for a robot to monitor the user's internet usage time and encourage appropriate breaks. This makes it possible to provide an appropriate usage environment while ensuring the safety and health of children's internet use.

[0294] "User" refers to an individual or their family member who uses this system, and especially to children who use the internet.

[0295] An "information processing device" is an electronic device used by users to obtain information or view content, and it is equipped with a camera and sensors.

[0296] A "shooting device" is a device that has a camera function to detect the user's gaze distance, posture, blink rate, and ambient brightness.

[0297] "Eye-gaze distance" refers to the physical distance between the user's eyes and the screen of the information processing device.

[0298] "Posture" refers to the user's body position and stance when using an information processing device.

[0299] "Blink count" refers to the frequency of a user's blinks within a given period of time.

[0300] "Ambient brightness" refers to the degree of brightness in the environment where the information processing device is placed.

[0301] "Display settings" refers to visual adjustments such as brightness and contrast of the content displayed on the screen of an information processing device.

[0302] The "Interactive Visual Element" refers to a graphical motif or character that can interact visually with the user.

[0303] The "Voice Guidance" refers to a function or device for transmitting instructions and information to the user using voice.

[0304] The "Usage Restriction" refers to settings that restrict the time and the content accessed regarding the use of the user's information processing device.

[0305] The "Dispute" refers to a difference or conflict in opinions between parents and children resulting from the use of the information processing device.

[0306] The "Internet Usage Time" refers to the total time that the user is connected to the Internet through the information processing device.

[0307] The "Robot" is a household machine that cooperates with the system installed in the information processing device and monitors and assists the user's activities.

[0308] In the implementation of the present invention, first, the user's information processing device is equipped with a high-precision imaging device and sensors. By this imaging device, the user's line-of-sight distance, posture, blink frequency, and ambient brightness are detected in real time. These data are first temporarily processed by the terminal and then transmitted to the server via the network.

[0309] At the server, an advanced algorithm that utilizes external AI analysis tools such as Google Cloud and evaluates the impact on the user's vision based on the transmitted data is executed. A machine learning model for evaluating the user's health condition is used for data analysis. Based on this evaluation result, the display settings are automatically adjusted. For example, when the ambient brightness is low, the visual burden can be reduced by appropriately lowering the brightness of the terminal screen.

[0310] Furthermore, the information users access is monitored on the server, and phishing sites and inappropriate content are blocked based on filtering criteria provided by external sources. AI-generated models are also utilized in this process. For example, if a user attempts to access a harmful site, the system immediately responds by displaying a message such as, "This page has been blocked because it is not safe."

[0311] Furthermore, interactive visual elements and audio guidance for usage restrictions are presented in a friendly manner, employing child-friendly animated characters. For example, after prolonged use, the character might prompt, "It's time to take a break!", encouraging rest and helping to maintain eye health and concentration. The robot then engages in natural interaction with the user based on this instruction.

[0312] As a concrete example, the prompt message would look like this:

[0313] "The robot should monitor the user's blinking frequency and prompt appropriate action."

[0314] "If the user is viewing the screen in a dark environment, the system will either brighten the surroundings or adjust the device's brightness."

[0315] Thus, the present invention promotes user health maintenance and safe use of the internet through cooperation between an information processing device and a server.

[0316] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0317] Step 1:

[0318] The terminal uses its built-in camera to sense the user's gaze distance, posture, blink rate, and ambient light. This data is obtained as input. The terminal then performs initial filtering on the input data, converts it to the required format, and prepares it for transmission to the server.

[0319] Step 2:

[0320] The server, upon receiving data transmitted from the terminal, processes the data using external AI analysis tools such as Google Cloud. In this processing step, a machine learning model analyzes the data to assess the user's health status. Specifically, it sets warning signals if the gaze distance is too short or the blinking frequency is insufficient. These analysis results are then output.

[0321] Step 3:

[0322] Based on the analysis results, the server sends necessary adjustment instructions to the user's information processing device. For example, if the ambient light is low, an instruction to lower the screen brightness is sent to the terminal. Based on this prompt, the terminal performs the action.

[0323] Step 4:

[0324] The server monitors the information users are trying to access. It applies filtering criteria provided by external sources to detect phishing sites and harmful content. This process uses an AI model to validate rules for blocking information deemed harmful, ensuring that such information is not output.

[0325] Step 5:

[0326] If a user uses the information processing device for a certain period of time, the server issues a command to send interactive visual elements and audio guidance to the terminal. Specifically, a robot displays an animated character and delivers an audio message saying, "Let's take a break." This guidance is provided in a way that is friendly and approachable to the user.

[0327] Step 6:

[0328] Based on feedback from the device, users adjust their actions, such as taking breaks. This step aims to elicit spontaneous actions from users by providing visual and auditory information. In particular, it is the moment when users hear a prompt like "Let's take a break" and decide whether or not to actually put the device down.

[0329] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0330] This invention aims to further consider the user's mental and physical health by incorporating an emotion engine that recognizes the user's emotions into a system that supports safe and healthy internet use on user devices. This system works in conjunction with the device's camera and emotion engine to evaluate the user's emotional state based on their gaze distance, posture, blinking frequency, ambient light, as well as their facial expressions and tone of voice.

[0331] First, the device not only collects data on the user's behavior and surrounding environment using a camera, but also activates an emotion engine to detect changes in the user's facial expressions and voice. This allows the device to understand the user's emotional state in real time. For example, if the user is nervous, the emotion engine will detect this from their facial expressions and tone of voice.

[0332] Next, the server receives the data sent from the device and analyzes it using multiple evaluation criteria. The server uses algorithms to assess the impact on the user's vision and emotional state, and instructs the user device to adjust display settings and notification methods as needed. For example, if the user is experiencing stress, the screen display may be changed to calmer colors, and notifications may be sent in a way that minimizes stress.

[0333] Furthermore, the server monitors the information users access and automatically blocks any harmful content. The emotion engine can also immediately issue a warning if it detects an abnormal change in the user's emotions. For example, if a user's emotions suddenly become unstable, the device will display a message such as "For your safety, this content will no longer be accessible," demonstrating consideration for the user's feelings.

[0334] Furthermore, the user (AI agent) uses interactive characters and voice guides to notify the user of usage restrictions in a way that is appropriate to their emotional state. If the emotion engine detects "joy," it will notify the user of usage restrictions with a positive message, thereby preventing conflicts between parents and children.

[0335] As described above, this system combines an emotion engine and is optimized to protect both the user's vision and emotions, enabling safe and healthy internet use.

[0336] The following describes the processing flow.

[0337] Step 1:

[0338] The device uses a camera to detect the user's gaze distance, posture, blink rate, and ambient light, while also activating an emotion engine to analyze the user's facial expressions and voice tone. This allows the system to understand the user's device usage and emotional state.

[0339] Step 2:

[0340] The device sends the collected data to the server. The server uses the received information to evaluate the impact on the user's vision and emotional state. It combines the gaze distance data with the results of the facial expression analysis to generate an evaluation result.

[0341] Step 3:

[0342] Based on the evaluation results, the server sends instructions to adjust display settings and notification methods for the user's device. For example, if the user is experiencing stress, the server instructs the device to change the screen to calming colors and play soothing audio guidance.

[0343] Step 4:

[0344] The device automatically adjusts display settings according to server instructions. Furthermore, if the emotion engine detects tension or anxiety, it provides relaxing content and messages to stabilize the user's emotions. For example, it might play an audio message such as, "Take a deep breath and relax."

[0345] Step 5:

[0346] The terminal continues to monitor the information the user accesses and reports it to the server. The server filters the received information and automatically blocks access if harmful information is detected.

[0347] Step 6:

[0348] The server continuously monitors the user's emotional state and, if it detects an unexpected change in emotion, sends a warning to the device. The device then notifies the user that "for safety reasons, viewing of the content has been stopped."

[0349] Step 7:

[0350] Users are notified of usage restrictions in a fun way through interactive characters and voice guides. For example, if an emotion of joy is detected, a positive message such as "Your smile is radiant!" is displayed, which helps to alleviate conflicts between parents and children.

[0351] This allows the terminal and server to work together to protect the user's health and emotions, ensuring safe and secure internet use.

[0352] (Example 2)

[0353] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0354] The widespread use of digital devices in modern society raises concerns about their impact on users' vision and emotional health. In particular, there is a need to mitigate the risks of exposure to harmful information when users access the internet, as well as the health effects of improper posture and eye movements while using devices. Furthermore, optimizing internet use to suit each user's individual emotional state is essential.

[0355] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0356] In this invention, the server includes means for detecting the user's gaze, posture, facial expressions, and voice tone using imaging and audio input devices installed on the user's device; means for analyzing the user's emotional state using the collected information and identifying emotions in real time; and means for adjusting display and notification settings based on the analysis results to protect the user's eyesight and emotions. This promotes the user's physical and mental health and enables safe and appropriate internet use.

[0357] "Photography equipment" refers to devices that include cameras and sensors mounted on digital devices for capturing the user's gaze, posture, and facial expressions.

[0358] A "voice input device" is a device that includes a microphone for collecting the user's voice and processing it as voice data.

[0359] An "emotion engine" is a system that includes algorithms and software to analyze collected visual and auditory data and identify the user's emotional state.

[0360] "Filtering" is the process of restricting information based on criteria to detect and prevent the display of harmful content when users use the internet.

[0361] "Display settings" refer to settings that adjust the brightness, color tone, contrast, etc., of a digital device's screen according to the user's health condition.

[0362] "Notification settings" are settings that allow you to adjust how information and warnings are displayed to the user, taking into account the user's emotional state and health condition.

[0363] An "interactive character" is a digital representation that provides appropriate usage restrictions and information through interaction with the user, and includes those that function as visual or audio guides.

[0364] "Abnormal emotional change" refers to a sudden and unstable state in which the user's emotional condition changes, which is not expected in a normal usage environment.

[0365] This invention is a system designed to make internet use on users' digital devices safe and healthy. The system uses a camera and an audio input device installed on the user's device as hardware. Specifically, the camera captures the user's gaze distance, posture, and facial expressions, and the microphone collects the tone of the user's voice as audio input. As software, an emotion engine analyzes this data to identify the user's emotional state.

[0366] The device acquires real-time user data through its camera and voice input devices. The emotion engine then analyzes this data to determine the user's emotional state, such as whether they are stressed, relaxed, or happy. Based on this analysis, the server instructs the user to configure display and notification settings optimized to protect their visual and emotional health.

[0367] As a concrete example, if the emotion engine detects that a user is experiencing stress, the server instructs the device to change the screen's color tone to a calming blue and to change notifications to a gentler sound. Furthermore, if an abnormal emotional shift is detected, a warning such as "For your safety, please stop viewing this content" can be displayed quickly.

[0368] The user (AI agent) utilizes interactive characters and voice guides to notify them of appropriate usage restrictions in an emotionally responsive manner. For example, if the emotion engine detects the user's happiness, it will send a positive message such as, "You're having a wonderful day. How about taking a break now?"

[0369] Examples of prompts for the generative AI model include: "Please describe in detail the algorithm that the emotion engine uses to determine emotional states," and "Please give specific examples of what display changes the server will instruct if a stressed state is detected."

[0370] As described above, this invention takes the form of close cooperation between hardware and software in order to protect the user's eyesight and emotional state and to support healthy and safe internet use.

[0371] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0372] Step 1:

[0373] The device uses its built-in camera to capture the user's gaze distance, posture, and facial expressions, and uses its voice input device to collect voice tone. In this step, the input is real-time acquired visual and audio information, and the output is sent to the emotion engine. Specifically, the device operates its camera and microphone simultaneously, continuously collecting the necessary data while the user is using the device.

[0374] Step 2:

[0375] The server receives visual and audio data transmitted from the terminal and performs data analysis using an emotion engine. The input is the various captured data, and the output is an evaluation of the user's emotional state. Specifically, the emotion engine processes the data using statistical models and machine learning algorithms to identify emotions such as "tension," "relaxation," and "joy."

[0376] Step 3:

[0377] Based on the analysis results, the server automatically adjusts the display and notification settings, taking into account the user's eyesight and emotional state. This process takes the evaluation results of the emotion engine as input and outputs new settings to the device. Specifically, if the server determines that the user is experiencing stress, it instructs the device to change the screen to a calmer color scheme and soften notification sounds.

[0378] Step 4:

[0379] The server monitors the content users access, filters it as needed, and prevents the display of harmful information. This step involves user access patterns and evaluation criteria as input, and filtered results as output. Based on the filtering criteria, the server prevents users from accessing harmful sites and ensures their safety.

[0380] Step 5:

[0381] The user (AI agent) activates interactive characters and voice guides to notify them of usage restrictions based on their emotional state. This process takes server notification settings as input and output usage restriction notifications via voice or messages. Specifically, when the user is feeling "joy," positive messages such as "You're doing great! Shall we prepare for the next activity?" are displayed.

[0382] (Application Example 2)

[0383] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0384] In modern urban life, while internet use has become commonplace, the amount of information that may negatively affect users' eyesight and mental health is also increasing. Furthermore, if the information users access is harmful, they are more susceptible to its effects, raising concerns about the burden on their individual mental and physical health. Moreover, it is not easy to efficiently provide user-optimized and comfortable services from the vast amount of information available in urban areas. This invention aims to solve these problems.

[0385] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0386] In this invention, the server includes means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using a camera mounted on the user's device; means for evaluating the impact on the user's vision and emotional state based on the information obtained from the camera and adjusting the display settings of the user's device based on the results; means for inspecting the information accessed by the user and preventing the display of harmful information; and interactive display means for suggesting urban services based on the emotional state. This makes it possible to realize safe and comfortable internet use while considering the user's mental health.

[0387] A "shooting device" is a device used to detect the user's gaze distance, posture, blink rate, and ambient brightness, and is installed within the device.

[0388] "Information" refers to data such as the user's eye-tracking distance, posture, blinking frequency, ambient light, and data on the content the user accesses.

[0389] "Display settings" refer to settings on the device's display, including screen brightness, color tone, and contrast, which are automatically adjusted according to the user's eyesight and emotional state.

[0390] "Harmful information" refers to information that may negatively impact a user's mental health or safety, and should be filtered according to appropriate standards.

[0391] An "interactive display system" is a mechanism that allows for the exchange of information with the user and is used to suggest appropriate urban services based on the user's emotional state.

[0392] "Emotional state" refers to a psychological state evaluated based on the user's facial expressions and tone of voice, and is an important element that plays a crucial role in system operation and service delivery.

[0393] In the system implementing this invention, a camera mounted on the user device collects data on the user's gaze distance, posture, blink rate, and ambient brightness. This data is used to evaluate the impact on the user's vision and emotional state in real time.

[0394] The server receives data transferred from the imaging device and performs data analysis. This analysis uses deep learning frameworks such as TensorFlow to execute algorithms that estimate the user's emotional state from their facial expressions and voice tone. This allows for necessary adjustments to protect the user's vision and mental health.

[0395] The terminal automatically adjusts the display settings based on instructions sent from the server. For example, it adjusts the screen brightness and color tone to reduce eye strain. Furthermore, it uses interactive display methods on the device to suggest urban services and safety information based on the user's emotional state.

[0396] This system constantly monitors the information users access and immediately filters out any harmful content. Furthermore, it can send notifications to prompt appropriate action if a user's emotional state deviates from normal.

[0397] For example, if the system detects that the user is feeling sad, it can suggest nearby relaxation spots or events that could improve their mood. An example of a prompt to input into the generating AI model would be, "Please tell me how to build a recommendation model that shows what services are suggested when the user's emotional state is 'joyful'."

[0398] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0399] Step 1:

[0400] The terminal uses a camera to collect environmental data such as the user's gaze distance, posture, blink rate, and ambient light. This data is temporarily stored locally and then sent to a server for analysis. The data input consists of video and sensor data acquired from the camera, while the output is organized data ready for transfer to the server.

[0401] Step 2:

[0402] The server analyzes the received data. First, it applies machine learning algorithms to evaluate the impact on the user's vision and emotional state. The input is the environmental data sent from the terminal in step 1, and the output is the analysis result, which is the assessment of the emotional state and the impact on vision. This allows for necessary adjustments to maintain the user's health. Specific technologies used include tools such as TensorFlow.

[0403] Step 3:

[0404] The terminal adjusts the display settings based on instructions from the server. For example, if it determines that the user's eyes are tired, it will lower the screen brightness and change the color tone to a warmer color. The input is the analysis result from the server in step 2, and the output is the adjusted display settings.

[0405] Step 4:

[0406] The server inspects the information accessed by the user and filters out harmful content. It selects and displays only non-harmful information from the information the user receives. The input is internet data accessed by the user, and filtered, safe data is output.

[0407] Step 5:

[0408] Users receive interactive suggestions. Here, personalized information about city services and events is suggested based on the user's emotional state. Input is the server's analysis results and filtered information, and output is notifications of services and information tailored to the user. Furthermore, new service suggestions can be generated using generative AI models.

[0409] Step 6:

[0410] The device issues a warning notification when it detects a sudden change in the user's emotional state. For example, if it senses that the user is in a state of sadness, it will present information that can provide emotional support. The input is the user's current emotional state, and the output is a warning notification or support information.

[0411] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0412] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0413] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0414] [Third Embodiment]

[0415] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0416] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0417] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0418] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0419] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0420] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0421] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0422] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0423] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0424] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0425] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0426] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0427] This invention is a system for ensuring safe and healthy internet use by children on a user's device. Specifically, this system involves a terminal equipped with a camera that continuously monitors the user's behavior and surrounding environment, and a server that analyzes this data and makes adjustments as needed.

[0428] First, the device uses a camera to detect the user's gaze distance, posture, blinking frequency, and ambient light. This allows the device to understand how the user is using the device. Next, the device sends this data to a server. The server evaluates the impact on the user's eyesight based on the transmitted data. For example, if the user is too close to the device or blinking infrequently, the server may determine that this is negatively impacting their eyesight.

[0429] Based on these evaluation results, the server sends instructions to the terminal to automatically adjust the display settings. For example, if the user's surroundings are dark, the terminal will reduce the screen brightness to prevent vision deterioration. Settings are optimized in this way to protect the user's eye health.

[0430] Furthermore, the server monitors the information users access and blocks harmful content. If a user attempts to access a phishing site deemed dangerous, the server immediately blocks the access and displays a message on the device stating, "This page has been blocked because it is not safe."

[0431] Furthermore, this system uses interactive characters and voice guidance to prevent conflicts between parents and children. For example, if the device detects that the user's usage time is nearing its end, an animated character on the device will inform the user with a message such as, "It's almost break time! Let's save the game and go have a snack together!"

[0432] In this way, this system is designed to allow users to use the internet with peace of mind, promoting healthy usage and also helping with parent-child communication.

[0433] The following describes the processing flow.

[0434] Step 1:

[0435] The device uses a camera to detect the user's gaze distance, posture, blink rate, and ambient light in real time. This allows the user's device usage to be understood.

[0436] Step 2:

[0437] The terminal sends the detected data to the server. Based on the received data, the server determines whether those values ​​exceed a predetermined threshold.

[0438] Step 3:

[0439] The server performs an assessment of the impact on visual acuity and determines what adjustments are needed based on the results. If the assessment results exceed a threshold, the server instructs the terminal to adjust the display settings.

[0440] Step 4:

[0441] The device receives instructions from the server and automatically adjusts screen brightness and contrast to protect the user's eye health. It also displays notifications to encourage the user to maintain an appropriate distance from the screen.

[0442] Step 5:

[0443] The terminal monitors the information accessed by the user and reports it to the server. The server analyzes the received information through a filtering system to determine if it contains harmful information.

[0444] Step 6:

[0445] If the server detects harmful information, it sends a command to the terminal to block it. The terminal blocks access to that information and displays a notification to the user stating, "This information has been blocked for security reasons."

[0446] Step 7:

[0447] The user (in this case, an AI agent) uses interactive characters and voice guides through the device to notify them of usage limits in a fun way. When the user's usage time is nearing its end, the character will notify them with a message such as, "Time for a break!"

[0448] Through this series of processes, the terminal and server work together to support the user's safe and healthy device usage.

[0449] (Example 1)

[0450] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0451] In recent years, concerns have grown regarding the impact of internet use on children's vision and its safety. In particular, vision loss due to prolonged device use and access to harmful information are major issues for parents and educators. Managing screen time and communicating with children are also challenges. There is a need to comprehensively address these issues and provide an environment where children can use the internet safely and healthily.

[0452] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0453] In this invention, the server includes means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using an information gathering device; means for evaluating the impact on the user's vision using the information obtained from the information gathering device and automatically adjusting the display settings of the user's device based on the evaluation results; and means for monitoring the information used by the user and preventing the display of harmful information. This makes it possible to provide an environment in which children can use the internet safely and healthily.

[0454] An "information gathering device" is a device such as a sensor or camera used to detect the user's gaze distance, posture, blink rate, and ambient brightness.

[0455] "User equipment" refers to electronic devices used by users to access the internet, including smartphones, tablets, and computers.

[0456] "Interactive characters" refer to animated characters or interactive objects that operate on the screen to provide users with information visually and aurally.

[0457] "Voice guidance" is a system that conveys necessary information to users via voice, and can also be used to restrict usage or issue warnings through voice messages.

[0458] "Evaluating the impact on vision" is a process of analyzing data obtained from information gathering devices to assess the impact of device use on the user's eyesight.

[0459] "Automatic adjustment" refers to the system independently changing the settings of the user's device, taking into account the impact on the user's eyesight.

[0460] "Harmful information" refers to content or websites that are inappropriate or dangerous for users and whose display should be blocked by the system.

[0461] This invention is a system designed to provide users with a safe and secure environment for using the internet. Specifically, an information gathering device installed in the terminal detects the user's gaze distance, posture, blink rate, and ambient brightness. Using this data, the server evaluates the impact on the user's vision and automatically adjusts the terminal's display settings as needed.

[0462] The terminal continuously collects this information using hardware equipped with cameras and various sensors. The data is transmitted to a server via a secure and efficient network protocol, and the server analyzes the data using advanced analytical software. The software used here requires the ability to process data in real time.

[0463] For example, if a user's gaze is too close to the device or they are not blinking frequently enough, the server can detect this and send an instruction to the device to automatically adjust the screen brightness. Furthermore, if the ambient light around the device decreases, the screen brightness can be reduced to lessen the strain on the eyes.

[0464] Furthermore, the server monitors the websites users visit and the information they access. If it detects access to phishing sites or harmful content, it immediately blocks it and displays a message on the device saying, "This page was blocked because it is not safe."

[0465] This system manages the time children spend using the device and provides friendly notifications using interactive characters and voice guidance. For example, when usage time is nearing its end, an animated character will inform the user, "It's almost break time! Let's save the game and go have a snack together!"

[0466] A concrete example of a prompt message when using a generative AI model is, "Please describe the details of a system that automatically adjusts the brightness of a device to prevent children from damaging their eyesight." This is an example of input used to describe the functionality and features of a system to the AI ​​model.

[0467] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0468] Step 1:

[0469] The device uses cameras and sensors to detect the user's gaze distance, posture, blink rate, and ambient light. The input for this step is physical information about the user's surroundings. This allows the device to collect data on the user's device usage and format it as digital data. The output is real-time data about the user's usage.

[0470] Step 2:

[0471] The terminal sends the data obtained in step 1 to the server. The input is the usage data provided by the terminal. The server receives this data and converts it into an appropriate format for data processing. The output is user status data in an analyzable format.

[0472] Step 3:

[0473] The server analyzes the received data and evaluates its impact on the user's visual health. The input for this step is data such as gaze distance and posture transmitted from the terminal. Based on this, the server performs visual simulations and statistical analysis to assess the potential impact on the user's vision. The output is the evaluation result regarding the degree of impact on vision.

[0474] Step 4:

[0475] Based on the evaluation results, the server sends instructions to the terminal for automatic adjustment of display settings. The input is the evaluation results obtained in step 3. Based on this, the server determines the necessary changes to display settings (e.g., adjusting screen brightness and contrast) to reduce the strain on the user's eyes. The output is the specific instructions for adjusting the display settings.

[0476] Step 5:

[0477] The terminal adjusts the display settings according to instructions received from the server. The input is the adjustment instructions from the server. The terminal dynamically changes the display settings based on these instructions. The output is the terminal's screen with the adjusted display settings.

[0478] Step 6:

[0479] The server monitors the information users access and immediately blocks any phishing sites or harmful content it detects. The input is the user's access information. The server uses a filtering algorithm to detect harmful information. The output is a notification that harmful information has been blocked.

[0480] Step 7:

[0481] The terminal uses interactive characters and voice to notify the user when their usage time is ending. The input is data from the server regarding the end of usage time. To encourage the user to take a break, the terminal displays a character on the screen and provides voice guidance such as, "It's almost break time! Save your game and let's go get a snack together!" The output is a visual and auditory message for the user.

[0482] (Application Example 1)

[0483] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0484] Concerns exist regarding the safety and health impacts of internet use by children today through information processing devices. Specifically, issues include deterioration of visual health, harmful content, and inappropriate usage time. These challenges need to be addressed comprehensively.

[0485] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0486] In this invention, the server includes means for detecting the user's gaze distance, posture, blink rate, and ambient brightness; means for inspecting the information the user accesses and preventing the display of harmful information; and means for a robot to monitor the user's internet usage time and encourage appropriate breaks. This makes it possible to provide an appropriate usage environment while ensuring the safety and health of children's internet use.

[0487] "User" refers to an individual or their family member who uses this system, and especially to children who use the internet.

[0488] An "information processing device" is an electronic device used by users to obtain information or view content, and it is equipped with a camera and sensors.

[0489] A "shooting device" is a device that has a camera function to detect the user's gaze distance, posture, blink rate, and ambient brightness.

[0490] "Eye-gaze distance" refers to the physical distance between the user's eyes and the screen of the information processing device.

[0491] "Posture" refers to the user's body position and stance when using an information processing device.

[0492] "Blink count" refers to the frequency of a user's blinks within a given period of time.

[0493] "Ambient brightness" refers to the degree of brightness in the environment where the information processing device is placed.

[0494] "Display settings" refers to visual adjustments such as brightness and contrast of the content displayed on the screen of an information processing device.

[0495] An "interactive visual element" is a graphical motif or character that can visually interact with the user.

[0496] "Voice guidance" refers to a function or device that uses voice to convey instructions or information to the user.

[0497] "Usage restrictions" refer to settings that limit the time or content a user can access when using their information processing device.

[0498] "Conflict" refers to disagreements or disagreements between parents and children arising from the use of information processing devices.

[0499] "Internet usage time" refers to the total time a user spends connected to the internet through an information processing device.

[0500] A "robot" is a household machine that works in conjunction with a system installed on an information processing device to monitor and assist the user's activities.

[0501] In implementing this invention, the user's information processing device is equipped with a high-precision imaging device and sensors. This imaging device detects the user's gaze distance, posture, blink rate, and ambient brightness in real time. This data is first temporarily processed at the terminal and then transmitted to a server via the network.

[0502] On the server, advanced algorithms are executed that utilize external AI analysis tools such as Google Cloud to evaluate the impact on the user's eyesight based on the transmitted data. Machine learning models are used in the data analysis to assess the user's health status. Based on these evaluation results, display settings are automatically adjusted. For example, if the ambient light is low, the brightness of the device screen can be appropriately reduced to alleviate visual strain.

[0503] Furthermore, the information users access is monitored on the server, and phishing sites and inappropriate content are blocked based on filtering criteria provided by external sources. AI-generated models are also utilized in this process. For example, if a user attempts to access a harmful site, the system immediately responds by displaying a message such as, "This page has been blocked because it is not safe."

[0504] Furthermore, interactive visual elements and audio guidance for usage restrictions are presented in a friendly manner, employing child-friendly animated characters. For example, after prolonged use, the character might prompt, "It's time to take a break!", encouraging rest and helping to maintain eye health and concentration. The robot then engages in natural interaction with the user based on this instruction.

[0505] As a concrete example, the prompt message would look like this:

[0506] "The robot should monitor the user's blinking frequency and prompt appropriate action."

[0507] "If the user is viewing the screen in a dark environment, the system will either brighten the surroundings or adjust the device's brightness."

[0508] Thus, the present invention promotes user health maintenance and safe use of the internet through cooperation between an information processing device and a server.

[0509] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0510] Step 1:

[0511] The terminal uses its built-in camera to sense the user's gaze distance, posture, blink rate, and ambient light. This data is obtained as input. The terminal then performs initial filtering on the input data, converts it to the required format, and prepares it for transmission to the server.

[0512] Step 2:

[0513] The server, upon receiving data transmitted from the terminal, processes the data using external AI analysis tools such as Google Cloud. In this processing step, a machine learning model analyzes the data to assess the user's health status. Specifically, it sets warning signals if the gaze distance is too short or the blinking frequency is insufficient. These analysis results are then output.

[0514] Step 3:

[0515] Based on the analysis results, the server sends necessary adjustment instructions to the user's information processing device. For example, if the ambient light is low, an instruction to lower the screen brightness is sent to the terminal. Based on this prompt, the terminal performs the action.

[0516] Step 4:

[0517] The server monitors the information users are trying to access. It applies filtering criteria provided by external sources to detect phishing sites and harmful content. This process uses an AI model to validate rules for blocking information deemed harmful, ensuring that such information is not output.

[0518] Step 5:

[0519] If a user uses the information processing device for a certain period of time, the server issues a command to send interactive visual elements and audio guidance to the terminal. Specifically, a robot displays an animated character and delivers an audio message saying, "Let's take a break." This guidance is provided in a way that is friendly and approachable to the user.

[0520] Step 6:

[0521] Based on feedback from the device, users adjust their actions, such as taking breaks. This step aims to elicit spontaneous actions from users by providing visual and auditory information. In particular, it is the moment when users hear a prompt like "Let's take a break" and decide whether or not to actually put the device down.

[0522] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0523] This invention aims to further consider the user's mental and physical health by incorporating an emotion engine that recognizes the user's emotions into a system that supports safe and healthy internet use on user devices. This system works in conjunction with the device's camera and emotion engine to evaluate the user's emotional state based on their gaze distance, posture, blinking frequency, ambient light, as well as their facial expressions and tone of voice.

[0524] First, the device not only collects data on the user's behavior and surrounding environment using a camera, but also activates an emotion engine to detect changes in the user's facial expressions and voice. This allows the device to understand the user's emotional state in real time. For example, if the user is nervous, the emotion engine will detect this from their facial expressions and tone of voice.

[0525] Next, the server receives the data sent from the device and analyzes it using multiple evaluation criteria. The server uses algorithms to assess the impact on the user's vision and emotional state, and instructs the user device to adjust display settings and notification methods as needed. For example, if the user is experiencing stress, the screen display may be changed to calmer colors, and notifications may be sent in a way that minimizes stress.

[0526] Furthermore, the server monitors the information users access and automatically blocks any harmful content. The emotion engine can also immediately issue a warning if it detects an abnormal change in the user's emotions. For example, if a user's emotions suddenly become unstable, the device will display a message such as "For your safety, this content will no longer be accessible," demonstrating consideration for the user's feelings.

[0527] Furthermore, the user (AI agent) uses interactive characters and voice guides to notify the user of usage restrictions in a way that is appropriate to their emotional state. If the emotion engine detects "joy," it will notify the user of usage restrictions with a positive message, thereby preventing conflicts between parents and children.

[0528] As described above, this system combines an emotion engine and is optimized to protect both the user's vision and emotions, enabling safe and healthy internet use.

[0529] The following describes the processing flow.

[0530] Step 1:

[0531] The device uses a camera to detect the user's gaze distance, posture, blink rate, and ambient light, while also activating an emotion engine to analyze the user's facial expressions and voice tone. This allows the system to understand the user's device usage and emotional state.

[0532] Step 2:

[0533] The device sends the collected data to the server. The server uses the received information to evaluate the impact on the user's vision and emotional state. It combines the gaze distance data with the results of the facial expression analysis to generate an evaluation result.

[0534] Step 3:

[0535] Based on the evaluation results, the server sends instructions to adjust display settings and notification methods for the user's device. For example, if the user is experiencing stress, the server instructs the device to change the screen to calming colors and play soothing audio guidance.

[0536] Step 4:

[0537] The device automatically adjusts display settings according to server instructions. Furthermore, if the emotion engine detects tension or anxiety, it provides relaxing content and messages to stabilize the user's emotions. For example, it might play an audio message such as, "Take a deep breath and relax."

[0538] Step 5:

[0539] The terminal continues to monitor the information the user accesses and reports it to the server. The server filters the received information and automatically blocks access if harmful information is detected.

[0540] Step 6:

[0541] The server continuously monitors the user's emotional state and, if it detects an unexpected change in emotion, sends a warning to the device. The device then notifies the user that "for safety reasons, viewing of the content has been stopped."

[0542] Step 7:

[0543] Users are notified of usage restrictions in a fun way through interactive characters and voice guides. For example, if an emotion of joy is detected, a positive message such as "Your smile is radiant!" is displayed, which helps to alleviate conflicts between parents and children.

[0544] This allows the terminal and server to work together to protect the user's health and emotions, ensuring safe and secure internet use.

[0545] (Example 2)

[0546] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0547] The widespread use of digital devices in modern society raises concerns about their impact on users' vision and emotional health. In particular, there is a need to mitigate the risks of exposure to harmful information when users access the internet, as well as the health effects of improper posture and eye movements while using devices. Furthermore, optimizing internet use to suit each user's individual emotional state is essential.

[0548] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0549] In this invention, the server includes means for detecting the user's gaze, posture, facial expressions, and voice tone using imaging and audio input devices installed on the user's device; means for analyzing the user's emotional state using the collected information and identifying emotions in real time; and means for adjusting display and notification settings based on the analysis results to protect the user's eyesight and emotions. This promotes the user's physical and mental health and enables safe and appropriate internet use.

[0550] "Photography equipment" refers to devices that include cameras and sensors mounted on digital devices for capturing the user's gaze, posture, and facial expressions.

[0551] A "voice input device" is a device that includes a microphone for collecting the user's voice and processing it as voice data.

[0552] An "emotion engine" is a system that includes algorithms and software to analyze collected visual and auditory data and identify the user's emotional state.

[0553] "Filtering" is the process of restricting information based on criteria to detect and prevent the display of harmful content when users use the internet.

[0554] "Display settings" refer to settings that adjust the brightness, color tone, contrast, etc., of a digital device's screen according to the user's health condition.

[0555] "Notification settings" are settings that allow you to adjust how information and warnings are displayed to the user, taking into account the user's emotional state and health condition.

[0556] An "interactive character" is a digital representation that provides appropriate usage restrictions and information through interaction with the user, and includes those that function as visual or audio guides.

[0557] "Abnormal emotional change" refers to a sudden and unstable state in which the user's emotional condition changes, which is not expected in a normal usage environment.

[0558] This invention is a system designed to make internet use on users' digital devices safe and healthy. The system uses a camera and an audio input device installed on the user's device as hardware. Specifically, the camera captures the user's gaze distance, posture, and facial expressions, and the microphone collects the tone of the user's voice as audio input. As software, an emotion engine analyzes this data to identify the user's emotional state.

[0559] The device acquires real-time user data through its camera and voice input devices. The emotion engine then analyzes this data to determine the user's emotional state, such as whether they are stressed, relaxed, or happy. Based on this analysis, the server instructs the user to configure display and notification settings optimized to protect their visual and emotional health.

[0560] As a concrete example, if the emotion engine detects that a user is experiencing stress, the server instructs the device to change the screen's color tone to a calming blue and to change notifications to a gentler sound. Furthermore, if an abnormal emotional shift is detected, a warning such as "For your safety, please stop viewing this content" can be displayed quickly.

[0561] The user (AI agent) utilizes interactive characters and voice guides to notify them of appropriate usage restrictions in an emotionally responsive manner. For example, if the emotion engine detects the user's happiness, it will send a positive message such as, "You're having a wonderful day. How about taking a break now?"

[0562] Examples of prompts for the generative AI model include: "Please describe in detail the algorithm that the emotion engine uses to determine emotional states," and "Please give specific examples of what display changes the server will instruct if a stressed state is detected."

[0563] As described above, this invention takes the form of close cooperation between hardware and software in order to protect the user's eyesight and emotional state and to support healthy and safe internet use.

[0564] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0565] Step 1:

[0566] The device uses its built-in camera to capture the user's gaze distance, posture, and facial expressions, and uses its voice input device to collect voice tone. In this step, the input is real-time acquired visual and audio information, and the output is sent to the emotion engine. Specifically, the device operates its camera and microphone simultaneously, continuously collecting the necessary data while the user is using the device.

[0567] Step 2:

[0568] The server receives visual and audio data transmitted from the terminal and performs data analysis using an emotion engine. The input is the various captured data, and the output is an evaluation of the user's emotional state. Specifically, the emotion engine processes the data using statistical models and machine learning algorithms to identify emotions such as "tension," "relaxation," and "joy."

[0569] Step 3:

[0570] Based on the analysis results, the server automatically adjusts the display and notification settings, taking into account the user's eyesight and emotional state. This process takes the evaluation results of the emotion engine as input and outputs new settings to the device. Specifically, if the server determines that the user is experiencing stress, it instructs the device to change the screen to a calmer color scheme and soften notification sounds.

[0571] Step 4:

[0572] The server monitors the content users access, filters it as needed, and prevents the display of harmful information. This step involves user access patterns and evaluation criteria as input, and filtered results as output. Based on the filtering criteria, the server prevents users from accessing harmful sites and ensures their safety.

[0573] Step 5:

[0574] The user (AI agent) activates interactive characters and voice guides to notify them of usage restrictions based on their emotional state. This process takes server notification settings as input and output usage restriction notifications via voice or messages. Specifically, when the user is feeling "joy," positive messages such as "You're doing great! Shall we prepare for the next activity?" are displayed.

[0575] (Application Example 2)

[0576] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0577] In modern urban life, while internet use has become commonplace, the amount of information that may negatively affect users' eyesight and mental health is also increasing. Furthermore, if the information users access is harmful, they are more susceptible to its effects, raising concerns about the burden on their individual mental and physical health. Moreover, it is not easy to efficiently provide user-optimized and comfortable services from the vast amount of information available in urban areas. This invention aims to solve these problems.

[0578] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0579] In this invention, the server includes means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using a camera mounted on the user's device; means for evaluating the impact on the user's vision and emotional state based on the information obtained from the camera and adjusting the display settings of the user's device based on the results; means for inspecting the information accessed by the user and preventing the display of harmful information; and interactive display means for suggesting urban services based on the emotional state. This makes it possible to realize safe and comfortable internet use while considering the user's mental health.

[0580] A "shooting device" is a device used to detect the user's gaze distance, posture, blink rate, and ambient brightness, and is installed within the device.

[0581] "Information" refers to data such as the user's eye-tracking distance, posture, blinking frequency, ambient light, and data on the content the user accesses.

[0582] "Display settings" refer to settings on the device's display, including screen brightness, color tone, and contrast, which are automatically adjusted according to the user's eyesight and emotional state.

[0583] "Harmful information" refers to information that may negatively impact a user's mental health or safety, and should be filtered according to appropriate standards.

[0584] An "interactive display system" is a mechanism that allows for the exchange of information with the user and is used to suggest appropriate urban services based on the user's emotional state.

[0585] "Emotional state" refers to a psychological state evaluated based on the user's facial expressions and tone of voice, and is an important element that plays a crucial role in system operation and service delivery.

[0586] In the system implementing this invention, a camera mounted on the user device collects data on the user's gaze distance, posture, blink rate, and ambient brightness. This data is used to evaluate the impact on the user's vision and emotional state in real time.

[0587] The server receives data transferred from the imaging device and performs data analysis. This analysis uses deep learning frameworks such as TensorFlow to execute algorithms that estimate the user's emotional state from their facial expressions and voice tone. This allows for necessary adjustments to protect the user's vision and mental health.

[0588] The terminal automatically adjusts the display settings based on instructions sent from the server. For example, it adjusts the screen brightness and color tone to reduce eye strain. Furthermore, it uses interactive display methods on the device to suggest urban services and safety information based on the user's emotional state.

[0589] This system constantly monitors the information users access and immediately filters out any harmful content. Furthermore, it can send notifications to prompt appropriate action if a user's emotional state deviates from normal.

[0590] For example, if the system detects that the user is feeling sad, it can suggest nearby relaxation spots or events that could improve their mood. An example of a prompt to input into the generating AI model would be, "Please tell me how to build a recommendation model that shows what services are suggested when the user's emotional state is 'joyful'."

[0591] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0592] Step 1:

[0593] The terminal uses a camera to collect environmental data such as the user's gaze distance, posture, blink rate, and ambient light. This data is temporarily stored locally and then sent to a server for analysis. The data input consists of video and sensor data acquired from the camera, while the output is organized data ready for transfer to the server.

[0594] Step 2:

[0595] The server analyzes the received data. First, it applies machine learning algorithms to evaluate the impact on the user's vision and emotional state. The input is the environmental data sent from the terminal in step 1, and the output is the analysis result, which is the assessment of the emotional state and the impact on vision. This allows for necessary adjustments to maintain the user's health. Specific technologies used include tools such as TensorFlow.

[0596] Step 3:

[0597] The terminal adjusts the display settings based on instructions from the server. For example, if it determines that the user's eyes are tired, it will lower the screen brightness and change the color tone to a warmer color. The input is the analysis result from the server in step 2, and the output is the adjusted display settings.

[0598] Step 4:

[0599] The server inspects the information accessed by the user and filters out harmful content. It selects and displays only non-harmful information from the information the user receives. The input is internet data accessed by the user, and filtered, safe data is output.

[0600] Step 5:

[0601] Users receive interactive suggestions. Here, personalized information about city services and events is suggested based on the user's emotional state. Input is the server's analysis results and filtered information, and output is notifications of services and information tailored to the user. Furthermore, new service suggestions can be generated using generative AI models.

[0602] Step 6:

[0603] The device issues a warning notification when it detects a sudden change in the user's emotional state. For example, if it senses that the user is in a state of sadness, it will present information that can provide emotional support. The input is the user's current emotional state, and the output is a warning notification or support information.

[0604] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0605] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0606] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0607] [Fourth Embodiment]

[0608] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0609] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0610] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0611] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0612] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0613] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0614] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0615] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0616] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0617] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0618] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0619] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0620] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0621] This invention is a system for ensuring safe and healthy internet use by children on a user's device. Specifically, this system involves a terminal equipped with a camera that continuously monitors the user's behavior and surrounding environment, and a server that analyzes this data and makes adjustments as needed.

[0622] First, the device uses a camera to detect the user's gaze distance, posture, blinking frequency, and ambient light. This allows the device to understand how the user is using the device. Next, the device sends this data to a server. The server evaluates the impact on the user's eyesight based on the transmitted data. For example, if the user is too close to the device or blinking infrequently, the server may determine that this is negatively impacting their eyesight.

[0623] Based on these evaluation results, the server sends instructions to the terminal to automatically adjust the display settings. For example, if the user's surroundings are dark, the terminal will reduce the screen brightness to prevent vision deterioration. Settings are optimized in this way to protect the user's eye health.

[0624] Furthermore, the server monitors the information users access and blocks harmful content. If a user attempts to access a phishing site deemed dangerous, the server immediately blocks the access and displays a message on the device stating, "This page has been blocked because it is not safe."

[0625] Furthermore, this system uses interactive characters and voice guidance to prevent conflicts between parents and children. For example, if the device detects that the user's usage time is nearing its end, an animated character on the device will inform the user with a message such as, "It's almost break time! Let's save the game and go have a snack together!"

[0626] In this way, this system is designed to allow users to use the internet with peace of mind, promoting healthy usage and also helping with parent-child communication.

[0627] The following describes the processing flow.

[0628] Step 1:

[0629] The device uses a camera to detect the user's gaze distance, posture, blink rate, and ambient light in real time. This allows the user's device usage to be understood.

[0630] Step 2:

[0631] The terminal sends the detected data to the server. Based on the received data, the server determines whether those values ​​exceed a predetermined threshold.

[0632] Step 3:

[0633] The server performs an assessment of the impact on visual acuity and determines what adjustments are needed based on the results. If the assessment results exceed a threshold, the server instructs the terminal to adjust the display settings.

[0634] Step 4:

[0635] The device receives instructions from the server and automatically adjusts screen brightness and contrast to protect the user's eye health. It also displays notifications to encourage the user to maintain an appropriate distance from the screen.

[0636] Step 5:

[0637] The terminal monitors the information accessed by the user and reports it to the server. The server analyzes the received information through a filtering system to determine if it contains harmful information.

[0638] Step 6:

[0639] If the server detects harmful information, it sends a command to the terminal to block it. The terminal blocks access to that information and displays a notification to the user stating, "This information has been blocked for security reasons."

[0640] Step 7:

[0641] The user (in this case, an AI agent) uses interactive characters and voice guides through the device to notify them of usage limits in a fun way. When the user's usage time is nearing its end, the character will notify them with a message such as, "Time for a break!"

[0642] Through this series of processes, the terminal and server work together to support the user's safe and healthy device usage.

[0643] (Example 1)

[0644] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0645] In recent years, concerns have grown regarding the impact of internet use on children's vision and its safety. In particular, vision loss due to prolonged device use and access to harmful information are major issues for parents and educators. Managing screen time and communicating with children are also challenges. There is a need to comprehensively address these issues and provide an environment where children can use the internet safely and healthily.

[0646] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0647] In this invention, the server includes means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using an information gathering device; means for evaluating the impact on the user's vision using the information obtained from the information gathering device and automatically adjusting the display settings of the user's device based on the evaluation results; and means for monitoring the information used by the user and preventing the display of harmful information. This makes it possible to provide an environment in which children can use the internet safely and healthily.

[0648] An "information gathering device" is a device such as a sensor or camera used to detect the user's gaze distance, posture, blink rate, and ambient brightness.

[0649] "User equipment" refers to electronic devices used by users to access the internet, including smartphones, tablets, and computers.

[0650] "Interactive characters" refer to animated characters or interactive objects that operate on the screen to provide users with information visually and aurally.

[0651] "Voice guidance" is a system that conveys necessary information to users via voice, and can also be used to restrict usage or issue warnings through voice messages.

[0652] "Evaluating the impact on vision" is a process of analyzing data obtained from information gathering devices to assess the impact of device use on the user's eyesight.

[0653] "Automatic adjustment" refers to the system independently changing the settings of the user's device, taking into account the impact on the user's eyesight.

[0654] "Harmful information" refers to content or websites that are inappropriate or dangerous for users and whose display should be blocked by the system.

[0655] This invention is a system designed to provide users with a safe and secure environment for using the internet. Specifically, an information gathering device installed in the terminal detects the user's gaze distance, posture, blink rate, and ambient brightness. Using this data, the server evaluates the impact on the user's vision and automatically adjusts the terminal's display settings as needed.

[0656] The terminal continuously collects this information using hardware equipped with cameras and various sensors. The data is transmitted to a server via a secure and efficient network protocol, and the server analyzes the data using advanced analytical software. The software used here requires the ability to process data in real time.

[0657] For example, if a user's gaze is too close to the device or they are not blinking frequently enough, the server can detect this and send an instruction to the device to automatically adjust the screen brightness. Furthermore, if the ambient light around the device decreases, the screen brightness can be reduced to lessen the strain on the eyes.

[0658] Furthermore, the server monitors the websites users visit and the information they access. If it detects access to phishing sites or harmful content, it immediately blocks it and displays a message on the device saying, "This page was blocked because it is not safe."

[0659] This system manages the time children spend using the device and provides friendly notifications using interactive characters and voice guidance. For example, when usage time is nearing its end, an animated character will inform the user, "It's almost break time! Let's save the game and go have a snack together!"

[0660] A concrete example of a prompt message when using a generative AI model is, "Please describe the details of a system that automatically adjusts the brightness of a device to prevent children from damaging their eyesight." This is an example of input used to describe the functionality and features of a system to the AI ​​model.

[0661] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0662] Step 1:

[0663] The device uses cameras and sensors to detect the user's gaze distance, posture, blink rate, and ambient light. The input for this step is physical information about the user's surroundings. This allows the device to collect data on the user's device usage and format it as digital data. The output is real-time data about the user's usage.

[0664] Step 2:

[0665] The terminal sends the data obtained in step 1 to the server. The input is the usage data provided by the terminal. The server receives this data and converts it into an appropriate format for data processing. The output is user status data in an analyzable format.

[0666] Step 3:

[0667] The server analyzes the received data and evaluates its impact on the user's visual health. The input for this step is data such as gaze distance and posture transmitted from the terminal. Based on this, the server performs visual simulations and statistical analysis to assess the potential impact on the user's vision. The output is the evaluation result regarding the degree of impact on vision.

[0668] Step 4:

[0669] Based on the evaluation results, the server sends instructions to the terminal for automatic adjustment of display settings. The input is the evaluation results obtained in step 3. Based on this, the server determines the necessary changes to display settings (e.g., adjusting screen brightness and contrast) to reduce the strain on the user's eyes. The output is the specific instructions for adjusting the display settings.

[0670] Step 5:

[0671] The terminal adjusts the display settings according to instructions received from the server. The input is the adjustment instructions from the server. The terminal dynamically changes the display settings based on these instructions. The output is the terminal's screen with the adjusted display settings.

[0672] Step 6:

[0673] The server monitors the information users access and immediately blocks any phishing sites or harmful content it detects. The input is the user's access information. The server uses a filtering algorithm to detect harmful information. The output is a notification that harmful information has been blocked.

[0674] Step 7:

[0675] The terminal uses interactive characters and voice to notify the user when their usage time is ending. The input is data from the server regarding the end of usage time. To encourage the user to take a break, the terminal displays a character on the screen and provides voice guidance such as, "It's almost break time! Save your game and let's go get a snack together!" The output is a visual and auditory message for the user.

[0676] (Application Example 1)

[0677] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0678] Concerns exist regarding the safety and health impacts of internet use by children today through information processing devices. Specifically, issues include deterioration of visual health, harmful content, and inappropriate usage time. These challenges need to be addressed comprehensively.

[0679] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0680] In this invention, the server includes means for detecting the user's gaze distance, posture, blink rate, and ambient brightness; means for inspecting the information the user accesses and preventing the display of harmful information; and means for a robot to monitor the user's internet usage time and encourage appropriate breaks. This makes it possible to provide an appropriate usage environment while ensuring the safety and health of children's internet use.

[0681] "User" refers to an individual or their family member who uses this system, and especially to children who use the internet.

[0682] An "information processing device" is an electronic device used by users to obtain information or view content, and it is equipped with a camera and sensors.

[0683] A "shooting device" is a device that has a camera function to detect the user's gaze distance, posture, blink rate, and ambient brightness.

[0684] "Eye-gaze distance" refers to the physical distance between the user's eyes and the screen of the information processing device.

[0685] "Posture" refers to the user's body position and stance when using an information processing device.

[0686] "Blink count" refers to the frequency of a user's blinks within a given period of time.

[0687] "Ambient brightness" refers to the degree of brightness in the environment where the information processing device is placed.

[0688] "Display settings" refers to visual adjustments such as brightness and contrast of the content displayed on the screen of an information processing device.

[0689] An "interactive visual element" is a graphical motif or character that can visually interact with the user.

[0690] "Voice guidance" refers to a function or device that uses voice to convey instructions or information to the user.

[0691] "Usage restrictions" refer to settings that limit the time or content a user can access when using their information processing device.

[0692] "Conflict" refers to disagreements or disagreements between parents and children arising from the use of information processing devices.

[0693] "Internet usage time" refers to the total time a user spends connected to the internet through an information processing device.

[0694] A "robot" is a household machine that works in conjunction with a system installed on an information processing device to monitor and assist the user's activities.

[0695] In implementing this invention, the user's information processing device is equipped with a high-precision imaging device and sensors. This imaging device detects the user's gaze distance, posture, blink rate, and ambient brightness in real time. This data is first temporarily processed at the terminal and then transmitted to a server via the network.

[0696] On the server, advanced algorithms are executed that utilize external AI analysis tools such as Google Cloud to evaluate the impact on the user's eyesight based on the transmitted data. Machine learning models are used in the data analysis to assess the user's health status. Based on these evaluation results, display settings are automatically adjusted. For example, if the ambient light is low, the brightness of the device screen can be appropriately reduced to alleviate visual strain.

[0697] Furthermore, the information users access is monitored on the server, and phishing sites and inappropriate content are blocked based on filtering criteria provided by external sources. AI-generated models are also utilized in this process. For example, if a user attempts to access a harmful site, the system immediately responds by displaying a message such as, "This page has been blocked because it is not safe."

[0698] Furthermore, interactive visual elements and audio guidance for usage restrictions are presented in a friendly manner, employing child-friendly animated characters. For example, after prolonged use, the character might prompt, "It's time to take a break!", encouraging rest and helping to maintain eye health and concentration. The robot then engages in natural interaction with the user based on this instruction.

[0699] As a concrete example, the prompt message would look like this:

[0700] "The robot should monitor the user's blinking frequency and prompt appropriate action."

[0701] "If the user is viewing the screen in a dark environment, the system will either brighten the surroundings or adjust the device's brightness."

[0702] Thus, the present invention promotes user health maintenance and safe use of the internet through cooperation between an information processing device and a server.

[0703] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0704] Step 1:

[0705] The terminal uses its built-in camera to sense the user's gaze distance, posture, blink rate, and ambient light. This data is obtained as input. The terminal then performs initial filtering on the input data, converts it to the required format, and prepares it for transmission to the server.

[0706] Step 2:

[0707] The server, upon receiving data transmitted from the terminal, processes the data using external AI analysis tools such as Google Cloud. In this processing step, a machine learning model analyzes the data to assess the user's health status. Specifically, it sets warning signals if the gaze distance is too short or the blinking frequency is insufficient. These analysis results are then output.

[0708] Step 3:

[0709] Based on the analysis results, the server sends necessary adjustment instructions to the user's information processing device. For example, if the ambient light is low, an instruction to lower the screen brightness is sent to the terminal. Based on this prompt, the terminal performs the action.

[0710] Step 4:

[0711] The server monitors the information users are trying to access. It applies filtering criteria provided by external sources to detect phishing sites and harmful content. This process uses an AI model to validate rules for blocking information deemed harmful, ensuring that such information is not output.

[0712] Step 5:

[0713] If a user uses the information processing device for a certain period of time, the server issues a command to send interactive visual elements and audio guidance to the terminal. Specifically, a robot displays an animated character and delivers an audio message saying, "Let's take a break." This guidance is provided in a way that is friendly and approachable to the user.

[0714] Step 6:

[0715] Based on feedback from the device, users adjust their actions, such as taking breaks. This step aims to elicit spontaneous actions from users by providing visual and auditory information. In particular, it is the moment when users hear a prompt like "Let's take a break" and decide whether or not to actually put the device down.

[0716] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0717] This invention aims to further consider the user's mental and physical health by incorporating an emotion engine that recognizes the user's emotions into a system that supports safe and healthy internet use on user devices. This system works in conjunction with the device's camera and emotion engine to evaluate the user's emotional state based on their gaze distance, posture, blinking frequency, ambient light, as well as their facial expressions and tone of voice.

[0718] First, the device not only collects data on the user's behavior and surrounding environment using a camera, but also activates an emotion engine to detect changes in the user's facial expressions and voice. This allows the device to understand the user's emotional state in real time. For example, if the user is nervous, the emotion engine will detect this from their facial expressions and tone of voice.

[0719] Next, the server receives the data sent from the device and analyzes it using multiple evaluation criteria. The server uses algorithms to assess the impact on the user's vision and emotional state, and instructs the user device to adjust display settings and notification methods as needed. For example, if the user is experiencing stress, the screen display may be changed to calmer colors, and notifications may be sent in a way that minimizes stress.

[0720] Furthermore, the server monitors the information users access and automatically blocks any harmful content. The emotion engine can also immediately issue a warning if it detects an abnormal change in the user's emotions. For example, if a user's emotions suddenly become unstable, the device will display a message such as "For your safety, this content will no longer be accessible," demonstrating consideration for the user's feelings.

[0721] Furthermore, the user (AI agent) uses interactive characters and voice guides to notify the user of usage restrictions in a way that is appropriate to their emotional state. If the emotion engine detects "joy," it will notify the user of usage restrictions with a positive message, thereby preventing conflicts between parents and children.

[0722] As described above, this system combines an emotion engine and is optimized to protect both the user's vision and emotions, enabling safe and healthy internet use.

[0723] The following describes the processing flow.

[0724] Step 1:

[0725] The device uses a camera to detect the user's gaze distance, posture, blink rate, and ambient light, while also activating an emotion engine to analyze the user's facial expressions and voice tone. This allows the system to understand the user's device usage and emotional state.

[0726] Step 2:

[0727] The device sends the collected data to the server. The server uses the received information to evaluate the impact on the user's vision and emotional state. It combines the gaze distance data with the results of the facial expression analysis to generate an evaluation result.

[0728] Step 3:

[0729] Based on the evaluation results, the server sends instructions to adjust display settings and notification methods for the user's device. For example, if the user is experiencing stress, the server instructs the device to change the screen to calming colors and play soothing audio guidance.

[0730] Step 4:

[0731] The device automatically adjusts display settings according to server instructions. Furthermore, if the emotion engine detects tension or anxiety, it provides relaxing content and messages to stabilize the user's emotions. For example, it might play an audio message such as, "Take a deep breath and relax."

[0732] Step 5:

[0733] The terminal continues to monitor the information the user accesses and reports it to the server. The server filters the received information and automatically blocks access if harmful information is detected.

[0734] Step 6:

[0735] The server continuously monitors the user's emotional state and, if it detects an unexpected change in emotion, sends a warning to the device. The device then notifies the user that "for safety reasons, viewing of the content has been stopped."

[0736] Step 7:

[0737] Users are notified of usage restrictions in a fun way through interactive characters and voice guides. For example, if an emotion of joy is detected, a positive message such as "Your smile is radiant!" is displayed, which helps to alleviate conflicts between parents and children.

[0738] This allows the terminal and server to work together to protect the user's health and emotions, ensuring safe and secure internet use.

[0739] (Example 2)

[0740] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0741] The widespread use of digital devices in modern society raises concerns about their impact on users' vision and emotional health. In particular, there is a need to mitigate the risks of exposure to harmful information when users access the internet, as well as the health effects of improper posture and eye movements while using devices. Furthermore, optimizing internet use to suit each user's individual emotional state is essential.

[0742] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0743] In this invention, the server includes means for detecting the user's gaze, posture, facial expressions, and voice tone using imaging and audio input devices installed on the user's device; means for analyzing the user's emotional state using the collected information and identifying emotions in real time; and means for adjusting display and notification settings based on the analysis results to protect the user's eyesight and emotions. This promotes the user's physical and mental health and enables safe and appropriate internet use.

[0744] "Photography equipment" refers to devices that include cameras and sensors mounted on digital devices for capturing the user's gaze, posture, and facial expressions.

[0745] A "voice input device" is a device that includes a microphone for collecting the user's voice and processing it as voice data.

[0746] An "emotion engine" is a system that includes algorithms and software to analyze collected visual and auditory data and identify the user's emotional state.

[0747] "Filtering" is the process of restricting information based on criteria to detect and prevent the display of harmful content when users use the internet.

[0748] "Display settings" refer to settings that adjust the brightness, color tone, contrast, etc., of a digital device's screen according to the user's health condition.

[0749] "Notification settings" are settings that allow you to adjust how information and warnings are displayed to the user, taking into account the user's emotional state and health condition.

[0750] An "interactive character" is a digital representation that provides appropriate usage restrictions and information through interaction with the user, and includes those that function as visual or audio guides.

[0751] "Abnormal emotional change" refers to a sudden and unstable state in which the user's emotional condition changes, which is not expected in a normal usage environment.

[0752] This invention is a system designed to make internet use on users' digital devices safe and healthy. The system uses a camera and an audio input device installed on the user's device as hardware. Specifically, the camera captures the user's gaze distance, posture, and facial expressions, and the microphone collects the tone of the user's voice as audio input. As software, an emotion engine analyzes this data to identify the user's emotional state.

[0753] The device acquires real-time user data through its camera and voice input devices. The emotion engine then analyzes this data to determine the user's emotional state, such as whether they are stressed, relaxed, or happy. Based on this analysis, the server instructs the user to configure display and notification settings optimized to protect their visual and emotional health.

[0754] As a concrete example, if the emotion engine detects that a user is experiencing stress, the server instructs the device to change the screen's color tone to a calming blue and to change notifications to a gentler sound. Furthermore, if an abnormal emotional shift is detected, a warning such as "For your safety, please stop viewing this content" can be displayed quickly.

[0755] The user (AI agent) utilizes interactive characters and voice guides to notify them of appropriate usage restrictions in an emotionally responsive manner. For example, if the emotion engine detects the user's happiness, it will send a positive message such as, "You're having a wonderful day. How about taking a break now?"

[0756] Examples of prompts for the generative AI model include: "Please describe in detail the algorithm that the emotion engine uses to determine emotional states," and "Please give specific examples of what display changes the server will instruct if a stressed state is detected."

[0757] As described above, this invention takes the form of close cooperation between hardware and software in order to protect the user's eyesight and emotional state and to support healthy and safe internet use.

[0758] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0759] Step 1:

[0760] The device uses its built-in camera to capture the user's gaze distance, posture, and facial expressions, and uses its voice input device to collect voice tone. In this step, the input is real-time acquired visual and audio information, and the output is sent to the emotion engine. Specifically, the device operates its camera and microphone simultaneously, continuously collecting the necessary data while the user is using the device.

[0761] Step 2:

[0762] The server receives visual and audio data transmitted from the terminal and performs data analysis using an emotion engine. The input is the various captured data, and the output is an evaluation of the user's emotional state. Specifically, the emotion engine processes the data using statistical models and machine learning algorithms to identify emotions such as "tension," "relaxation," and "joy."

[0763] Step 3:

[0764] Based on the analysis results, the server automatically adjusts the display and notification settings, taking into account the user's eyesight and emotional state. This process takes the evaluation results of the emotion engine as input and outputs new settings to the device. Specifically, if the server determines that the user is experiencing stress, it instructs the device to change the screen to a calmer color scheme and soften notification sounds.

[0765] Step 4:

[0766] The server monitors the content users access, filters it as needed, and prevents the display of harmful information. This step involves user access patterns and evaluation criteria as input, and filtered results as output. Based on the filtering criteria, the server prevents users from accessing harmful sites and ensures their safety.

[0767] Step 5:

[0768] The user (AI agent) activates interactive characters and voice guides to notify them of usage restrictions based on their emotional state. This process takes server notification settings as input and output usage restriction notifications via voice or messages. Specifically, when the user is feeling "joy," positive messages such as "You're doing great! Shall we prepare for the next activity?" are displayed.

[0769] (Application Example 2)

[0770] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0771] In modern urban life, while internet use has become commonplace, the amount of information that may negatively affect users' eyesight and mental health is also increasing. Furthermore, if the information users access is harmful, they are more susceptible to its effects, raising concerns about the burden on their individual mental and physical health. Moreover, it is not easy to efficiently provide user-optimized and comfortable services from the vast amount of information available in urban areas. This invention aims to solve these problems.

[0772] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0773] In this invention, the server includes means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using a camera mounted on the user's device; means for evaluating the impact on the user's vision and emotional state based on the information obtained from the camera and adjusting the display settings of the user's device based on the results; means for inspecting the information accessed by the user and preventing the display of harmful information; and interactive display means for suggesting urban services based on the emotional state. This makes it possible to realize safe and comfortable internet use while considering the user's mental health.

[0774] A "shooting device" is a device used to detect the user's gaze distance, posture, blink rate, and ambient brightness, and is installed within the device.

[0775] "Information" refers to data such as the user's eye-tracking distance, posture, blinking frequency, ambient light, and data on the content the user accesses.

[0776] "Display settings" refer to settings on the device's display, including screen brightness, color tone, and contrast, which are automatically adjusted according to the user's eyesight and emotional state.

[0777] "Harmful information" refers to information that may negatively impact a user's mental health or safety, and should be filtered according to appropriate standards.

[0778] An "interactive display system" is a mechanism that allows for the exchange of information with the user and is used to suggest appropriate urban services based on the user's emotional state.

[0779] "Emotional state" refers to a psychological state evaluated based on the user's facial expressions and tone of voice, and is an important element that plays a crucial role in system operation and service delivery.

[0780] In the system implementing this invention, a camera mounted on the user device collects data on the user's gaze distance, posture, blink rate, and ambient brightness. This data is used to evaluate the impact on the user's vision and emotional state in real time.

[0781] The server receives data transferred from the imaging device and performs data analysis. This analysis uses deep learning frameworks such as TensorFlow to execute algorithms that estimate the user's emotional state from their facial expressions and voice tone. This allows for necessary adjustments to protect the user's vision and mental health.

[0782] The terminal automatically adjusts the display settings based on instructions sent from the server. For example, it adjusts the screen brightness and color tone to reduce eye strain. Furthermore, it uses interactive display methods on the device to suggest urban services and safety information based on the user's emotional state.

[0783] This system constantly monitors the information users access and immediately filters out any harmful content. Furthermore, it can send notifications to prompt appropriate action if a user's emotional state deviates from normal.

[0784] For example, if the system detects that the user is feeling sad, it can suggest nearby relaxation spots or events that could improve their mood. An example of a prompt to input into the generating AI model would be, "Please tell me how to build a recommendation model that shows what services are suggested when the user's emotional state is 'joyful'."

[0785] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0786] Step 1:

[0787] The terminal uses a camera to collect environmental data such as the user's gaze distance, posture, blink rate, and ambient light. This data is temporarily stored locally and then sent to a server for analysis. The data input consists of video and sensor data acquired from the camera, while the output is organized data ready for transfer to the server.

[0788] Step 2:

[0789] The server analyzes the received data. First, it applies machine learning algorithms to evaluate the impact on the user's vision and emotional state. The input is the environmental data sent from the terminal in step 1, and the output is the analysis result, which is the assessment of the emotional state and the impact on vision. This allows for necessary adjustments to maintain the user's health. Specific technologies used include tools such as TensorFlow.

[0790] Step 3:

[0791] The terminal adjusts the display settings based on instructions from the server. For example, if it determines that the user's eyes are tired, it will lower the screen brightness and change the color tone to a warmer color. The input is the analysis result from the server in step 2, and the output is the adjusted display settings.

[0792] Step 4:

[0793] The server inspects the information accessed by the user and filters out harmful content. It selects and displays only non-harmful information from the information the user receives. The input is internet data accessed by the user, and filtered, safe data is output.

[0794] Step 5:

[0795] Users receive interactive suggestions. Here, personalized information about city services and events is suggested based on the user's emotional state. Input is the server's analysis results and filtered information, and output is notifications of services and information tailored to the user. Furthermore, new service suggestions can be generated using generative AI models.

[0796] Step 6:

[0797] The device issues a warning notification when it detects a sudden change in the user's emotional state. For example, if it senses that the user is in a state of sadness, it will present information that can provide emotional support. The input is the user's current emotional state, and the output is a warning notification or support information.

[0798] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0799] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0800] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0801] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0802] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0803] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0804] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0805] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0806] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0807] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0808] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0809] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0810] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0811] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0812] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0813] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0814] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0815] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0816] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0817] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0818] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0819] The following is further disclosed regarding the embodiments described above.

[0820] (Claim 1)

[0821] A means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using a camera mounted on the user's device,

[0822] A means for evaluating the impact on the user's vision using information obtained from the aforementioned imaging device, and for automatically adjusting the display settings of the user device based on the evaluation results,

[0823] A means to inspect the information users access and prevent the display of harmful information,

[0824] A means for notifying the user device of usage restrictions using interactive characters or voice guides to prevent problems from occurring,

[0825] A system that includes this.

[0826] (Claim 2)

[0827] The system according to claim 1, wherein the display settings of the user device are adjusted to be optimized to protect the user's eye health.

[0828] (Claim 3)

[0829] The system according to claim 1, wherein the means for preventing the display of the harmful information uses filtering criteria provided from an external source to ensure user safety.

[0830] "Example 1"

[0831] (Claim 1)

[0832] The information gathering device includes means for detecting the user's gaze distance, posture, blink rate, and ambient brightness,

[0833] A means for evaluating the impact on the user's vision using the information obtained from the aforementioned information collection device, and for automatically adjusting the display settings of the user's device based on the evaluation results,

[0834] Means to monitor the information users access and prevent the display of harmful information,

[0835] A means of notifying the user's device of usage restrictions using interactive characters or voice guidance to prevent problems from occurring,

[0836] A means of recording user behavior and providing appropriate feedback,

[0837] A system that includes this.

[0838] (Claim 2)

[0839] The system according to claim 1, wherein the display settings of the user device are adjusted to be optimized to protect the user's visual health.

[0840] (Claim 3)

[0841] The system according to claim 1, wherein the means for preventing the display of the harmful information uses filtering criteria provided from an external source to ensure the safety of the user.

[0842] "Application Example 1"

[0843] (Claim 1)

[0844] A means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using a camera mounted on the user's information processing device,

[0845] A means for evaluating the impact on the user's vision using information obtained from the aforementioned imaging device, and for automatically adjusting the display settings of the user information processing device based on the evaluation results,

[0846] A means to inspect the information users access and prevent the display of harmful information,

[0847] The aforementioned user information processing device is provided with means for notifying the user of usage restrictions using interactive visual elements and voice guidance, thereby preventing disputes from arising.

[0848] A means of using robots to monitor users' internet usage time and encourage appropriate breaks,

[0849] A system that includes this.

[0850] (Claim 2)

[0851] The system according to claim 1, wherein the display settings of the user information processing device are adjusted to be optimized to protect the user's eye health.

[0852] (Claim 3)

[0853] The system according to claim 1, wherein the means for preventing the display of the harmful information uses filtering criteria provided from an external source to ensure user safety.

[0854] "Example 2 of combining an emotion engine"

[0855] (Claim 1)

[0856] A means for detecting the user's gaze distance, posture, blink rate, ambient brightness, facial expression, and voice tone using imaging and audio input devices mounted on the user's device,

[0857] A means for analyzing the user's emotional state and identifying emotions in real time using information obtained from the aforementioned shooting and voice input devices,

[0858] Based on the aforementioned analysis results, means for evaluating the impact on the user's vision and emotional state and automatically adjusting the display and notification settings of the device,

[0859] A means to prevent the display of harmful information by monitoring the content of information accessed by users and filtering related information,

[0860] The device is provided with means for notifying users of usage restrictions based on their emotional state using interactive characters and voice guides, thereby promoting safe use.

[0861] A means to immediately display a warning when an abnormal emotional change is detected by the emotion engine,

[0862] A system that includes this.

[0863] (Claim 2)

[0864] The system according to claim 1, wherein the display and notification settings of the device are optimized to protect the user's eyesight and physical and mental health.

[0865] (Claim 3)

[0866] The system according to claim 1, wherein the filtering means uses evaluation criteria provided externally for the purpose of ensuring user safety.

[0867] "Application example 2 when combining with an emotional engine"

[0868] (Claim 1)

[0869] A means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using a camera mounted on the user's device,

[0870] A means for evaluating the impact on the user's vision using information obtained from the aforementioned imaging device, and for automatically adjusting the display settings of the user device based on the evaluation results,

[0871] A means to inspect the information users access and prevent the display of harmful information,

[0872] A means for proposing urban services based on emotional state to the user device using an interactive display means,

[0873] A means for notifying the user device of usage restrictions using interactive characters or voice guides to prevent problems from occurring,

[0874] A system that includes this.

[0875] (Claim 2)

[0876] The system according to claim 1, wherein the display settings of the user device are adjusted to be optimized to provide information appropriate to the user's emotional state, taking into account the user's mental health.

[0877] (Claim 3)

[0878] The system according to claim 1, wherein the means for preventing the display of the harmful information uses filtering criteria provided from an external source to ensure user safety and criteria that take into account the user's emotional state. [Explanation of Symbols]

[0879] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for detecting the user's gaze distance, posture, blink rate, and ambient brightness using a camera mounted on the user's device, A means for evaluating the impact on the user's vision using information obtained from the aforementioned imaging device, and for automatically adjusting the display settings of the user device based on the evaluation results, A means to inspect the information users access and prevent the display of harmful information, A means for notifying the user device of usage restrictions using interactive characters or voice guides to prevent problems from occurring, A system that includes this.

2. The system according to claim 1, wherein the display settings of the user device are optimized to protect the user's eye health.

3. The system according to claim 1, wherein the means for preventing the display of the harmful information uses filtering criteria provided from an external source to ensure user safety.

Images