system

Abstract

We provide the system. [Solution] A means comprising a user interface for transmitting captured image data to an information processing device in order to identify dangerous areas within the home, A means of using an information processing device that analyzes received image data and executes a generative model to identify risks, A means including a message generation unit that presents appropriate countermeasures based on risk identification, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventionally, accidents in the home occur in an unexpected manner, and many of them often start from risks that are easily overlooked. However, it is not easy to identify potential dangerous locations in the home in advance and take safety measures. Therefore, means for preventing accidents in the home and improving safety are required.

Means for Solving the Problems

[0005] This invention provides a system for analyzing captured image data from within a home and identifying potential hazards. Specifically, it transmits image data to an information processing device via a user interface, and then uses a generative model to identify hazardous elements within the image. Based on the identified risks, the system presents safety measures as messages, thereby enabling the prevention of accidents within the home.

[0006] An "information processing device" is a device for inputting, processing, and outputting data, and enables the analysis of image data and the execution of generative models.

[0007] A "user interface" refers to an interface that allows a user to interact with an information processing device and exchange information, including the means by which image data can be transmitted.

[0008] "Image data" refers to data recorded in a digital format from visual information captured by a camera or similar imaging device.

[0009] A "generative model" refers to an algorithm or program designed to perform a specific task using patterns learned from large amounts of data.

[0010] "Risk identification" is the process of analyzing received data to discover potential risk factors.

[0011] The "message generation unit" is a component that has the function of generating messages that inform the user of appropriate countermeasures or alerts based on the identified risks.

[0012] An "encryption protocol" is a communication method that defines procedures and rules for encrypting or decrypting information in order to ensure the secure transmission and reception of data. [Brief explanation of the drawing]

[0013] [Figure 1]This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention is a system for preventing accidents within the home. This system involves the collaborative operation of user, terminal, and server elements.

[0035] First, the user takes pictures of areas of concern in their home using the device's camera. This is intended to identify accident-prone areas, such as scattered cables or dangerous corners, and sends the images. The user then transfers these images to the server in digital data format, following the instructions on the device's built-in user interface.

[0036] The terminal properly formats the image data received from the user and transmits it to the server using a secure communication protocol. This ensures the confidentiality and integrity of the data.

[0037] The server runs a generative model to analyze the received image data. Based on a pre-trained near-miss database, the generative model identifies potential hazards in the image and assesses the risk. This assessment process, for example, recognizes slippery floors or sharp objects in the image and determines their likelihood of causing an accident.

[0038] Once a risk is identified, the server uses its message generation unit to create a message about safety measures. This message proposes preventative measures for the specific risk identified. The server sends this information back to the terminal and also provides the user with visual alerts and informational displays.

[0039] As a concrete example, suppose a user sends an image taken in their living room to the server. This image shows unorganized cables next to the sofa. The server uses a generative model to identify the risk, considers measures to secure the cables to prevent tripping, and displays these suggestions to the user.

[0040] Thus, this system provides a practical and effective solution for dramatically reducing the risk of accidents in the home.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The user takes a picture of an area of ​​interest in their home using the device's camera. After taking the picture, they select the image through the user interface and send the image data to the system.

[0044] Step 2:

[0045] The terminal receives image data sent by the user. Next, it converts this data to an appropriate digital format (e.g., JPEG), applies an encryption protocol to the data, and securely transmits it to the server.

[0046] Step 3:

[0047] The server decodes the image data received from the terminal and converts it into a format that can be processed. This prepares the data for analysis using a generative model.

[0048] Step 4:

[0049] The server runs a generative model and analyzes the received images. The model consults a near-miss database to identify potential hazards in the images. For example, it can recognize code that could cause tripping or sharp corners of furniture.

[0050] Step 5:

[0051] The server generates a message suggesting appropriate security measures based on the identified risks. This message includes specific countermeasures and aims to provide useful information to the user.

[0052] Step 6:

[0053] The server sends the generated risk assessment results and countermeasure messages to the terminal. Communication is conducted securely using an encrypted protocol.

[0054] Step 7:

[0055] The terminal receives messages from the server and presents them visually to the user through the user interface. The user can then use this information to take necessary safety measures within their home.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] Accidents within the home are a serious problem, especially in environments with elderly people or children. Current technology makes it difficult to efficiently and continuously identify risks within the home and propose appropriate countermeasures, resulting in insufficient improvements in safety. Therefore, there is a need for a system that can automatically detect potential risks within the home and propose rapid countermeasures.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for providing an interface for a user to transmit image data to a data processing device to identify risks within the home; means for using a data processing device that examines the received image data and executes a generation algorithm to identify hazards; means for providing a message generation unit that suggests preventive measures based on the identified hazards; and means for using an encrypted communication protocol to maintain the confidentiality and integrity of communications. This makes it possible to effectively and quickly reduce the risk of accidents within the home.

[0061] "Household risks" refer to situations or objects within the home that could potentially cause accidents.

[0062] A "data processing device" refers to a computer system that takes data as input and analyzes that data based on specific processing steps.

[0063] "Image data" refers to data containing visual information expressed in a digital format.

[0064] A "user interface" refers to a user interface that is operable by the user and enables interaction with devices and systems.

[0065] A "generative algorithm" refers to a set of computational procedures or methods used to automatically extract specific information from data.

[0066] "Identifying hazards" refers to identifying factors that could potentially cause accidents or damage in a given area or object.

[0067] A "message generation unit" refers to a device or program that has the function of automatically creating text or suggestions to be conveyed to the user based on specific information.

[0068] An "encrypted communication protocol" refers to a communication method used to protect data from being read by third parties when sending or receiving data.

[0069] "Confidentiality and integrity of communications" refers to the information security requirement that transmitted data is not leaked to others and is received without being altered.

[0070] This invention is a system aimed at preventing accidents in the home, in which the user, terminal, and server elements work together.

[0071] The user first takes photos of areas of concern in their home, such as scattered cables or sharp furniture corners, using the device's camera. The device is equipped with a camera that has sensor functions for generating digital data. The captured image data is temporarily stored on the device.

[0072] The device formats the saved images appropriately, encrypts them, and then sends them to the server. Secure communication protocols such as HTTPS are used for this purpose. The server functions as a data processing unit to receive this secure communication.

[0073] The server runs a generative AI model, which has been previously trained using a near-miss database, to analyze the received image data. This AI model automatically identifies potential hazards within the image and performs a risk assessment. For example, it might highlight areas where people are prone to slipping or tripping.

[0074] Once a risk is identified, the server's message generation unit is utilized to generate a message containing specific preventative measures for that particular risk. This information is sent back to the terminal and presented to the user with a visual alert.

[0075] For example, if a user takes a picture in their living room and it's sent from their device to the server, and that picture includes scattered cables next to the sofa, the server's generative model will analyze the risk. It will assess the risk of tripping and generate a message recommending that the cables be tidied up.

[0076] An example of a prompt might be, "Analyze the image of the living room and suggest safety measures." This approach allows the system to provide useful solutions that dramatically reduce the risk of accidents in the home.

[0077] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0078] Step 1:

[0079] The user takes a picture of an area of ​​concern in their home using the device's camera. The input is the image captured by the camera, and the output is image data saved in digital format. Specifically, the user might take a picture of cluttered cables or sharp furniture corners, and the image data is temporarily stored on the device.

[0080] Step 2:

[0081] The terminal formats the stored images into a secure format and prepares them for transmission to the server. The input is the stored image data, and the output is encrypted image data. Specifically, the terminal compresses the images into a format such as JPEG and encrypts the data using the HTTPS protocol.

[0082] Step 3:

[0083] The terminal sends encrypted image data to the server. The input is encrypted image data, and the output is the image data received by the server. Here, the terminal uses the HTTPS protocol to securely transmit the image data over the network.

[0084] Step 4:

[0085] The server inputs the received image data into a generating AI model to perform a risk assessment. The input is the received image data, and the output is the result of the risk assessment. Specifically, the server identifies hazardous elements within the image, such as slippery areas or sharp objects.

[0086] Step 5:

[0087] The server generates safety messages based on evaluation results from a generative AI model. The input is the result of the risk assessment, and the output is a message containing specific preventive measures. The server creates suggestions for the identified risks, such as a message like "Secure the cables."

[0088] Step 6:

[0089] The server sends the generated message to the terminal, which then presents it to the user. The input is the generated message, and the output is the visual alert presented to the user by the terminal. The terminal uses its user interface to display the message and prompt the user to take action to mitigate the risk.

[0090] (Application Example 1)

[0091] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] Maintaining safety within the home requires identifying potential hazards early and taking appropriate measures. However, conventional methods require users to recognize hazards and devise countermeasures themselves, which is time-consuming and laborious. This invention aims to solve these difficulties and provide an efficient method for automatically enhancing safety within the home.

[0093] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0094] In this invention, the server includes means having a user interface for transmitting captured image data to an information processing device in order to identify dangerous areas in the home; means using an information processing device that analyzes the received image data and executes a generative model to identify risks; means including a message generation unit that presents appropriate countermeasures based on the risk identification; means including a communication unit that proposes safety measures based on the identified risks; and means having an output unit that notifies a smart device of the proposed countermeasures. This reduces the need for the user to discover dangers themselves and makes it possible to efficiently implement safety measures in the home.

[0095] "Dangerous areas in the home" refers to parts of the living space where users are prone to accidents or problems.

[0096] An "information processing device" is an electronic device used to receive and analyze digital data.

[0097] "Image data" refers to visual information collected by cameras or other photographic devices, represented in digital format.

[0098] A "user interface" refers to the means or screens that allow a user to interact with a device, enabling information input and result display.

[0099] A "generative model" is an algorithm created based on pre-collected data, used to analyze and identify specific patterns or features.

[0100] The "message generation unit" refers to a function that automatically creates a message to notify the target person.

[0101] A "communication unit" is a technical element equipped with an interface for sending and receiving data to and from other devices or servers.

[0102] The "output unit" is the part of an information processing device that has the function of physically displaying or transmitting data or messages generated by the device.

[0103] To implement this invention, a system utilizing electronic equipment including an information processing device is configured. This system includes various elements for identifying potential hazards within the home and proposing safety measures.

[0104] First, the user takes a picture of an area of ​​interest in their home using a smartphone or smart glasses. The captured image data is transmitted to the information processing device via the terminal's user interface. The terminal has a built-in camera module and communication module, enabling smooth data input and transmission.

[0105] The server is responsible for processing the received image data. Specifically, the server runs generative AI models such as TENSORFLOW® and PyTorch to identify risks from the received visual information. For example, it might detect scattered cables on the floor or sharp-edged furniture. The generative models used here are trained on datasets used to assess risks in advance.

[0106] Once a risk is identified, the server activates its message generation unit to generate a message outlining appropriate preventative measures. This message is sent from the server to the terminal and displayed on the user's smart device. The display incorporates visual alerts to ensure the user's attention is immediately drawn to the message. This information, provided by the output unit, serves as a guide for taking physical countermeasures.

[0107] For example, when a user takes a picture of their living room, image analysis might detect unorganized cables next to the sofa. The server then uses this result to generate a message such as, "Please use clips to secure the cables to prevent tripping," and sends this message to the user's device. Additionally, prompts such as, "Please take a picture of your front door and send it. We will diagnose potential hazards," can be displayed.

[0108] This system provides users with a practical means to easily maintain safety within their homes and can significantly reduce the risk of accidents.

[0109] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0110] Step 1:

[0111] Users use their smartphone cameras to photograph areas of interest within their homes. This input data is saved on the device as an image file.

[0112] Step 2:

[0113] The terminal selects images captured via a user interface and sends the image data to the server using a secure communication protocol (e.g., TLS). The input is image data, and the output is data to be sent to the server.

[0114] Step 3:

[0115] The server retrieves the received image data, uses a generative AI model to analyze the potential risks within the images, and generates risk assessment information. The input is the transmitted image data, and the output is the risk assessment result. TensorFlow or PyTorch are used for this analysis.

[0116] Step 4:

[0117] Based on the risk assessment, the server uses a message generation unit to create a message that warns the user. This message includes specific countermeasures for the identified hazards. The input is risk assessment information, and the output is a countermeasure message for the user.

[0118] Step 5:

[0119] The server sends the generated message to the terminal via the communication unit. The terminal displays the received message using its output unit, providing the user with visual feedback. The message may include specific suggestions, such as "Use clips to secure the cable to prevent tripping." The input is message data from the server, and the output is the information displayed on the terminal's screen.

[0120] Step 6:

[0121] Based on the safety messages displayed on the device, users take actual safety measures at specific locations within their homes. This action reduces the risk of accidents within the home.

[0122] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0123] This invention provides a system that enables accident prevention within the home and interaction based on user emotions. The system consists of a user, a terminal, a server, and an emotion engine.

[0124] The user uses the device's camera to photograph a specific area within their home and transmits the image data. The device receives this image data through its user interface, converts it to a secure digital format, and then transfers it to the server.

[0125] The server analyzes the received image data using a generative model to identify potential hazards. This process involves referencing a near-miss database to determine hazards such as sharp edges on furniture or scattered objects. It also utilizes an emotion engine to analyze emotional data received from the user. This data is used to assess stress levels and emotional states based on the user's facial expressions and voice.

[0126] Based on these analysis results, the server generates appropriate countermeasures according to the characteristics of the risk and the user's emotions. These countermeasures are customized by the message generation unit and sent back to the terminal.

[0127] The terminal displays messages received from the server via a user interface. The tone and approach of the messages are adjusted to take into account the user's emotional state. For example, if the user is stressed, the system prioritizes suggesting simple, immediate solutions.

[0128] For example, if a user sends a photo of scattered toys in their living room, the server uses a generative model to identify the risks and an emotion engine to assess the user's anxiety level. The server then sends back suggested solutions to the device that address the risks and take the user's feelings into consideration. For instance, the user might receive a tailored message such as, "You'll feel more at ease if you tidy up the toys this way."

[0129] In this way, this system helps prevent accidents in the home and provides flexible interaction functions that respond to the user's emotions, thereby encouraging the implementation of effective safety measures.

[0130] The following describes the processing flow.

[0131] Step 1:

[0132] The user takes photos of areas of interest in their home using a smartphone or tablet. The user grants permission to provide the system with facial expressions and voice recordings to document their emotional state. After taking the photos, the user sends the image data to the server via the user interface.

[0133] Step 2:

[0134] The device encodes the image and sentiment data sent by the user into the appropriate format. The device then uses a secure communication protocol to send the data to the server, ensuring data confidentiality.

[0135] Step 3:

[0136] The server analyzes image data and emotion data received from the terminal. Using a generative model, the server identifies risks within the images and uses an emotion engine to analyze the user's emotional state. Specifically, it evaluates the stress level and anxiety level indicated by the user's emotions.

[0137] Step 4:

[0138] The server generates individualized countermeasures based on identified risk information and the user's emotional state. The message generation unit creates messages that include a tone that takes emotional data into account and specific countermeasures. For example, if the user is feeling stressed, it prioritizes presenting simple countermeasures and risk reduction methods.

[0139] Step 5:

[0140] The server sends the generated message to the terminal using an encryption protocol.

[0141] Step 6:

[0142] The terminal displays messages received from the server in its user interface. The user reviews the displayed messages and understands the security measures that should be taken based on them. The terminal interface is configured to provide guidance in a gentle tone that is sensitive to the user's feelings.

[0143] Step 7:

[0144] Users implement the suggested measures to improve safety within their homes. This process, through empathetic follow-up, promotes sustainable safety maintenance behaviors.

[0145] (Example 2)

[0146] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0147] The challenge lies in preventing accidents within the home and realizing two-way information communication that takes into account the user's emotional state. In particular, it is necessary to quickly identify potential hazards that could cause unexpected accidents and to provide appropriate countermeasures that take into account the user's emotional response. Conventional systems have often failed to integrate physical risk assessment and emotional state analysis, treating them as separate functions. To solve this problem, advanced technology is needed that simultaneously analyzes risk factors and considers emotions.

[0148] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0149] In this invention, the server includes a person who transmits data via a terminal with video recording capabilities to acquire data on the home environment; a means; a function and means that uses a computing device to process the received environmental data and execute a generation algorithm for identifying potential risks; and a means that includes an information generation unit that presents countermeasures tailored based on potential risks and the user's emotional assessment. This makes it possible to identify potential hazards in the user's environment and automatically provide safety measures and emotionally sensitive feedback.

[0150] "The home environment" refers to the physical space within a house, the objects within it, their arrangement, and the overall circumstances associated with them.

[0151] "Video recording function" refers to technology that can capture visual information as an image or video, and then save or transmit it.

[0152] A "terminal" refers to an electronic device used for acquiring, processing, transmitting, and receiving data.

[0153] "Potential risks" refer to factors that are not yet apparent but, if their existence were confirmed, could potentially cause accidents or dangers.

[0154] A "generative algorithm" refers to a set of computational procedures performed on a computer to analyze data and obtain specific judgments or results.

[0155] A "computational device" refers to a system consisting of hardware and software for processing and analyzing data.

[0156] "Emotional evaluation" refers to the process of analyzing and understanding a user's psychological state using numerical values ​​or categories.

[0157] The "information generation unit" refers to a component that has the function of generating information and countermeasures to be presented to the user based on the analysis results.

[0158] The system of this invention provides accident prevention within the home and information communication that responds to the user's emotions. The system mainly consists of a terminal, a server, a generative AI model, and an emotion engine.

[0159] The user uses a device equipped with video recording capabilities to film specific areas within their home. The device features a high-resolution camera that captures the environment. The captured data is first converted into a secure digital format via a user interface. This conversion process utilizes data compression and format conversion technologies. The converted data is then transmitted to a server using secure communication methods. Data protection technologies, such as the SSL / TLS protocol, are applied during transmission.

[0160] The server analyzes the received data. A generative AI model is used for the analysis. This model is based on machine learning algorithms and identifies potential risk factors by cross-referencing it with a near-miss database. The server also evaluates facial expressions and voice data received from the user using an emotion engine. The emotion engine is software that quantifies the user's psychological state and analyzes stress and anxiety levels.

[0161] Using this data, the server generates countermeasures through its information generation unit. This process, based on a generation AI model, automatically generates countermeasures that are customized to take the user's emotional state into consideration. For example, a possible prompt might be, "I sent a photo of my living room. Please provide information on any security issues and any related advice."

[0162] The countermeasures generated by the system are sent back to the terminal and presented to the user via a user interface. This interface is designed to provide feedback through both sight and sound, allowing the user to intuitively understand and act upon the information. This enables the effective identification and elimination of potential hazards within the home, while simultaneously providing emotionally sensitive interactions.

[0163] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0164] Step 1:

[0165] The user uses the device's camera to photograph a specific area within their home. As input, the user provides the device with image data of the physical environment. This image data is converted directly into a digital format within the device. This conversion involves data compression and format conversion, resulting in an image format suitable for analysis (e.g., JPEG to binary data).

[0166] Step 2:

[0167] The terminal sends converted digital image data to the server. As input, the terminal has compressed and converted digital image data. This data is securely transferred to the server using communication methods such as the SSL / TLS protocol. The output is the data received by the server.

[0168] Step 3:

[0169] The server analyzes the received image data. The input is digital image data sent from the terminal. The server applies a generative AI model and compares it with a near-miss database to identify potential risk factors. The output is a list of identified risks. Here, the AI ​​model identifies, for example, sharp-angled furniture or scattered objects.

[0170] Step 4:

[0171] The server receives and analyzes emotional data from the user. The input consists of the user's facial expressions and voice data. Using an emotion engine, the server evaluates the user's emotional state based on this data. The output is a numerical evaluation of the user's emotional state. The emotional state is expressed as levels of stress and anxiety.

[0172] Step 5:

[0173] The server generates countermeasures based on the identified risks and emotional state assessments. The inputs are a list of risks and emotional state evaluation values ​​obtained from steps 3 and 4. An AI generation model is used to generate appropriate countermeasures for the risks, which are then customized in the message generation unit according to the user's emotional state. The output is the adjusted countermeasure message.

[0174] Step 6:

[0175] The terminal receives a response message sent from the server and displays it to the user via a user interface. The input is a pre-configured message from the server. The terminal displays this message visually and audibly in a human-readable format. As output, the user receives the displayed message and can take the necessary actions.

[0176] (Application Example 2)

[0177] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0178] The problem that this invention aims to solve is to improve safety in the work environment and to provide appropriate feedback functions to reduce operator stress and anxiety. Conventional systems have difficulty identifying potential hazards in real time and suggesting countermeasures that are appropriate to the operator's emotional state. As a result, the safety of the work environment is not sufficiently ensured, and the anxiety and stress experienced by operators are not alleviated.

[0179] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0180] In this invention, the server includes means for receiving and analyzing environmental data of the workspace from a terminal, means for identifying potential risks using a generative AI model, and means for determining the operator's emotional state using an emotion analysis engine and adjusting countermeasure messages accordingly. This not only improves the safety of the workspace but also provides flexible feedback that takes the operator's feelings into consideration, making it possible to create an environment where the operator can work with peace of mind.

[0181] "Environmental data" refers to images and video information taken to capture conditions within a home or workspace.

[0182] An "information processing device" is a combination of hardware and software used to analyze and process received data.

[0183] A "user interface" is a component of a system that provides functions for users to input or output information.

[0184] A "generative AI model" is an algorithm that utilizes artificial intelligence to extract and analyze necessary information from received data.

[0185] "Potential risks" refer to factors or situations that, even if not currently apparent, could potentially cause accidents or dangers in the future.

[0186] An "emotion analysis engine" is a type of software that analyzes a user's facial expressions and voice to evaluate their emotional state.

[0187] "Emotional state" refers to data that is evaluated as indicating the user's mental or psychological response or situation.

[0188] A "countermeasure message" is a written document containing suggestions and instructions provided to the user based on identified risks.

[0189] This invention is a system that uses an "information processing device" for collecting "environmental data" and a "generative AI model" for analyzing the collected data to identify potential hazards and generate appropriate "countermeasure messages." The system consists of the following hardware and software.

[0190] The server receives "environmental data" sent from the terminal and analyzes the images and videos captured by the "generative AI model." This model utilizes AI algorithms (e.g., frameworks such as TensorFlow and PyTorch) to identify "potential risks" in the home or workspace.

[0191] The device provides the user with the aforementioned "countermeasure message" through its "user interface." This allows the user to visually understand the situation and take action to ensure their safety by following the instructions.

[0192] Furthermore, the server uses an "emotion analysis engine" to analyze the operator's voice tone and facial expressions, and evaluates their "emotional state." This result is used to customize the content and tone of the messages presented to match the user's psychological state.

[0193] For example, if sharp tools are scattered around the workspace, the "generative AI model" identifies the situation as dangerous, and the "sentiment analysis engine" detects the operator's stress. The system can then display a message such as, "Please move these tools to a safe location. This will make the work environment safer."

[0194] An example of a prompt for a generating AI model is: "Evaluate the safety of the workspace based on image data, analyze the operator's tone of voice, and suggest safety measures if they appear stressed."

[0195] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0196] Step 1:

[0197] Users capture images of their home or workspace using a camera-equipped device, generating "environmental data." The input is image data, which is converted to a secure digital format as output. The device then transmits this data to a server using an encryption protocol.

[0198] Step 2:

[0199] The server receives image data sent from the terminal. The input is encrypted image data, and the output is decrypted image data. The server converts the received data into an analyzable format by decrypting the end-to-end encryption.

[0200] Step 3:

[0201] The server analyzes the decoded image data using a "generative AI model" to identify "potential risks." The input is the decoded image data, and the output is the identified risk information. Specifically, the AI ​​model detects and labels hazardous elements within the image.

[0202] Step 4:

[0203] The server receives voice or facial expression data from the operator and uses an "emotion analysis engine" to evaluate their "emotional state." The input is voice or video data, and the output is the evaluation result of the emotional state. Specifically, it uses voice tone analysis and facial expression recognition technology to quantify stress and anxiety levels.

[0204] Step 5:

[0205] The server generates a "countermeasure message" based on identified risk information and emotional state assessment results. The input is risk information and emotional assessment data, and the output is a customized message. Specifically, natural language generation technology is used to adjust the tone of the message to match the user's psychological state.

[0206] Step 6:

[0207] The terminal displays a "countermeasure message" generated by the server to the user through a user interface. The input is the message sent from the server, and the output is the display information that the user can see on the screen. Specifically, the display device visually shows the user the countermeasures and encourages safe actions.

[0208] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0209] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0210] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0211] [Second Embodiment]

[0212] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0213] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0214] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0215] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0216] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0217] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0218] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0219] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0220] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0221] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0222] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0223] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0224] This invention is a system for preventing accidents within the home. This system involves the collaborative operation of user, terminal, and server elements.

[0225] First, the user takes pictures of areas of concern in their home using the device's camera. This is intended to identify accident-prone areas, such as scattered cables or dangerous corners, and sends the images. The user then transfers these images to the server in digital data format, following the instructions on the device's built-in user interface.

[0226] The terminal properly formats the image data received from the user and transmits it to the server using a secure communication protocol. This ensures the confidentiality and integrity of the data.

[0227] The server runs a generative model to analyze the received image data. Based on a pre-trained near-miss database, the generative model identifies potential hazards in the image and assesses the risk. This assessment process, for example, recognizes slippery floors or sharp objects in the image and determines their likelihood of causing an accident.

[0228] Once a risk is identified, the server uses its message generation unit to create a message about safety measures. This message proposes preventative measures for the specific risk identified. The server sends this information back to the terminal and also provides the user with visual alerts and informational displays.

[0229] As a concrete example, suppose a user sends an image taken in their living room to the server. This image shows unorganized cables next to the sofa. The server uses a generative model to identify the risk, considers measures to secure the cables to prevent tripping, and displays these suggestions to the user.

[0230] Thus, this system provides a practical and effective solution for dramatically reducing the risk of accidents in the home.

[0231] The following describes the processing flow.

[0232] Step 1:

[0233] The user takes a picture of an area of ​​interest in their home using the device's camera. After taking the picture, they select the image through the user interface and send the image data to the system.

[0234] Step 2:

[0235] The terminal receives image data sent by the user. Next, it converts this data to an appropriate digital format (e.g., JPEG), applies an encryption protocol to the data, and securely transmits it to the server.

[0236] Step 3:

[0237] The server decodes the image data received from the terminal and converts it into a format that can be processed. This prepares the data for analysis using a generative model.

[0238] Step 4:

[0239] The server runs a generative model and analyzes the received images. The model consults a near-miss database to identify potential hazards in the images. For example, it can recognize code that could cause tripping or sharp corners of furniture.

[0240] Step 5:

[0241] The server generates a message suggesting appropriate security measures based on the identified risks. This message includes specific countermeasures and aims to provide useful information to the user.

[0242] Step 6:

[0243] The server sends the generated risk assessment results and countermeasure messages to the terminal. Communication is conducted securely using an encrypted protocol.

[0244] Step 7:

[0245] The terminal receives messages from the server and presents them visually to the user through the user interface. The user can then use this information to take necessary safety measures within their home.

[0246] (Example 1)

[0247] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0248] Accidents within the home are a serious problem, especially in environments with elderly people or children. Current technology makes it difficult to efficiently and continuously identify risks within the home and propose appropriate countermeasures, resulting in insufficient improvements in safety. Therefore, there is a need for a system that can automatically detect potential risks within the home and propose rapid countermeasures.

[0249] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0250] In this invention, the server includes means for providing an interface for a user to transmit image data to a data processing device to identify risks within the home; means for using a data processing device that examines the received image data and executes a generation algorithm to identify hazards; means for providing a message generation unit that suggests preventive measures based on the identified hazards; and means for using an encrypted communication protocol to maintain the confidentiality and integrity of communications. This makes it possible to effectively and quickly reduce the risk of accidents within the home.

[0251] "Household risks" refer to situations or objects within the home that could potentially cause accidents.

[0252] A "data processing device" refers to a computer system that takes data as input and analyzes that data based on specific processing steps.

[0253] "Image data" refers to data containing visual information expressed in a digital format.

[0254] A "user interface" refers to a user interface that is operable by the user and enables interaction with devices and systems.

[0255] A "generative algorithm" refers to a set of computational procedures or methods used to automatically extract specific information from data.

[0256] "Identifying hazards" refers to identifying factors that could potentially cause accidents or damage in a given area or object.

[0257] A "message generation unit" refers to a device or program that has the function of automatically creating text or suggestions to be conveyed to the user based on specific information.

[0258] An "encrypted communication protocol" refers to a communication method used to protect data from being read by third parties when sending or receiving data.

[0259] "Confidentiality and integrity of communications" refers to the information security requirement that transmitted data is not leaked to others and is received without being altered.

[0260] This invention is a system aimed at preventing accidents in the home, in which the user, terminal, and server elements work together.

[0261] The user first takes photos of areas of concern in their home, such as scattered cables or sharp furniture corners, using the device's camera. The device is equipped with a camera that has sensor functions for generating digital data. The captured image data is temporarily stored on the device.

[0262] The device formats the saved images appropriately, encrypts them, and then sends them to the server. Secure communication protocols such as HTTPS are used for this purpose. The server functions as a data processing unit to receive this secure communication.

[0263] The server runs a generative AI model, which has been previously trained using a near-miss database, to analyze the received image data. This AI model automatically identifies potential hazards within the image and performs a risk assessment. For example, it might highlight areas where people are prone to slipping or tripping.

[0264] Once a risk is identified, the server's message generation unit is utilized to generate a message containing specific preventative measures for that particular risk. This information is sent back to the terminal and presented to the user with a visual alert.

[0265] For example, if a user takes a picture in their living room and it's sent from their device to the server, and that picture includes scattered cables next to the sofa, the server's generative model will analyze the risk. It will assess the risk of tripping and generate a message recommending that the cables be tidied up.

[0266] An example of a prompt might be, "Analyze the image of the living room and suggest safety measures." This approach allows the system to provide useful solutions that dramatically reduce the risk of accidents in the home.

[0267] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0268] Step 1:

[0269] The user takes a picture of an area of ​​concern in their home using the device's camera. The input is the image captured by the camera, and the output is image data saved in digital format. Specifically, the user might take a picture of cluttered cables or sharp furniture corners, and the image data is temporarily stored on the device.

[0270] Step 2:

[0271] The terminal formats the stored images into a secure format and prepares them for transmission to the server. The input is the stored image data, and the output is encrypted image data. Specifically, the terminal compresses the images into a format such as JPEG and encrypts the data using the HTTPS protocol.

[0272] Step 3:

[0273] The terminal sends encrypted image data to the server. The input is encrypted image data, and the output is the image data received by the server. Here, the terminal uses the HTTPS protocol to securely transmit the image data over the network.

[0274] Step 4:

[0275] The server inputs the received image data into a generating AI model to perform a risk assessment. The input is the received image data, and the output is the result of the risk assessment. Specifically, the server identifies hazardous elements within the image, such as slippery areas or sharp objects.

[0276] Step 5:

[0277] The server generates safety messages based on evaluation results from a generative AI model. The input is the result of the risk assessment, and the output is a message containing specific preventive measures. The server creates suggestions for the identified risks, such as a message like "Secure the cables."

[0278] Step 6:

[0279] The server sends the generated message to the terminal, which then presents it to the user. The input is the generated message, and the output is the visual alert presented to the user by the terminal. The terminal uses its user interface to display the message and prompt the user to take action to mitigate the risk.

[0280] (Application Example 1)

[0281] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0282] In order to maintain safety within a home, it is necessary to identify potential dangers at an early stage and take appropriate measures. However, with conventional methods, the user had to recognize the danger themselves and consider countermeasures, which was laborious and time-consuming. The present invention aims to solve these difficulties and provide an efficient method for automatically enhancing the safety within a home.

[0283] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0284] In this invention, the server includes means having a user interface for transmitting image data captured by an information processing device to identify dangerous locations within a home, means using an information processing device that analyzes the received image data and executes a generation model for identifying risks, means including a message generation unit for presenting appropriate countermeasures based on risk identification, means having a communication unit for proposing safety countermeasures based on the identified risks, and means having an output unit for notifying the proposed countermeasures to a smart device. As a result, it becomes possible to reduce the need for the user to discover dangers themselves and efficiently take safety measures within the home.

[0285] The "dangerous locations within a home" refer to parts in the space where the user lives that are prone to accidents or troubles.

[0286] The "information processing device" is an electronic device used to receive and analyze digital data.

[0287] The "image data" is visual information collected by a camera or other imaging device expressed in digital form.

[0288] A "user interface" refers to the means or screens that allow a user to interact with a device, enabling information input and result display.

[0289] A "generative model" is an algorithm created based on pre-collected data, used to analyze and identify specific patterns or features.

[0290] The "message generation unit" refers to a function that automatically creates a message to notify the target person.

[0291] A "communication unit" is a technical element equipped with an interface for sending and receiving data to and from other devices or servers.

[0292] The "output unit" is the part of an information processing device that has the function of physically displaying or transmitting data or messages generated by the device.

[0293] To implement this invention, a system utilizing electronic equipment including an information processing device is configured. This system includes various elements for identifying potential hazards within the home and proposing safety measures.

[0294] First, the user takes a picture of an area of ​​interest in their home using a smartphone or smart glasses. The captured image data is transmitted to the information processing device via the terminal's user interface. The terminal has a built-in camera module and communication module, enabling smooth data input and transmission.

[0295] The server is responsible for processing the received image data. Specifically, the server runs generative AI models such as TensorFlow or PyTorch to identify risks from the received visual information. For example, it might detect scattered cables on the floor or sharp-edged furniture. The generative models used here are trained on datasets used to assess risks in advance.

[0296] Once a risk is identified, the server activates its message generation unit to generate a message outlining appropriate preventative measures. This message is sent from the server to the terminal and displayed on the user's smart device. The display incorporates visual alerts to ensure the user's attention is immediately drawn to the message. This information, provided by the output unit, serves as a guide for taking physical countermeasures.

[0297] For example, when a user takes a picture of their living room, image analysis might detect unorganized cables next to the sofa. The server then uses this result to generate a message such as, "Please use clips to secure the cables to prevent tripping," and sends this message to the user's device. Additionally, prompts such as, "Please take a picture of your front door and send it. We will diagnose potential hazards," can be displayed.

[0298] This system provides users with a practical means to easily maintain safety within their homes and can significantly reduce the risk of accidents.

[0299] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0300] Step 1:

[0301] Users use their smartphone cameras to photograph areas of interest within their homes. This input data is saved on the device as an image file.

[0302] Step 2:

[0303] The terminal selects images captured via a user interface and sends the image data to the server using a secure communication protocol (e.g., TLS). The input is image data, and the output is data to be sent to the server.

[0304] Step 3:

[0305] The server acquires the received image data, analyzes the potential risks in the image using a generative AI model, and generates risk assessment information. Here, the input is the transmitted image data, and the output is the risk assessment result. TensorFlow or PyTorch is used for this analysis.

[0306] Step 4:

[0307] Based on the risk assessment, the server creates a message using the message generation unit to inform the user of a warning. This message includes specific countermeasures for the identified risks. The input is the risk assessment information, and the output is a countermeasure message for the user.

[0308] Step 5:

[0309] The server transmits the generated message to the terminal through the communication unit. The terminal displays the received message using the output unit, providing visual feedback to the user. The message includes, for example, a specific proposal such as "Please use clips to fix the cable to prevent tripping." The input is the message data from the server, and the output is the information displayed on the terminal screen.

[0310] Step 6:

[0311] Based on the countermeasure message displayed on the terminal, the user takes actual safety measures for specific locations within the home. Through this operation, the accident risk within the home is reduced.

[0312] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform specific processing using the user's emotion.

[0313] The present invention is a system that enables accident prevention within the home and interaction based on the user's emotion. This system is composed of a user, a terminal, a server, and an emotion engine.

[0314] The user uses the device's camera to photograph a specific area within their home and transmits the image data. The device receives this image data through its user interface, converts it to a secure digital format, and then transfers it to the server.

[0315] The server analyzes the received image data using a generative model to identify potential hazards. This process involves referencing a near-miss database to determine hazards such as sharp edges on furniture or scattered objects. It also utilizes an emotion engine to analyze emotional data received from the user. This data is used to assess stress levels and emotional states based on the user's facial expressions and voice.

[0316] Based on these analysis results, the server generates appropriate countermeasures according to the characteristics of the risk and the user's emotions. These countermeasures are customized by the message generation unit and sent back to the terminal.

[0317] The terminal displays messages received from the server via a user interface. The tone and approach of the messages are adjusted to take into account the user's emotional state. For example, if the user is stressed, the system prioritizes suggesting simple, immediate solutions.

[0318] For example, if a user sends a photo of scattered toys in their living room, the server uses a generative model to identify the risks and an emotion engine to assess the user's anxiety level. The server then sends back suggested solutions to the device that address the risks and take the user's feelings into consideration. For instance, the user might receive a tailored message such as, "You'll feel more at ease if you tidy up the toys this way."

[0319] In this way, this system helps prevent accidents in the home and provides flexible interaction functions that respond to the user's emotions, thereby encouraging the implementation of effective safety measures.

[0320] The following describes the processing flow.

[0321] Step 1:

[0322] The user takes photos of areas of interest in their home using a smartphone or tablet. The user grants permission to provide the system with facial expressions and voice recordings to document their emotional state. After taking the photos, the user sends the image data to the server via the user interface.

[0323] Step 2:

[0324] The device encodes the image and sentiment data sent by the user into the appropriate format. The device then uses a secure communication protocol to send the data to the server, ensuring data confidentiality.

[0325] Step 3:

[0326] The server analyzes image data and emotion data received from the terminal. Using a generative model, the server identifies risks within the images and uses an emotion engine to analyze the user's emotional state. Specifically, it evaluates the stress level and anxiety level indicated by the user's emotions.

[0327] Step 4:

[0328] The server generates individualized countermeasures based on identified risk information and the user's emotional state. The message generation unit creates messages that include a tone that takes emotional data into account and specific countermeasures. For example, if the user is feeling stressed, it prioritizes presenting simple countermeasures and risk reduction methods.

[0329] Step 5:

[0330] The server sends the generated message to the terminal using an encryption protocol.

[0331] Step 6:

[0332] The terminal displays messages received from the server in its user interface. The user reviews the displayed messages and understands the security measures that should be taken based on them. The terminal interface is configured to provide guidance in a gentle tone that is sensitive to the user's feelings.

[0333] Step 7:

[0334] Users implement the suggested measures to improve safety within their homes. This process, through empathetic follow-up, promotes sustainable safety maintenance behaviors.

[0335] (Example 2)

[0336] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0337] The challenge lies in preventing accidents within the home and realizing two-way information communication that takes into account the user's emotional state. In particular, it is necessary to quickly identify potential hazards that could cause unexpected accidents and to provide appropriate countermeasures that take into account the user's emotional response. Conventional systems have often failed to integrate physical risk assessment and emotional state analysis, treating them as separate functions. To solve this problem, advanced technology is needed that simultaneously analyzes risk factors and considers emotions.

[0338] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0339] In this invention, the server includes a person who transmits data via a terminal with video recording capabilities to acquire data on the home environment; a means; a function and means that uses a computing device to process the received environmental data and execute a generation algorithm for identifying potential risks; and a means that includes an information generation unit that presents countermeasures tailored based on potential risks and the user's emotional assessment. This makes it possible to identify potential hazards in the user's environment and automatically provide safety measures and emotionally sensitive feedback.

[0340] "The home environment" refers to the physical space within a house, the objects within it, their arrangement, and the overall circumstances associated with them.

[0341] "Video recording function" refers to technology that can capture visual information as an image or video, and then save or transmit it.

[0342] A "terminal" refers to an electronic device used for acquiring, processing, transmitting, and receiving data.

[0343] "Potential risks" refer to factors that are not yet apparent but, if their existence were confirmed, could potentially cause accidents or dangers.

[0344] A "generative algorithm" refers to a set of computational procedures performed on a computer to analyze data and obtain specific judgments or results.

[0345] A "computational device" refers to a system consisting of hardware and software for processing and analyzing data.

[0346] "Emotional evaluation" refers to the process of analyzing and understanding a user's psychological state using numerical values ​​or categories.

[0347] The "information generation unit" refers to a component that has the function of generating information and countermeasures to be presented to the user based on the analysis results.

[0348] The system of this invention provides accident prevention within the home and information communication that responds to the user's emotions. The system mainly consists of a terminal, a server, a generative AI model, and an emotion engine.

[0349] The user uses a device equipped with video recording capabilities to film specific areas within their home. The device features a high-resolution camera that captures the environment. The captured data is first converted into a secure digital format via a user interface. This conversion process utilizes data compression and format conversion technologies. The converted data is then transmitted to a server using secure communication methods. Data protection technologies, such as the SSL / TLS protocol, are applied during transmission.

[0350] The server analyzes the received data. A generative AI model is used for the analysis. This model is based on machine learning algorithms and identifies potential risk factors by cross-referencing it with a near-miss database. The server also evaluates facial expressions and voice data received from the user using an emotion engine. The emotion engine is software that quantifies the user's psychological state and analyzes stress and anxiety levels.

[0351] Using this data, the server generates countermeasures through its information generation unit. This process, based on a generation AI model, automatically generates countermeasures that are customized to take the user's emotional state into consideration. For example, a possible prompt might be, "I sent a photo of my living room. Please provide information on any security issues and any related advice."

[0352] The countermeasures generated by the system are sent back to the terminal and presented to the user via a user interface. This interface is designed to provide feedback through both sight and sound, allowing the user to intuitively understand and act upon the information. This enables the effective identification and elimination of potential hazards within the home, while simultaneously providing emotionally sensitive interactions.

[0353] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0354] Step 1:

[0355] The user uses the device's camera to photograph a specific area within their home. As input, the user provides the device with image data of the physical environment. This image data is converted directly into a digital format within the device. This conversion involves data compression and format conversion, resulting in an image format suitable for analysis (e.g., JPEG to binary data).

[0356] Step 2:

[0357] The terminal sends converted digital image data to the server. As input, the terminal has compressed and converted digital image data. This data is securely transferred to the server using communication methods such as the SSL / TLS protocol. The output is the data received by the server.

[0358] Step 3:

[0359] The server analyzes the received image data. The input is digital image data sent from the terminal. The server applies a generative AI model and compares it with a near-miss database to identify potential risk factors. The output is a list of identified risks. Here, the AI ​​model identifies, for example, sharp-angled furniture or scattered objects.

[0360] Step 4:

[0361] The server receives and analyzes emotional data from the user. The input consists of the user's facial expressions and voice data. Using an emotion engine, the server evaluates the user's emotional state based on this data. The output is a numerical evaluation of the user's emotional state. The emotional state is expressed as levels of stress and anxiety.

[0362] Step 5:

[0363] The server generates countermeasures based on the identified risks and emotional state assessments. The inputs are a list of risks and emotional state evaluation values ​​obtained from steps 3 and 4. An AI generation model is used to generate appropriate countermeasures for the risks, which are then customized in the message generation unit according to the user's emotional state. The output is the adjusted countermeasure message.

[0364] Step 6:

[0365] The terminal receives a response message sent from the server and displays it to the user via a user interface. The input is a pre-configured message from the server. The terminal displays this message visually and audibly in a human-readable format. As output, the user receives the displayed message and can take the necessary actions.

[0366] (Application Example 2)

[0367] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0368] The problem that this invention aims to solve is to improve safety in the work environment and to provide appropriate feedback functions to reduce operator stress and anxiety. Conventional systems have difficulty identifying potential hazards in real time and suggesting countermeasures that are appropriate to the operator's emotional state. As a result, the safety of the work environment is not sufficiently ensured, and the anxiety and stress experienced by operators are not alleviated.

[0369] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0370] In this invention, the server includes means for receiving and analyzing environmental data of the workspace from a terminal, means for identifying potential risks using a generative AI model, and means for determining the operator's emotional state using an emotion analysis engine and adjusting countermeasure messages accordingly. This not only improves the safety of the workspace but also provides flexible feedback that takes the operator's feelings into consideration, making it possible to create an environment where the operator can work with peace of mind.

[0371] "Environmental data" refers to images and video information taken to capture conditions within a home or workspace.

[0372] An "information processing device" is a combination of hardware and software used to analyze and process received data.

[0373] A "user interface" is a component of a system that provides functions for users to input or output information.

[0374] A "generative AI model" is an algorithm that utilizes artificial intelligence to extract and analyze necessary information from received data.

[0375] "Potential risks" refer to factors or situations that, even if not currently apparent, could potentially cause accidents or dangers in the future.

[0376] An "emotion analysis engine" is a type of software that analyzes a user's facial expressions and voice to evaluate their emotional state.

[0377] "Emotional state" refers to data that is evaluated as indicating the user's mental or psychological response or situation.

[0378] A "countermeasure message" is a written document containing suggestions and instructions provided to the user based on identified risks.

[0379] This invention is a system that uses an "information processing device" for collecting "environmental data" and a "generative AI model" for analyzing the collected data to identify potential hazards and generate appropriate "countermeasure messages." The system consists of the following hardware and software.

[0380] The server receives "environmental data" sent from the terminal and analyzes the images and videos captured by the "generative AI model." This model utilizes AI algorithms (e.g., frameworks such as TensorFlow and PyTorch) to identify "potential risks" in the home or workspace.

[0381] The device provides the user with the aforementioned "countermeasure message" through its "user interface." This allows the user to visually understand the situation and take action to ensure their safety by following the instructions.

[0382] Furthermore, the server uses an "emotion analysis engine" to analyze the operator's voice tone and facial expressions, and evaluates their "emotional state." This result is used to customize the content and tone of the messages presented to match the user's psychological state.

[0383] For example, if sharp tools are scattered around the workspace, the "generative AI model" identifies the situation as dangerous, and the "sentiment analysis engine" detects the operator's stress. The system can then display a message such as, "Please move these tools to a safe location. This will make the work environment safer."

[0384] An example of a prompt for a generating AI model is: "Evaluate the safety of the workspace based on image data, analyze the operator's tone of voice, and suggest safety measures if they appear stressed."

[0385] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0386] Step 1:

[0387] Users capture images of their home or workspace using a camera-equipped device, generating "environmental data." The input is image data, which is converted to a secure digital format as output. The device then transmits this data to a server using an encryption protocol.

[0388] Step 2:

[0389] The server receives image data sent from the terminal. The input is encrypted image data, and the output is decrypted image data. The server converts the received data into an analyzable format by decrypting the end-to-end encryption.

[0390] Step 3:

[0391] The server analyzes the decoded image data using a "generative AI model" to identify "potential risks." The input is the decoded image data, and the output is the identified risk information. Specifically, the AI ​​model detects and labels hazardous elements within the image.

[0392] Step 4:

[0393] The server receives voice or facial expression data from the operator and uses an "emotion analysis engine" to evaluate their "emotional state." The input is voice or video data, and the output is the evaluation result of the emotional state. Specifically, it uses voice tone analysis and facial expression recognition technology to quantify stress and anxiety levels.

[0394] Step 5:

[0395] The server generates a "countermeasure message" based on identified risk information and emotional state assessment results. The input is risk information and emotional assessment data, and the output is a customized message. Specifically, natural language generation technology is used to adjust the tone of the message to match the user's psychological state.

[0396] Step 6:

[0397] The terminal displays a "countermeasure message" generated by the server to the user through a user interface. The input is the message sent from the server, and the output is the display information that the user can see on the screen. Specifically, the display device visually shows the user the countermeasures and encourages safe actions.

[0398] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0399] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0400] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0401] [Third Embodiment]

[0402] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0403] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0404] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0405] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0406] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0407] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0408] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0409] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0410] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0411] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0412] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0413] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0414] This invention is a system for preventing accidents within the home. This system involves the collaborative operation of user, terminal, and server elements.

[0415] First, the user takes pictures of areas of concern in their home using the device's camera. This is intended to identify accident-prone areas, such as scattered cables or dangerous corners, and sends the images. The user then transfers these images to the server in digital data format, following the instructions on the device's built-in user interface.

[0416] The terminal properly formats the image data received from the user and transmits it to the server using a secure communication protocol. This ensures the confidentiality and integrity of the data.

[0417] The server runs a generative model to analyze the received image data. Based on a pre-trained near-miss database, the generative model identifies potential hazards in the image and assesses the risk. This assessment process, for example, recognizes slippery floors or sharp objects in the image and determines their likelihood of causing an accident.

[0418] Once a risk is identified, the server uses its message generation unit to create a message about safety measures. This message proposes preventative measures for the specific risk identified. The server sends this information back to the terminal and also provides the user with visual alerts and informational displays.

[0419] As a concrete example, suppose a user sends an image taken in their living room to the server. This image shows unorganized cables next to the sofa. The server uses a generative model to identify the risk, considers measures to secure the cables to prevent tripping, and displays these suggestions to the user.

[0420] Thus, this system provides a practical and effective solution for dramatically reducing the risk of accidents in the home.

[0421] The following describes the processing flow.

[0422] Step 1:

[0423] The user takes a picture of an area of ​​interest in their home using the device's camera. After taking the picture, they select the image through the user interface and send the image data to the system.

[0424] Step 2:

[0425] The terminal receives image data sent by the user. Next, it converts this data to an appropriate digital format (e.g., JPEG), applies an encryption protocol to the data, and securely transmits it to the server.

[0426] Step 3:

[0427] The server decodes the image data received from the terminal and converts it into a format that can be processed. This prepares the data for analysis using a generative model.

[0428] Step 4:

[0429] The server runs a generative model and analyzes the received images. The model consults a near-miss database to identify potential hazards in the images. For example, it can recognize code that could cause tripping or sharp corners of furniture.

[0430] Step 5:

[0431] The server generates a message suggesting appropriate security measures based on the identified risks. This message includes specific countermeasures and aims to provide useful information to the user.

[0432] Step 6:

[0433] The server sends the generated risk assessment results and countermeasure messages to the terminal. Communication is conducted securely using an encrypted protocol.

[0434] Step 7:

[0435] The terminal receives messages from the server and presents them visually to the user through the user interface. The user can then use this information to take necessary safety measures within their home.

[0436] (Example 1)

[0437] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0438] Accidents within the home are a serious problem, especially in environments with elderly people or children. Current technology makes it difficult to efficiently and continuously identify risks within the home and propose appropriate countermeasures, resulting in insufficient improvements in safety. Therefore, there is a need for a system that can automatically detect potential risks within the home and propose rapid countermeasures.

[0439] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0440] In this invention, the server includes means for providing an interface for a user to transmit image data to a data processing device to identify risks within the home; means for using a data processing device that examines the received image data and executes a generation algorithm to identify hazards; means for providing a message generation unit that suggests preventive measures based on the identified hazards; and means for using an encrypted communication protocol to maintain the confidentiality and integrity of communications. This makes it possible to effectively and quickly reduce the risk of accidents within the home.

[0441] "Household risks" refer to situations or objects within the home that could potentially cause accidents.

[0442] A "data processing device" refers to a computer system that takes data as input and analyzes that data based on specific processing steps.

[0443] "Image data" refers to data containing visual information expressed in a digital format.

[0444] A "user interface" refers to a user interface that is operable by the user and enables interaction with devices and systems.

[0445] A "generative algorithm" refers to a set of computational procedures or methods used to automatically extract specific information from data.

[0446] "Identifying hazards" refers to identifying factors that could potentially cause accidents or damage in a given area or object.

[0447] A "message generation unit" refers to a device or program that has the function of automatically creating text or suggestions to be conveyed to the user based on specific information.

[0448] An "encrypted communication protocol" refers to a communication method used to protect data from being read by third parties when sending or receiving data.

[0449] "Confidentiality and integrity of communications" refers to the information security requirement that transmitted data is not leaked to others and is received without being altered.

[0450] This invention is a system aimed at preventing accidents in the home, in which the user, terminal, and server elements work together.

[0451] The user first takes photos of areas of concern in their home, such as scattered cables or sharp furniture corners, using the device's camera. The device is equipped with a camera that has sensor functions for generating digital data. The captured image data is temporarily stored on the device.

[0452] The device formats the saved images appropriately, encrypts them, and then sends them to the server. Secure communication protocols such as HTTPS are used for this purpose. The server functions as a data processing unit to receive this secure communication.

[0453] The server runs a generative AI model, which has been previously trained using a near-miss database, to analyze the received image data. This AI model automatically identifies potential hazards within the image and performs a risk assessment. For example, it might highlight areas where people are prone to slipping or tripping.

[0454] Once a risk is identified, the server's message generation unit is utilized to generate a message containing specific preventative measures for that particular risk. This information is sent back to the terminal and presented to the user with a visual alert.

[0455] For example, if a user takes a picture in their living room and it's sent from their device to the server, and that picture includes scattered cables next to the sofa, the server's generative model will analyze the risk. It will assess the risk of tripping and generate a message recommending that the cables be tidied up.

[0456] An example of a prompt might be, "Analyze the image of the living room and suggest safety measures." This approach allows the system to provide useful solutions that dramatically reduce the risk of accidents in the home.

[0457] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0458] Step 1:

[0459] The user takes a picture of an area of ​​concern in their home using the device's camera. The input is the image captured by the camera, and the output is image data saved in digital format. Specifically, the user might take a picture of cluttered cables or sharp furniture corners, and the image data is temporarily stored on the device.

[0460] Step 2:

[0461] The terminal formats the stored images into a secure format and prepares them for transmission to the server. The input is the stored image data, and the output is encrypted image data. Specifically, the terminal compresses the images into a format such as JPEG and encrypts the data using the HTTPS protocol.

[0462] Step 3:

[0463] The terminal sends encrypted image data to the server. The input is encrypted image data, and the output is the image data received by the server. Here, the terminal uses the HTTPS protocol to securely transmit the image data over the network.

[0464] Step 4:

[0465] The server inputs the received image data into a generating AI model to perform a risk assessment. The input is the received image data, and the output is the result of the risk assessment. Specifically, the server identifies hazardous elements within the image, such as slippery areas or sharp objects.

[0466] Step 5:

[0467] The server generates safety messages based on evaluation results from a generative AI model. The input is the result of the risk assessment, and the output is a message containing specific preventive measures. The server creates suggestions for the identified risks, such as a message like "Secure the cables."

[0468] Step 6:

[0469] The server sends the generated message to the terminal, which then presents it to the user. The input is the generated message, and the output is the visual alert presented to the user by the terminal. The terminal uses its user interface to display the message and prompt the user to take action to mitigate the risk.

[0470] (Application Example 1)

[0471] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0472] Maintaining safety within the home requires identifying potential hazards early and taking appropriate measures. However, conventional methods require users to recognize hazards and devise countermeasures themselves, which is time-consuming and laborious. This invention aims to solve these difficulties and provide an efficient method for automatically enhancing safety within the home.

[0473] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0474] In this invention, the server includes means having a user interface for transmitting captured image data to an information processing device in order to identify dangerous areas in the home; means using an information processing device that analyzes the received image data and executes a generative model to identify risks; means including a message generation unit that presents appropriate countermeasures based on the risk identification; means including a communication unit that proposes safety measures based on the identified risks; and means having an output unit that notifies a smart device of the proposed countermeasures. This reduces the need for the user to discover dangers themselves and makes it possible to efficiently implement safety measures in the home.

[0475] "Dangerous areas in the home" refers to parts of the living space where users are prone to accidents or problems.

[0476] An "information processing device" is an electronic device used to receive and analyze digital data.

[0477] "Image data" refers to visual information collected by cameras or other photographic devices, represented in digital format.

[0478] A "user interface" refers to the means or screens that allow a user to interact with a device, enabling information input and result display.

[0479] A "generative model" is an algorithm created based on pre-collected data, used to analyze and identify specific patterns or features.

[0480] The "message generation unit" refers to a function that automatically creates a message to notify the target person.

[0481] A "communication unit" is a technical element equipped with an interface for sending and receiving data to and from other devices or servers.

[0482] The "output unit" is the part of an information processing device that has the function of physically displaying or transmitting data or messages generated by the device.

[0483] To implement this invention, a system utilizing electronic equipment including an information processing device is configured. This system includes various elements for identifying potential hazards within the home and proposing safety measures.

[0484] First, the user takes a picture of an area of ​​interest in their home using a smartphone or smart glasses. The captured image data is transmitted to the information processing device via the terminal's user interface. The terminal has a built-in camera module and communication module, enabling smooth data input and transmission.

[0485] The server is responsible for processing the received image data. Specifically, the server runs generative AI models such as TensorFlow or PyTorch to identify risks from the received visual information. For example, it might detect scattered cables on the floor or sharp-edged furniture. The generative models used here are trained on datasets used to assess risks in advance.

[0486] Once a risk is identified, the server activates its message generation unit to generate a message outlining appropriate preventative measures. This message is sent from the server to the terminal and displayed on the user's smart device. The display incorporates visual alerts to ensure the user's attention is immediately drawn to the message. This information, provided by the output unit, serves as a guide for taking physical countermeasures.

[0487] For example, when a user takes a picture of their living room, image analysis might detect unorganized cables next to the sofa. The server then uses this result to generate a message such as, "Please use clips to secure the cables to prevent tripping," and sends this message to the user's device. Additionally, prompts such as, "Please take a picture of your front door and send it. We will diagnose potential hazards," can be displayed.

[0488] This system provides users with a practical means to easily maintain safety within their homes and can significantly reduce the risk of accidents.

[0489] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0490] Step 1:

[0491] Users use their smartphone cameras to photograph areas of interest within their homes. This input data is saved on the device as an image file.

[0492] Step 2:

[0493] The terminal selects images captured via a user interface and sends the image data to the server using a secure communication protocol (e.g., TLS). The input is image data, and the output is data to be sent to the server.

[0494] Step 3:

[0495] The server retrieves the received image data, uses a generative AI model to analyze the potential risks within the images, and generates risk assessment information. The input is the transmitted image data, and the output is the risk assessment result. TensorFlow or PyTorch are used for this analysis.

[0496] Step 4:

[0497] Based on the risk assessment, the server uses a message generation unit to create a message that warns the user. This message includes specific countermeasures for the identified hazards. The input is risk assessment information, and the output is a countermeasure message for the user.

[0498] Step 5:

[0499] The server sends the generated message to the terminal via the communication unit. The terminal displays the received message using its output unit, providing the user with visual feedback. The message may include specific suggestions, such as "Use clips to secure the cable to prevent tripping." The input is message data from the server, and the output is the information displayed on the terminal's screen.

[0500] Step 6:

[0501] Based on the safety messages displayed on the device, users take actual safety measures at specific locations within their homes. This action reduces the risk of accidents within the home.

[0502] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0503] This invention provides a system that enables accident prevention within the home and interaction based on user emotions. The system consists of a user, a terminal, a server, and an emotion engine.

[0504] The user uses the device's camera to photograph a specific area within their home and transmits the image data. The device receives this image data through its user interface, converts it to a secure digital format, and then transfers it to the server.

[0505] The server analyzes the received image data using a generative model to identify potential hazards. This process involves referencing a near-miss database to determine hazards such as sharp edges on furniture or scattered objects. It also utilizes an emotion engine to analyze emotional data received from the user. This data is used to assess stress levels and emotional states based on the user's facial expressions and voice.

[0506] Based on these analysis results, the server generates appropriate countermeasures according to the characteristics of the risk and the user's emotions. These countermeasures are customized by the message generation unit and sent back to the terminal.

[0507] The terminal displays messages received from the server via a user interface. The tone and approach of the messages are adjusted to take into account the user's emotional state. For example, if the user is stressed, the system prioritizes suggesting simple, immediate solutions.

[0508] For example, if a user sends a photo of scattered toys in their living room, the server uses a generative model to identify the risks and an emotion engine to assess the user's anxiety level. The server then sends back suggested solutions to the device that address the risks and take the user's feelings into consideration. For instance, the user might receive a tailored message such as, "You'll feel more at ease if you tidy up the toys this way."

[0509] In this way, this system helps prevent accidents in the home and provides flexible interaction functions that respond to the user's emotions, thereby encouraging the implementation of effective safety measures.

[0510] The following describes the processing flow.

[0511] Step 1:

[0512] The user takes photos of areas of interest in their home using a smartphone or tablet. The user grants permission to provide the system with facial expressions and voice recordings to document their emotional state. After taking the photos, the user sends the image data to the server via the user interface.

[0513] Step 2:

[0514] The device encodes the image and sentiment data sent by the user into the appropriate format. The device then uses a secure communication protocol to send the data to the server, ensuring data confidentiality.

[0515] Step 3:

[0516] The server analyzes image data and emotion data received from the terminal. Using a generative model, the server identifies risks within the images and uses an emotion engine to analyze the user's emotional state. Specifically, it evaluates the stress level and anxiety level indicated by the user's emotions.

[0517] Step 4:

[0518] The server generates individualized countermeasures based on identified risk information and the user's emotional state. The message generation unit creates messages that include a tone that takes emotional data into account and specific countermeasures. For example, if the user is feeling stressed, it prioritizes presenting simple countermeasures and risk reduction methods.

[0519] Step 5:

[0520] The server sends the generated message to the terminal using an encryption protocol.

[0521] Step 6:

[0522] The terminal displays messages received from the server in its user interface. The user reviews the displayed messages and understands the security measures that should be taken based on them. The terminal interface is configured to provide guidance in a gentle tone that is sensitive to the user's feelings.

[0523] Step 7:

[0524] Users implement the suggested measures to improve safety within their homes. This process, through empathetic follow-up, promotes sustainable safety maintenance behaviors.

[0525] (Example 2)

[0526] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0527] The challenge lies in preventing accidents within the home and realizing two-way information communication that takes into account the user's emotional state. In particular, it is necessary to quickly identify potential hazards that could cause unexpected accidents and to provide appropriate countermeasures that take into account the user's emotional response. Conventional systems have often failed to integrate physical risk assessment and emotional state analysis, treating them as separate functions. To solve this problem, advanced technology is needed that simultaneously analyzes risk factors and considers emotions.

[0528] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0529] In this invention, the server includes a person who transmits data via a terminal with video recording capabilities to acquire data on the home environment; a means; a function and means that uses a computing device to process the received environmental data and execute a generation algorithm for identifying potential risks; and a means that includes an information generation unit that presents countermeasures tailored based on potential risks and the user's emotional assessment. This makes it possible to identify potential hazards in the user's environment and automatically provide safety measures and emotionally sensitive feedback.

[0530] "The home environment" refers to the physical space within a house, the objects within it, their arrangement, and the overall circumstances associated with them.

[0531] "Video recording function" refers to technology that can capture visual information as an image or video, and then save or transmit it.

[0532] A "terminal" refers to an electronic device used for acquiring, processing, transmitting, and receiving data.

[0533] "Potential risks" refer to factors that are not yet apparent but, if their existence were confirmed, could potentially cause accidents or dangers.

[0534] A "generative algorithm" refers to a set of computational procedures performed on a computer to analyze data and obtain specific judgments or results.

[0535] A "computational device" refers to a system consisting of hardware and software for processing and analyzing data.

[0536] "Emotional evaluation" refers to the process of analyzing and understanding a user's psychological state using numerical values ​​or categories.

[0537] The "information generation unit" refers to a component that has the function of generating information and countermeasures to be presented to the user based on the analysis results.

[0538] The system of this invention provides accident prevention within the home and information communication that responds to the user's emotions. The system mainly consists of a terminal, a server, a generative AI model, and an emotion engine.

[0539] The user uses a device equipped with video recording capabilities to film specific areas within their home. The device features a high-resolution camera that captures the environment. The captured data is first converted into a secure digital format via a user interface. This conversion process utilizes data compression and format conversion technologies. The converted data is then transmitted to a server using secure communication methods. Data protection technologies, such as the SSL / TLS protocol, are applied during transmission.

[0540] The server analyzes the received data. A generative AI model is used for the analysis. This model is based on machine learning algorithms and identifies potential risk factors by cross-referencing it with a near-miss database. The server also evaluates facial expressions and voice data received from the user using an emotion engine. The emotion engine is software that quantifies the user's psychological state and analyzes stress and anxiety levels.

[0541] Using this data, the server generates countermeasures through its information generation unit. This process, based on a generation AI model, automatically generates countermeasures that are customized to take the user's emotional state into consideration. For example, a possible prompt might be, "I sent a photo of my living room. Please provide information on any security issues and any related advice."

[0542] The countermeasures generated by the system are sent back to the terminal and presented to the user via a user interface. This interface is designed to provide feedback through both sight and sound, allowing the user to intuitively understand and act upon the information. This enables the effective identification and elimination of potential hazards within the home, while simultaneously providing emotionally sensitive interactions.

[0543] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0544] Step 1:

[0545] The user uses the device's camera to photograph a specific area within their home. As input, the user provides the device with image data of the physical environment. This image data is converted directly into a digital format within the device. This conversion involves data compression and format conversion, resulting in an image format suitable for analysis (e.g., JPEG to binary data).

[0546] Step 2:

[0547] The terminal sends converted digital image data to the server. As input, the terminal has compressed and converted digital image data. This data is securely transferred to the server using communication methods such as the SSL / TLS protocol. The output is the data received by the server.

[0548] Step 3:

[0549] The server analyzes the received image data. The input is digital image data sent from the terminal. The server applies a generative AI model and compares it with a near-miss database to identify potential risk factors. The output is a list of identified risks. Here, the AI ​​model identifies, for example, sharp-angled furniture or scattered objects.

[0550] Step 4:

[0551] The server receives and analyzes emotional data from the user. The input consists of the user's facial expressions and voice data. Using an emotion engine, the server evaluates the user's emotional state based on this data. The output is a numerical evaluation of the user's emotional state. The emotional state is expressed as levels of stress and anxiety.

[0552] Step 5:

[0553] The server generates countermeasures based on the identified risks and emotional state assessments. The inputs are a list of risks and emotional state evaluation values ​​obtained from steps 3 and 4. An AI generation model is used to generate appropriate countermeasures for the risks, which are then customized in the message generation unit according to the user's emotional state. The output is the adjusted countermeasure message.

[0554] Step 6:

[0555] The terminal receives a response message sent from the server and displays it to the user via a user interface. The input is a pre-configured message from the server. The terminal displays this message visually and audibly in a human-readable format. As output, the user receives the displayed message and can take the necessary actions.

[0556] (Application Example 2)

[0557] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0558] The problem that this invention aims to solve is to improve safety in the work environment and to provide appropriate feedback functions to reduce operator stress and anxiety. Conventional systems have difficulty identifying potential hazards in real time and suggesting countermeasures that are appropriate to the operator's emotional state. As a result, the safety of the work environment is not sufficiently ensured, and the anxiety and stress experienced by operators are not alleviated.

[0559] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0560] In this invention, the server includes means for receiving and analyzing environmental data of the workspace from a terminal, means for identifying potential risks using a generative AI model, and means for determining the operator's emotional state using an emotion analysis engine and adjusting countermeasure messages accordingly. This not only improves the safety of the workspace but also provides flexible feedback that takes the operator's feelings into consideration, making it possible to create an environment where the operator can work with peace of mind.

[0561] "Environmental data" refers to images and video information taken to capture conditions within a home or workspace.

[0562] An "information processing device" is a combination of hardware and software used to analyze and process received data.

[0563] A "user interface" is a component of a system that provides functions for users to input or output information.

[0564] A "generative AI model" is an algorithm that utilizes artificial intelligence to extract and analyze necessary information from received data.

[0565] "Potential risks" refer to factors or situations that, even if not currently apparent, could potentially cause accidents or dangers in the future.

[0566] An "emotion analysis engine" is a type of software that analyzes a user's facial expressions and voice to evaluate their emotional state.

[0567] "Emotional state" refers to data that is evaluated as indicating the user's mental or psychological response or situation.

[0568] A "countermeasure message" is a written document containing suggestions and instructions provided to the user based on identified risks.

[0569] This invention is a system that uses an "information processing device" for collecting "environmental data" and a "generative AI model" for analyzing the collected data to identify potential hazards and generate appropriate "countermeasure messages." The system consists of the following hardware and software.

[0570] The server receives "environmental data" sent from the terminal and analyzes the images and videos captured by the "generative AI model." This model utilizes AI algorithms (e.g., frameworks such as TensorFlow and PyTorch) to identify "potential risks" in the home or workspace.

[0571] The device provides the user with the aforementioned "countermeasure message" through its "user interface." This allows the user to visually understand the situation and take action to ensure their safety by following the instructions.

[0572] Furthermore, the server uses an "emotion analysis engine" to analyze the operator's voice tone and facial expressions, and evaluates their "emotional state." This result is used to customize the content and tone of the messages presented to match the user's psychological state.

[0573] For example, if sharp tools are scattered around the workspace, the "generative AI model" identifies the situation as dangerous, and the "sentiment analysis engine" detects the operator's stress. The system can then display a message such as, "Please move these tools to a safe location. This will make the work environment safer."

[0574] An example of a prompt for a generating AI model is: "Evaluate the safety of the workspace based on image data, analyze the operator's tone of voice, and suggest safety measures if they appear stressed."

[0575] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0576] Step 1:

[0577] Users capture images of their home or workspace using a camera-equipped device, generating "environmental data." The input is image data, which is converted to a secure digital format as output. The device then transmits this data to a server using an encryption protocol.

[0578] Step 2:

[0579] The server receives image data sent from the terminal. The input is encrypted image data, and the output is decrypted image data. The server converts the received data into an analyzable format by decrypting the end-to-end encryption.

[0580] Step 3:

[0581] The server analyzes the decoded image data using a "generative AI model" to identify "potential risks." The input is the decoded image data, and the output is the identified risk information. Specifically, the AI ​​model detects and labels hazardous elements within the image.

[0582] Step 4:

[0583] The server receives voice or facial expression data from the operator and uses an "emotion analysis engine" to evaluate their "emotional state." The input is voice or video data, and the output is the evaluation result of the emotional state. Specifically, it uses voice tone analysis and facial expression recognition technology to quantify stress and anxiety levels.

[0584] Step 5:

[0585] The server generates a "countermeasure message" based on identified risk information and emotional state assessment results. The input is risk information and emotional assessment data, and the output is a customized message. Specifically, natural language generation technology is used to adjust the tone of the message to match the user's psychological state.

[0586] Step 6:

[0587] The terminal displays a "countermeasure message" generated by the server to the user through a user interface. The input is the message sent from the server, and the output is the display information that the user can see on the screen. Specifically, the display device visually shows the user the countermeasures and encourages safe actions.

[0588] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0589] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0590] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0591] [Fourth Embodiment]

[0592] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0593] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0594] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0595] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0596] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0597] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0598] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0599] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0600] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0601] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0602] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0603] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0604] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0605] This invention is a system for preventing accidents within the home. This system involves the collaborative operation of user, terminal, and server elements.

[0606] First, the user takes pictures of areas of concern in their home using the device's camera. This is intended to identify accident-prone areas, such as scattered cables or dangerous corners, and sends the images. The user then transfers these images to the server in digital data format, following the instructions on the device's built-in user interface.

[0607] The terminal properly formats the image data received from the user and transmits it to the server using a secure communication protocol. This ensures the confidentiality and integrity of the data.

[0608] The server runs a generative model to analyze the received image data. Based on a pre-trained near-miss database, the generative model identifies potential hazards in the image and assesses the risk. This assessment process, for example, recognizes slippery floors or sharp objects in the image and determines their likelihood of causing an accident.

[0609] Once a risk is identified, the server uses its message generation unit to create a message about safety measures. This message proposes preventative measures for the specific risk identified. The server sends this information back to the terminal and also provides the user with visual alerts and informational displays.

[0610] As a concrete example, suppose a user sends an image taken in their living room to the server. This image shows unorganized cables next to the sofa. The server uses a generative model to identify the risk, considers measures to secure the cables to prevent tripping, and displays these suggestions to the user.

[0611] Thus, this system provides a practical and effective solution for dramatically reducing the risk of accidents in the home.

[0612] The following describes the processing flow.

[0613] Step 1:

[0614] The user takes a picture of an area of ​​interest in their home using the device's camera. After taking the picture, they select the image through the user interface and send the image data to the system.

[0615] Step 2:

[0616] The terminal receives image data sent by the user. Next, it converts this data to an appropriate digital format (e.g., JPEG), applies an encryption protocol to the data, and securely transmits it to the server.

[0617] Step 3:

[0618] The server decodes the image data received from the terminal and converts it into a format that can be processed. This prepares the data for analysis using a generative model.

[0619] Step 4:

[0620] The server runs a generative model and analyzes the received images. The model consults a near-miss database to identify potential hazards in the images. For example, it can recognize code that could cause tripping or sharp corners of furniture.

[0621] Step 5:

[0622] The server generates a message suggesting appropriate security measures based on the identified risks. This message includes specific countermeasures and aims to provide useful information to the user.

[0623] Step 6:

[0624] The server sends the generated risk assessment results and countermeasure messages to the terminal. Communication is conducted securely using an encrypted protocol.

[0625] Step 7:

[0626] The terminal receives messages from the server and presents them visually to the user through the user interface. The user can then use this information to take necessary safety measures within their home.

[0627] (Example 1)

[0628] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0629] Accidents within the home are a serious problem, especially in environments with elderly people or children. Current technology makes it difficult to efficiently and continuously identify risks within the home and propose appropriate countermeasures, resulting in insufficient improvements in safety. Therefore, there is a need for a system that can automatically detect potential risks within the home and propose rapid countermeasures.

[0630] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0631] In this invention, the server includes means for providing an interface for a user to transmit image data to a data processing device to identify risks within the home; means for using a data processing device that examines the received image data and executes a generation algorithm to identify hazards; means for providing a message generation unit that suggests preventive measures based on the identified hazards; and means for using an encrypted communication protocol to maintain the confidentiality and integrity of communications. This makes it possible to effectively and quickly reduce the risk of accidents within the home.

[0632] "Household risks" refer to situations or objects within the home that could potentially cause accidents.

[0633] A "data processing device" refers to a computer system that takes data as input and analyzes that data based on specific processing steps.

[0634] "Image data" refers to data containing visual information expressed in a digital format.

[0635] A "user interface" refers to a user interface that is operable by the user and enables interaction with devices and systems.

[0636] A "generative algorithm" refers to a set of computational procedures or methods used to automatically extract specific information from data.

[0637] "Identifying hazards" refers to identifying factors that could potentially cause accidents or damage in a given area or object.

[0638] A "message generation unit" refers to a device or program that has the function of automatically creating text or suggestions to be conveyed to the user based on specific information.

[0639] An "encrypted communication protocol" refers to a communication method used to protect data from being read by third parties when sending or receiving data.

[0640] "Confidentiality and integrity of communications" refers to the information security requirement that transmitted data is not leaked to others and is received without being altered.

[0641] This invention is a system aimed at preventing accidents in the home, in which the user, terminal, and server elements work together.

[0642] The user first takes photos of areas of concern in their home, such as scattered cables or sharp furniture corners, using the device's camera. The device is equipped with a camera that has sensor functions for generating digital data. The captured image data is temporarily stored on the device.

[0643] The device formats the saved images appropriately, encrypts them, and then sends them to the server. Secure communication protocols such as HTTPS are used for this purpose. The server functions as a data processing unit to receive this secure communication.

[0644] The server runs a generative AI model, which has been previously trained using a near-miss database, to analyze the received image data. This AI model automatically identifies potential hazards within the image and performs a risk assessment. For example, it might highlight areas where people are prone to slipping or tripping.

[0645] Once a risk is identified, the server's message generation unit is utilized to generate a message containing specific preventative measures for that particular risk. This information is sent back to the terminal and presented to the user with a visual alert.

[0646] For example, if a user takes a picture in their living room and it's sent from their device to the server, and that picture includes scattered cables next to the sofa, the server's generative model will analyze the risk. It will assess the risk of tripping and generate a message recommending that the cables be tidied up.

[0647] An example of a prompt might be, "Analyze the image of the living room and suggest safety measures." This approach allows the system to provide useful solutions that dramatically reduce the risk of accidents in the home.

[0648] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0649] Step 1:

[0650] The user takes a picture of an area of ​​concern in their home using the device's camera. The input is the image captured by the camera, and the output is image data saved in digital format. Specifically, the user might take a picture of cluttered cables or sharp furniture corners, and the image data is temporarily stored on the device.

[0651] Step 2:

[0652] The terminal formats the stored images into a secure format and prepares them for transmission to the server. The input is the stored image data, and the output is encrypted image data. Specifically, the terminal compresses the images into a format such as JPEG and encrypts the data using the HTTPS protocol.

[0653] Step 3:

[0654] The terminal sends encrypted image data to the server. The input is encrypted image data, and the output is the image data received by the server. Here, the terminal uses the HTTPS protocol to securely transmit the image data over the network.

[0655] Step 4:

[0656] The server inputs the received image data into a generating AI model to perform a risk assessment. The input is the received image data, and the output is the result of the risk assessment. Specifically, the server identifies hazardous elements within the image, such as slippery areas or sharp objects.

[0657] Step 5:

[0658] The server generates safety messages based on evaluation results from a generative AI model. The input is the result of the risk assessment, and the output is a message containing specific preventive measures. The server creates suggestions for the identified risks, such as a message like "Secure the cables."

[0659] Step 6:

[0660] The server sends the generated message to the terminal, which then presents it to the user. The input is the generated message, and the output is the visual alert presented to the user by the terminal. The terminal uses its user interface to display the message and prompt the user to take action to mitigate the risk.

[0661] (Application Example 1)

[0662] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0663] Maintaining safety within the home requires identifying potential hazards early and taking appropriate measures. However, conventional methods require users to recognize hazards and devise countermeasures themselves, which is time-consuming and laborious. This invention aims to solve these difficulties and provide an efficient method for automatically enhancing safety within the home.

[0664] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0665] In this invention, the server includes means having a user interface for transmitting captured image data to an information processing device in order to identify dangerous areas in the home; means using an information processing device that analyzes the received image data and executes a generative model to identify risks; means including a message generation unit that presents appropriate countermeasures based on the risk identification; means including a communication unit that proposes safety measures based on the identified risks; and means having an output unit that notifies a smart device of the proposed countermeasures. This reduces the need for the user to discover dangers themselves and makes it possible to efficiently implement safety measures in the home.

[0666] "Dangerous areas in the home" refers to parts of the living space where users are prone to accidents or problems.

[0667] An "information processing device" is an electronic device used to receive and analyze digital data.

[0668] "Image data" refers to visual information collected by cameras or other photographic devices, represented in digital format.

[0669] A "user interface" refers to the means or screens that allow a user to interact with a device, enabling information input and result display.

[0670] A "generative model" is an algorithm created based on pre-collected data, used to analyze and identify specific patterns or features.

[0671] The "message generation unit" refers to a function that automatically creates a message to notify the target person.

[0672] A "communication unit" is a technical element equipped with an interface for sending and receiving data to and from other devices or servers.

[0673] The "output unit" is the part of an information processing device that has the function of physically displaying or transmitting data or messages generated by the device.

[0674] To implement this invention, a system utilizing electronic equipment including an information processing device is configured. This system includes various elements for identifying potential hazards within the home and proposing safety measures.

[0675] First, the user takes a picture of an area of ​​interest in their home using a smartphone or smart glasses. The captured image data is transmitted to the information processing device via the terminal's user interface. The terminal has a built-in camera module and communication module, enabling smooth data input and transmission.

[0676] The server is responsible for processing the received image data. Specifically, the server runs generative AI models such as TensorFlow or PyTorch to identify risks from the received visual information. For example, it might detect scattered cables on the floor or sharp-edged furniture. The generative models used here are trained on datasets used to assess risks in advance.

[0677] Once a risk is identified, the server activates its message generation unit to generate a message outlining appropriate preventative measures. This message is sent from the server to the terminal and displayed on the user's smart device. The display incorporates visual alerts to ensure the user's attention is immediately drawn to the message. This information, provided by the output unit, serves as a guide for taking physical countermeasures.

[0678] For example, when a user takes a picture of their living room, image analysis might detect unorganized cables next to the sofa. The server then uses this result to generate a message such as, "Please use clips to secure the cables to prevent tripping," and sends this message to the user's device. Additionally, prompts such as, "Please take a picture of your front door and send it. We will diagnose potential hazards," can be displayed.

[0679] This system provides users with a practical means to easily maintain safety within their homes and can significantly reduce the risk of accidents.

[0680] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0681] Step 1:

[0682] Users use their smartphone cameras to photograph areas of interest within their homes. This input data is saved on the device as an image file.

[0683] Step 2:

[0684] The terminal selects images captured via a user interface and sends the image data to the server using a secure communication protocol (e.g., TLS). The input is image data, and the output is data to be sent to the server.

[0685] Step 3:

[0686] The server retrieves the received image data, uses a generative AI model to analyze the potential risks within the images, and generates risk assessment information. The input is the transmitted image data, and the output is the risk assessment result. TensorFlow or PyTorch are used for this analysis.

[0687] Step 4:

[0688] Based on the risk assessment, the server uses a message generation unit to create a message that warns the user. This message includes specific countermeasures for the identified hazards. The input is risk assessment information, and the output is a countermeasure message for the user.

[0689] Step 5:

[0690] The server sends the generated message to the terminal via the communication unit. The terminal displays the received message using its output unit, providing the user with visual feedback. The message may include specific suggestions, such as "Use clips to secure the cable to prevent tripping." The input is message data from the server, and the output is the information displayed on the terminal's screen.

[0691] Step 6:

[0692] Based on the safety messages displayed on the device, users take actual safety measures at specific locations within their homes. This action reduces the risk of accidents within the home.

[0693] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0694] This invention provides a system that enables accident prevention within the home and interaction based on user emotions. The system consists of a user, a terminal, a server, and an emotion engine.

[0695] The user uses the device's camera to photograph a specific area within their home and transmits the image data. The device receives this image data through its user interface, converts it to a secure digital format, and then transfers it to the server.

[0696] The server analyzes the received image data using a generative model to identify potential hazards. This process involves referencing a near-miss database to determine hazards such as sharp edges on furniture or scattered objects. It also utilizes an emotion engine to analyze emotional data received from the user. This data is used to assess stress levels and emotional states based on the user's facial expressions and voice.

[0697] Based on these analysis results, the server generates appropriate countermeasures according to the characteristics of the risk and the user's emotions. These countermeasures are customized by the message generation unit and sent back to the terminal.

[0698] The terminal displays messages received from the server via a user interface. The tone and approach of the messages are adjusted to take into account the user's emotional state. For example, if the user is stressed, the system prioritizes suggesting simple, immediate solutions.

[0699] For example, if a user sends a photo of scattered toys in their living room, the server uses a generative model to identify the risks and an emotion engine to assess the user's anxiety level. The server then sends back suggested solutions to the device that address the risks and take the user's feelings into consideration. For instance, the user might receive a tailored message such as, "You'll feel more at ease if you tidy up the toys this way."

[0700] In this way, this system helps prevent accidents in the home and provides flexible interaction functions that respond to the user's emotions, thereby encouraging the implementation of effective safety measures.

[0701] The following describes the processing flow.

[0702] Step 1:

[0703] The user takes photos of areas of interest in their home using a smartphone or tablet. The user grants permission to provide the system with facial expressions and voice recordings to document their emotional state. After taking the photos, the user sends the image data to the server via the user interface.

[0704] Step 2:

[0705] The device encodes the image and sentiment data sent by the user into the appropriate format. The device then uses a secure communication protocol to send the data to the server, ensuring data confidentiality.

[0706] Step 3:

[0707] The server analyzes image data and emotion data received from the terminal. Using a generative model, the server identifies risks within the images and uses an emotion engine to analyze the user's emotional state. Specifically, it evaluates the stress level and anxiety level indicated by the user's emotions.

[0708] Step 4:

[0709] The server generates individualized countermeasures based on identified risk information and the user's emotional state. The message generation unit creates messages that include a tone that takes emotional data into account and specific countermeasures. For example, if the user is feeling stressed, it prioritizes presenting simple countermeasures and risk reduction methods.

[0710] Step 5:

[0711] The server sends the generated message to the terminal using an encryption protocol.

[0712] Step 6:

[0713] The terminal displays messages received from the server in its user interface. The user reviews the displayed messages and understands the security measures that should be taken based on them. The terminal interface is configured to provide guidance in a gentle tone that is sensitive to the user's feelings.

[0714] Step 7:

[0715] Users implement the suggested measures to improve safety within their homes. This process, through empathetic follow-up, promotes sustainable safety maintenance behaviors.

[0716] (Example 2)

[0717] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0718] The challenge lies in preventing accidents within the home and realizing two-way information communication that takes into account the user's emotional state. In particular, it is necessary to quickly identify potential hazards that could cause unexpected accidents and to provide appropriate countermeasures that take into account the user's emotional response. Conventional systems have often failed to integrate physical risk assessment and emotional state analysis, treating them as separate functions. To solve this problem, advanced technology is needed that simultaneously analyzes risk factors and considers emotions.

[0719] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0720] In this invention, the server includes a person who transmits data via a terminal with video recording capabilities to acquire data on the home environment; a means; a function and means that uses a computing device to process the received environmental data and execute a generation algorithm for identifying potential risks; and a means that includes an information generation unit that presents countermeasures tailored based on potential risks and the user's emotional assessment. This makes it possible to identify potential hazards in the user's environment and automatically provide safety measures and emotionally sensitive feedback.

[0721] "The home environment" refers to the physical space within a house, the objects within it, their arrangement, and the overall circumstances associated with them.

[0722] "Video recording function" refers to technology that can capture visual information as an image or video, and then save or transmit it.

[0723] A "terminal" refers to an electronic device used for acquiring, processing, transmitting, and receiving data.

[0724] "Potential risks" refer to factors that are not yet apparent but, if their existence were confirmed, could potentially cause accidents or dangers.

[0725] A "generative algorithm" refers to a set of computational procedures performed on a computer to analyze data and obtain specific judgments or results.

[0726] A "computational device" refers to a system consisting of hardware and software for processing and analyzing data.

[0727] "Emotional evaluation" refers to the process of analyzing and understanding a user's psychological state using numerical values ​​or categories.

[0728] The "information generation unit" refers to a component that has the function of generating information and countermeasures to be presented to the user based on the analysis results.

[0729] The system of this invention provides accident prevention within the home and information communication that responds to the user's emotions. The system mainly consists of a terminal, a server, a generative AI model, and an emotion engine.

[0730] The user uses a device equipped with video recording capabilities to film specific areas within their home. The device features a high-resolution camera that captures the environment. The captured data is first converted into a secure digital format via a user interface. This conversion process utilizes data compression and format conversion technologies. The converted data is then transmitted to a server using secure communication methods. Data protection technologies, such as the SSL / TLS protocol, are applied during transmission.

[0731] The server analyzes the received data. A generative AI model is used for the analysis. This model is based on machine learning algorithms and identifies potential risk factors by cross-referencing it with a near-miss database. The server also evaluates facial expressions and voice data received from the user using an emotion engine. The emotion engine is software that quantifies the user's psychological state and analyzes stress and anxiety levels.

[0732] Using this data, the server generates countermeasures through its information generation unit. This process, based on a generation AI model, automatically generates countermeasures that are customized to take the user's emotional state into consideration. For example, a possible prompt might be, "I sent a photo of my living room. Please provide information on any security issues and any related advice."

[0733] The countermeasures generated by the system are sent back to the terminal and presented to the user via a user interface. This interface is designed to provide feedback through both sight and sound, allowing the user to intuitively understand and act upon the information. This enables the effective identification and elimination of potential hazards within the home, while simultaneously providing emotionally sensitive interactions.

[0734] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0735] Step 1:

[0736] The user uses the device's camera to photograph a specific area within their home. As input, the user provides the device with image data of the physical environment. This image data is converted directly into a digital format within the device. This conversion involves data compression and format conversion, resulting in an image format suitable for analysis (e.g., JPEG to binary data).

[0737] Step 2:

[0738] The terminal sends converted digital image data to the server. As input, the terminal has compressed and converted digital image data. This data is securely transferred to the server using communication methods such as the SSL / TLS protocol. The output is the data received by the server.

[0739] Step 3:

[0740] The server analyzes the received image data. The input is digital image data sent from the terminal. The server applies a generative AI model and compares it with a near-miss database to identify potential risk factors. The output is a list of identified risks. Here, the AI ​​model identifies, for example, sharp-angled furniture or scattered objects.

[0741] Step 4:

[0742] The server receives and analyzes emotional data from the user. The input consists of the user's facial expressions and voice data. Using an emotion engine, the server evaluates the user's emotional state based on this data. The output is a numerical evaluation of the user's emotional state. The emotional state is expressed as levels of stress and anxiety.

[0743] Step 5:

[0744] The server generates countermeasures based on the identified risks and emotional state assessments. The inputs are a list of risks and emotional state evaluation values ​​obtained from steps 3 and 4. An AI generation model is used to generate appropriate countermeasures for the risks, which are then customized in the message generation unit according to the user's emotional state. The output is the adjusted countermeasure message.

[0745] Step 6:

[0746] The terminal receives a response message sent from the server and displays it to the user via a user interface. The input is a pre-configured message from the server. The terminal displays this message visually and audibly in a human-readable format. As output, the user receives the displayed message and can take the necessary actions.

[0747] (Application Example 2)

[0748] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0749] The problem that this invention aims to solve is to improve safety in the work environment and to provide appropriate feedback functions to reduce operator stress and anxiety. Conventional systems have difficulty identifying potential hazards in real time and suggesting countermeasures that are appropriate to the operator's emotional state. As a result, the safety of the work environment is not sufficiently ensured, and the anxiety and stress experienced by operators are not alleviated.

[0750] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0751] In this invention, the server includes means for receiving and analyzing environmental data of the workspace from a terminal, means for identifying potential risks using a generative AI model, and means for determining the operator's emotional state using an emotion analysis engine and adjusting countermeasure messages accordingly. This not only improves the safety of the workspace but also provides flexible feedback that takes the operator's feelings into consideration, making it possible to create an environment where the operator can work with peace of mind.

[0752] "Environmental data" refers to images and video information taken to capture conditions within a home or workspace.

[0753] An "information processing device" is a combination of hardware and software used to analyze and process received data.

[0754] A "user interface" is a component of a system that provides functions for users to input or output information.

[0755] A "generative AI model" is an algorithm that utilizes artificial intelligence to extract and analyze necessary information from received data.

[0756] "Potential risks" refer to factors or situations that, even if not currently apparent, could potentially cause accidents or dangers in the future.

[0757] An "emotion analysis engine" is a type of software that analyzes a user's facial expressions and voice to evaluate their emotional state.

[0758] "Emotional state" refers to data that is evaluated as indicating the user's mental or psychological response or situation.

[0759] A "countermeasure message" is a written document containing suggestions and instructions provided to the user based on identified risks.

[0760] This invention is a system that uses an "information processing device" for collecting "environmental data" and a "generative AI model" for analyzing the collected data to identify potential hazards and generate appropriate "countermeasure messages." The system consists of the following hardware and software.

[0761] The server receives "environmental data" sent from the terminal and analyzes the images and videos captured by the "generative AI model." This model utilizes AI algorithms (e.g., frameworks such as TensorFlow and PyTorch) to identify "potential risks" in the home or workspace.

[0762] The device provides the user with the aforementioned "countermeasure message" through its "user interface." This allows the user to visually understand the situation and take action to ensure their safety by following the instructions.

[0763] Furthermore, the server uses an "emotion analysis engine" to analyze the operator's voice tone and facial expressions, and evaluates their "emotional state." This result is used to customize the content and tone of the messages presented to match the user's psychological state.

[0764] For example, if sharp tools are scattered around the workspace, the "generative AI model" identifies the situation as dangerous, and the "sentiment analysis engine" detects the operator's stress. The system can then display a message such as, "Please move these tools to a safe location. This will make the work environment safer."

[0765] An example of a prompt for a generating AI model is: "Evaluate the safety of the workspace based on image data, analyze the operator's tone of voice, and suggest safety measures if they appear stressed."

[0766] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0767] Step 1:

[0768] Users capture images of their home or workspace using a camera-equipped device, generating "environmental data." The input is image data, which is converted to a secure digital format as output. The device then transmits this data to a server using an encryption protocol.

[0769] Step 2:

[0770] The server receives image data sent from the terminal. The input is encrypted image data, and the output is decrypted image data. The server converts the received data into an analyzable format by decrypting the end-to-end encryption.

[0771] Step 3:

[0772] The server analyzes the decoded image data using a "generative AI model" to identify "potential risks." The input is the decoded image data, and the output is the identified risk information. Specifically, the AI ​​model detects and labels hazardous elements within the image.

[0773] Step 4:

[0774] The server receives voice or facial expression data from the operator and uses an "emotion analysis engine" to evaluate their "emotional state." The input is voice or video data, and the output is the evaluation result of the emotional state. Specifically, it uses voice tone analysis and facial expression recognition technology to quantify stress and anxiety levels.

[0775] Step 5:

[0776] The server generates a "countermeasure message" based on identified risk information and emotional state assessment results. The input is risk information and emotional assessment data, and the output is a customized message. Specifically, natural language generation technology is used to adjust the tone of the message to match the user's psychological state.

[0777] Step 6:

[0778] The terminal displays a "countermeasure message" generated by the server to the user through a user interface. The input is the message sent from the server, and the output is the display information that the user can see on the screen. Specifically, the display device visually shows the user the countermeasures and encourages safe actions.

[0779] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0780] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0781] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0782] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0783] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. In the upper and lower directions of the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Also, the upper side of the concentric circles is where "pleasant" emotions are located, and the lower side is where "unpleasant" emotions are located. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0784] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0785] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0786] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0787] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0788] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0789] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0790] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0791] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0792] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0793] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0794] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0795] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0796] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0797] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0798] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0799] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0800] The following is further disclosed regarding the embodiments described above.

[0801] (Claim 1)

[0802] A means comprising a user interface for transmitting captured image data to an information processing device in order to identify dangerous areas within the home,

[0803] A means of using an information processing device that analyzes received image data and executes a generative model to identify risks,

[0804] A means including a message generation unit that presents appropriate countermeasures based on risk identification,

[0805] A system that includes this.

[0806] (Claim 2)

[0807] The system according to claim 1, comprising a display unit that displays received risk information on a user interface, and providing visual feedback to the user.

[0808] (Claim 3)

[0809] The system according to claim 1, which applies an encryption protocol for securely communicating transmitted image data.

[0810] "Example 1"

[0811] (Claim 1)

[0812] A means for providing an interface for users to transmit image data to a data processing device in order to identify risks within the home,

[0813] A means of using a data processing device that examines received image data and executes a generation algorithm to identify danger,

[0814] A means comprising a message generation unit that presents preventive measures based on identified risks,

[0815] In order to maintain the confidentiality and integrity of communications, means of using encrypted communication protocols,

[0816] A system that includes this.

[0817] (Claim 2)

[0818] The system according to claim 1, comprising a display unit that presents received danger information on an interface for the user, and providing a visual response to the user.

[0819] (Claim 3)

[0820] The system according to claim 1, which applies a generation algorithm to transmitted image data and evaluates potential risks based on a pre-stored database.

[0821] "Application Example 1"

[0822] (Claim 1)

[0823] A means comprising a user interface for transmitting captured image data to an information processing device in order to identify dangerous areas within the home,

[0824] A means of using an information processing device that analyzes received image data and executes a generative model to identify risks,

[0825] A means including a message generation unit that presents appropriate countermeasures based on risk identification,

[0826] A means equipped with a communication unit that proposes safety measures based on identified risks,

[0827] A means having an output unit that notifies a smart device of the proposed countermeasures,

[0828] A system that includes this.

[0829] (Claim 2)

[0830] The system according to claim 1, comprising a device that displays received risk information on a user interface and provides visual feedback to the user.

[0831] (Claim 3)

[0832] The system according to claim 1, which applies encrypted communication means to securely transmit transmitted image data, thereby enabling protected data transfer.

[0833] "Example 2 of combining an emotion engine"

[0834] (Claim 1)

[0835] To acquire data on the home environment, a person transmits the data via a device with video recording capabilities,

[0836] The system includes a computing device that processes received environmental data and executes a generation algorithm to identify potential risks, and

[0837] A means including an information generation unit that presents countermeasures adjusted based on potential risks and user sentiment evaluations,

[0838] A system that includes this.

[0839] (Claim 2)

[0840] The system according to claim 1, which presents received risk assessment information and sentiment analysis results to a human through a display function and provides visual and auditory feedback.

[0841] (Claim 3)

[0842] The system according to claim 1, which applies a data protection protocol to protect transmitted environmental data.

[0843] "Application example 2 when combining with an emotional engine"

[0844] (Claim 1)

[0845] A means comprising a user interface for transmitting captured environmental data to an information processing device in order to identify hazardous elements in a home or workspace,

[0846] A means of using an information processing device that analyzes received environmental data and runs a generative AI model to identify potential risks,

[0847] A means of generating messages that suggest appropriate countermeasures based on risk identification, and adjusting them using an emotion analysis engine that takes into account the user's emotional state,

[0848] Means of displaying and notifying via a user interface,

[0849] A system that includes this.

[0850] (Claim 2)

[0851] The system according to claim 1, which displays adjusted messages based on received risk information and emotional state on the user interface, and provides the user with visual and emotionally adaptive feedback.

[0852] (Claim 3)

[0853] The system according to claim 1, which applies an encryption protocol to securely communicate transmitted image and emotion data, thereby protecting the confidentiality and integrity of the data. [Explanation of Symbols]

[0854] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means comprising a user interface for transmitting captured image data to an information processing device in order to identify dangerous areas within the home, A means of using an information processing device that analyzes received image data and executes a generative model to identify risks, A means including a message generation unit that presents appropriate countermeasures based on risk identification, A system that includes this.

2. The system according to claim 1, comprising a display unit that displays received risk information on a user interface, thereby providing the user with visual feedback.

3. The system according to claim 1, which applies an encryption protocol for securely communicating transmitted image data.

Images