system

Abstract

The system according to this embodiment aims to understand the individual learning and behavioral status and provide an optimal learning plan and behavioral modification program based on that understanding. [Solution] The system according to the embodiment comprises a shooting unit, an analysis unit, a generation unit, and a provision unit. The shooting unit captures the subject's daily activities in photographs and videos. The analysis unit analyzes the information captured by the shooting unit to understand the subject's learning and behavioral status. The generation unit generates an optimal learning plan and behavioral modification program based on the situation understood by the analysis unit. The provision unit provides the program generated by the generation unit.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, there is a problem that individual learning and behavior situations are not sufficiently grasped, and an optimal plan or program is not provided based on them.

[0005] The system according to the embodiment aims to grasp individual learning and behavior situations and provide an optimal learning plan or behavior modification program based on them.

Means for Solving the Problems

[0006] The system according to this embodiment comprises a shooting unit, an analysis unit, a generation unit, and a provision unit. The shooting unit captures the subject's daily activities in photographs or videos. The analysis unit analyzes the information captured by the shooting unit to understand the subject's learning and behavioral status. The generation unit generates an optimal learning plan and behavioral modification program based on the situation understood by the analysis unit. The provision unit provides the program generated by the generation unit. [Effects of the Invention]

[0007] The system according to this embodiment can grasp the individual learning and behavioral status and provide an optimal learning plan and behavioral modification program based on that. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F manages communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 comprises a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The receiving device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The learning and behavioral status monitoring system according to an embodiment of the present invention is a system that uses a generative AI to understand the individual learning and behavioral status and provides an optimal learning plan and behavioral modification program. This system is applicable not only to humans but also to pets and zoo animals. The learning and behavioral status monitoring system works by having the user take photos or videos of the target human or animal's daily behavior. For example, the user can take photos of a pet's behavior and learning status with a smartphone. This information is input into the generative AI. Next, the generative AI analyzes the input photos and videos to understand the target's learning and behavioral status. The generative AI uses image recognition technology to analyze the target's behavior, facial expressions, physical condition, etc. For example, it can determine whether the pet is playing energetically or appears unwell. Based on the results of the generative AI's analysis, it generates an optimal learning plan and behavioral modification program. For example, if the pet is unwell, it provides information on how to deal with the situation and information on veterinary hospitals. If the pet is playing energetically, it suggests new ways to play that will make it even more enjoyable. The generated learning plan and behavioral modification program are provided to the user. The user can modify the target's learning and behavior based on the information provided by the generative AI. For example, if a pet is unwell, the system can implement the suggested solutions provided by the AI ​​and take the pet to a veterinary hospital. This system allows users to understand the subject's learning and behavioral status and take the most appropriate action. For instance, even if a pet is unwell, users can quickly learn how to deal with the situation and provide appropriate care. Furthermore, even if a pet is playing happily, the system can suggest new ways to play, making the experience even more enjoyable. This system is applicable to a variety of subjects, including humans, pets, and zoo animals. For example, by filming the behavior of zoo animals and having the AI ​​analyze the footage, the system can understand the animal's health and stress levels, enabling the provision of optimal care. It can also support efficient learning by understanding a person's learning progress and providing an optimal learning plan. Thus, the learning and behavioral status monitoring system can automatically understand the subject's learning and behavioral status and provide optimal learning plans and behavioral modification programs.

[0029] The learning and behavioral status monitoring system according to this embodiment comprises a shooting unit, an analysis unit, a generation unit, and a provision unit. The shooting unit captures the subject's daily behavior in photographs and videos. The shooting unit can, for example, use a smartphone or digital camera to capture the subject's behavior. The shooting unit can also perform long-term shooting using a surveillance camera. For example, the shooting unit can photograph a pet's behavior with a smartphone and upload the data to the cloud. Furthermore, the shooting unit can also photograph the behavior of animals in a zoo with a surveillance camera and collect data in real time. The analysis unit analyzes the information captured by the shooting unit to understand the subject's learning and behavioral status. The analysis unit uses generation AI and image recognition technology to analyze the subject's behavior, expressions, and physical condition. For example, the analysis unit can analyze a pet's behavior to determine if it is playing energetically or appears unwell. The analysis unit can also analyze the behavior of animals in a zoo to understand their health status and stress levels. Furthermore, the analysis unit can analyze a person's learning situation to understand their learning progress and level of comprehension. The generation unit generates an optimal learning plan and behavior modification program based on the situation grasped by the analysis unit. The generation unit uses a generation AI to generate a program that is appropriate to the target situation. For example, if a pet is unwell, the generation unit generates information on how to deal with the situation and information on veterinary hospitals. The generation unit can also generate a program that suggests new ways to play if the pet is playing happily. Furthermore, the generation unit can also generate an optimal learning plan according to the human's learning progress. The provision unit provides the program generated by the generation unit to the user. The provision unit can provide the generated program to the user, for example, through a smartphone app or a web app. The provision unit can also notify the user of the generated program using email or a messaging app. Furthermore, the provision unit can also provide the generated program as printed material. As a result, the learning and behavior situation awareness system according to this embodiment can capture and analyze the target's daily behavior and provide an optimal learning plan and behavior modification program.

[0030] The photography department captures the subject's daily activities in photos and videos. For example, the photography department can use smartphones or digital cameras to film the subject's behavior. Specifically, high-resolution smartphone cameras can clearly capture the subject's subtle movements and expressions. Digital cameras, utilizing zoom and continuous shooting functions, can capture detailed footage even from long distances. Furthermore, the photography department can use surveillance cameras for long-term recording. Multiple surveillance cameras are installed to cover a wide area, recording the subject's behavior 24 hours a day. For example, the photography department can film a pet's behavior with a smartphone and upload the data to the cloud. The data uploaded to the cloud is managed so that the analysis and generation departments can access it. Additionally, the photography department can use surveillance cameras to film the behavior of animals in zoos and collect data in real time. Surveillance cameras allow for long-term observation without disrupting the animals' natural behavior. This allows the photography department to use a variety of devices to record the subject's behavior in detail and continuously, improving the overall data collection capability of the system.

[0031] The analysis unit analyzes information captured by the shooting unit to understand the subject's learning and behavioral status. Using generative AI and advanced image recognition technology, the analysis unit analyzes the subject's behavior, facial expressions, and physical condition. Specifically, the generative AI uses deep learning algorithms to automatically identify the subject's movements and facial expressions from image and video data. For example, the analysis unit analyzes a pet's behavior to determine if it is playing energetically or appears unwell. The generative AI learns the pet's movement patterns and facial expression changes to detect abnormal behavior or changes in physical condition. The analysis unit can also analyze the behavior of animals in zoos to understand their health and stress levels. Based on animal behavioral data, the generative AI predicts signs of stress and changes in health. Furthermore, the analysis unit can analyze human learning to understand learning progress and comprehension. The generative AI analyzes the learner's facial expressions and behavior to evaluate concentration and comprehension. This allows the analysis unit to quickly and accurately analyze collected data and understand the subject's learning and behavioral status in real time. Furthermore, the analysis unit can utilize historical data and statistical information to analyze long-term behavioral patterns and learning trends. This allows the analysis unit to comprehensively evaluate the target behavior and learning status, improving the overall accuracy and reliability of the system.

[0032] The generation unit generates optimal learning plans and behavioral modification programs based on the situation grasped by the analysis unit. The generation unit uses a generation AI to generate programs tailored to the target situation. Specifically, the generation AI automatically designs the optimal program for the target's needs and situation based on the analysis results. For example, if a pet is unwell, the generation unit generates information on how to deal with the situation and information on veterinary hospitals. The generation AI analyzes the pet's symptoms and behavioral data to provide appropriate treatment methods and information on the nearest veterinary hospital. The generation unit can also generate a program that suggests new ways to play if the pet is playing happily. The generation AI learns the pet's preferences and behavioral patterns and suggests new ways to play and training methods that will pique their interest. Furthermore, the generation unit can also generate an optimal learning plan according to the human's learning progress. The generation AI creates an individually customized learning plan based on the learner's progress and understanding, supporting effective learning. This allows the generation unit to quickly and accurately generate the optimal program tailored to the target situation, maximizing the overall system's effectiveness. In addition, the generation unit continuously evaluates the effectiveness of the generated programs and can modify and improve them as needed. This allows the generation unit to consistently provide high-quality programs based on the latest information and technology, supporting the improvement of the target's learning and behavioral status.

[0033] The service provider delivers the programs generated by the generation unit to the user. The service provider can deliver the generated programs to users, for example, through smartphone apps or web apps. Specifically, a smartphone app provides an easily accessible interface for users, visually displaying the generated programs. A web app provides a platform accessible from anywhere via the internet, allowing users to view and run the generated programs. Furthermore, the service provider can also notify users of the generated programs using email or messaging apps. For example, important notices and urgent instructions can be quickly communicated to users via email or messaging apps. The service provider can also provide the generated programs as printed materials. For example, learning plans and behavioral modification programs can be printed and distributed to users, making them available offline. This allows the service provider to deliver generated programs to users in diverse ways, improving convenience and accessibility. Furthermore, the service provider can collect user feedback and use it to improve delivery methods and program content. This enables the service provider to respond flexibly to user needs, improving overall system satisfaction and effectiveness.

[0034] The analysis unit can analyze the subject's behavior, facial expressions, and physical condition using image recognition technology. For example, the analysis unit can analyze the subject's facial expressions using face recognition technology and estimate emotions. The analysis unit can also analyze the subject's behavior using motion recognition technology and understand behavioral patterns. Furthermore, the analysis unit can analyze the subject's physical condition using physical condition analysis technology and understand its health status. For example, the analysis unit can analyze the subject's facial expressions using face recognition technology and estimate emotions such as smiles or anger. Motion recognition technology analyzes the subject's movements and understands behavioral patterns such as walking or running. Physical condition analysis technology analyzes data such as the subject's body temperature and heart rate and evaluates its health status. In this way, by using image recognition technology, the subject's behavior, facial expressions, and physical condition can be accurately analyzed. Some or all of the above-described processes in the analysis unit may be performed using, for example, a generative AI, or without a generative AI. For example, the analysis unit can input data on the subject's behavior, facial expressions, and physical condition into a generative AI, which can then perform the analysis.

[0035] The generation unit can generate information on how to treat a pet and information on veterinary hospitals when the pet is unwell. For example, if the pet is unwell, the generation unit can generate first aid methods. The generation unit can also generate information such as the location, hours of operation, and specialty of veterinary hospitals. Furthermore, the generation unit can generate methods for administering medication according to the pet's condition. For example, if the pet is unwell, the generation unit can suggest methods of cooling or warming as first aid. The veterinary hospital information provides information on specialists appropriate to the pet's symptoms. The medication administration methods suggest appropriate dosages based on the pet's weight and age. This allows for the provision of appropriate treatment methods and information on veterinary hospitals when a pet is unwell. Some or all of the above processing in the generation unit may be performed using, for example, a generation AI, or without using a generation AI. For example, the generation unit can input pet health data into a generation AI, and the generation AI can generate treatment methods and information on veterinary hospitals.

[0036] The generation unit can generate a program that suggests new ways to play when the pet is playing energetically. For example, the generation unit can suggest new ways to play that the pet can enjoy. The generation unit can also suggest ways to play that increase the pet's exercise level. Furthermore, the generation unit can suggest ways to play that promote the pet's intellectual development. For example, the generation unit can suggest ball games or hide-and-seek as new ways to play that the pet can enjoy. For ways to increase exercise, it can suggest playing in a large space where the pet can run around. For ways to promote intellectual development, it can suggest puzzles and tricks that cultivate the pet's thinking skills. In this way, by suggesting new ways to play when the pet is playing energetically, it can provide even more enjoyable time. Some or all of the above processing in the generation unit may be performed using a generation AI, for example, or without a generation AI. For example, the generation unit can input pet behavior data into a generation AI, and the generation AI can generate a program that suggests new ways to play.

[0037] The service provider can provide users with generated learning plans and behavioral modification programs. The service provider can provide the generated programs to users, for example, through a smartphone app or a web app. The service provider can also notify users of the generated programs via email or messaging apps. Furthermore, the service provider can provide the generated programs as printed materials. For example, the service provider can display the generated learning plan to the user through a smartphone app. A web app is a platform accessible to users via the internet and provides the generated programs. Email and messaging apps send notifications directly to users and provide the generated programs. Printed materials are provided in a format that allows users to receive the programs on paper. This allows users to take appropriate action by providing them with generated learning plans and behavioral modification programs. Some or all of the above processing in the service provider may be performed using, for example, a generation AI, or without a generation AI. For example, the service provider can input the generated programs into a generation AI, and the generation AI can determine how to provide them to the user.

[0038] The analysis unit can analyze the behavior of zoo animals to understand their health status and stress levels. For example, the analysis unit can analyze the animals' behavior patterns to assess their health status. It can also analyze the animals' facial expressions to estimate their stress levels. Furthermore, the analysis unit can analyze the animals' physical condition data to understand their health status. For example, the analysis unit can analyze the animals' behavior patterns to detect abnormal behavior that differs from normal behavior. Facial expression analysis identifies facial expressions that indicate stress and estimates the stress level. Physical condition data analysis analyzes biometric data such as the animals' body temperature and heart rate to assess their health status. This allows for the provision of appropriate care by understanding the health status and stress levels of zoo animals. Some or all of the above-described processes in the analysis unit may be performed using, for example, a generative AI, or not using a generative AI. For example, the analysis unit can input animal behavior data into a generative AI, which can then analyze the health status and stress levels.

[0039] The camera unit can predict the subject's movements and behavior patterns during shooting and automatically select the optimal shooting angle. For example, if the subject is an animal, the camera unit automatically adjusts the camera angle based on the animal's predicted movements. If the subject is a human, the camera unit can also analyze their behavior patterns and select the optimal shooting position. Furthermore, if the subject is a pet, the camera unit can predict the pet's play movements and shoot accordingly. For example, the camera unit predicts animal movements and automatically adjusts the camera angle. It analyzes human behavior patterns and selects the optimal shooting position. It predicts pet play movements and shoots accordingly. By predicting the subject's movements and behavior patterns and selecting the optimal shooting angle, more effective shooting becomes possible. Some or all of the above processing in the camera unit may be performed using, for example, a generative AI, or without a generative AI. For example, the camera unit can input data on the subject's movements and behavior patterns into a generative AI, which can then select the optimal shooting angle.

[0040] The camera unit can automatically adjust the optimal shooting settings during shooting, taking into account the ambient light and background of the subject. For example, if the ambient light is strong, the camera unit can automatically adjust the exposure to capture the image at the optimal brightness. Furthermore, if the background is complex, the camera unit can blur the background to emphasize the subject. Additionally, if the ambient light is weak, the camera unit can automatically adjust the ISO sensitivity to capture a brighter image. For example, if the ambient light is strong, the camera unit can automatically adjust the exposure to capture the image at the optimal brightness. If the background is complex, it can blur the background to emphasize the subject. If the ambient light is weak, it can automatically adjust the ISO sensitivity to capture a brighter image. This allows for higher quality photography by adjusting the optimal shooting settings considering the ambient light and background of the subject. Some or all of the above processing in the camera unit may be performed using, for example, a generative AI, or without a generative AI. For example, the camera unit can input ambient light and background data into a generative AI, which can then adjust the optimal shooting settings.

[0041] The recording unit can simultaneously record the subject's voice and sound environment during recording and use this information for analysis. For example, if the subject is an animal, the recording unit can simultaneously record the animal's vocalizations and use this information for behavioral analysis. If the subject is a human, the recording unit can also record the content of conversations and use this information for analyzing learning progress. Furthermore, if the subject is a pet, the recording unit can record the sounds of play and use this information for analyzing behavioral patterns. For example, the recording unit can simultaneously record animal vocalizations and use this information for behavioral analysis. It can record the content of human conversations and use this information for analyzing learning progress. It can record the sounds of pets playing and use this information for analyzing behavioral patterns. By simultaneously recording the subject's voice and sound environment, more detailed analysis becomes possible. Some or all of the above processing in the recording unit may be performed using, for example, a generative AI, or it may be performed without a generative AI. For example, the recording unit can input the subject's voice data into a generative AI, which can then analyze the voice data.

[0042] The camera unit can acquire location information of the subject during shooting and use it for analysis. For example, if the subject is an animal, the camera unit can record the animal's movement path and use it to analyze its range of activity. If the subject is a human, the camera unit can also record location information of learning locations and use it to analyze the learning environment. Furthermore, if the subject is a pet, the camera unit can record play locations and use it to analyze behavioral patterns. For example, the camera unit can record the movement path of an animal and use it to analyze its range of activity. It can record location information of a human's learning locations and use it to analyze the learning environment. It can record play locations of a pet and use it to analyze behavioral patterns. This makes it possible to analyze the range of activity and movement patterns by acquiring the subject's location information. Some or all of the above processing in the camera unit may be performed using, for example, a generative AI, or it may be performed without a generative AI. For example, the camera unit can input the subject's location information data into a generative AI, and the generative AI can analyze the location information.

[0043] The analysis unit can improve the accuracy of its analysis by referring to the subject's past behavioral data during the analysis. For example, if the subject is an animal, the analysis unit can analyze the current behavior by referring to past behavioral patterns. If the subject is a human, the analysis unit can also analyze the current learning status by referring to past learning history. Furthermore, if the subject is a pet, the analysis unit can analyze the current behavior by referring to past play data. For example, the analysis unit can refer to the animal's past behavioral patterns to analyze the current behavior. It can refer to the human's past learning history to analyze the current learning status. It can refer to the pet's past play data to analyze the current behavior. By referring to the subject's past behavioral data, the accuracy of the analysis is improved. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or it may be performed without using a generative AI. For example, the analysis unit can input the subject's past behavioral data into a generative AI, which can then improve the accuracy of the analysis.

[0044] The analysis unit can monitor the target's behavioral patterns in real time during analysis and detect anomalies. For example, if the target is an animal, the analysis unit can detect movements that deviate from normal behavioral patterns in real time. Furthermore, if the target is a human, the analysis unit can also detect abnormal behavior during learning in real time. Additionally, if the target is a pet, the analysis unit can detect behavior that deviates from normal play patterns in real time. For example, the analysis unit can detect movements that deviate from the normal behavioral patterns of animals in real time. It can detect abnormal behavior during learning in humans in real time. It can detect behavior that deviates from normal play patterns of pets in real time. This allows for rapid detection of anomalies by monitoring the target's behavioral patterns in real time. Some or all of the above-described processes in the analysis unit may be performed using, for example, a generative AI, or without a generative AI. For example, the analysis unit can input the target's behavioral pattern data into a generative AI, which can then detect anomalies.

[0045] The analysis unit can analyze the target's audio data during analysis to understand changes in behavior and emotion. For example, if the target is an animal, the analysis unit can analyze changes in its vocalizations to understand changes in behavior and emotion. If the target is a human, the analysis unit can also analyze the content of conversations to understand learning progress and changes in emotion. Furthermore, if the target is a pet, the analysis unit can analyze the sounds of play to understand changes in behavior and emotion. For example, the analysis unit can analyze changes in animal vocalizations to understand changes in behavior and emotion. It can analyze the content of human conversations to understand learning progress and changes in emotion. It can analyze the sounds of pets playing to understand changes in behavior and emotion. In this way, by analyzing the target's audio data, changes in behavior and emotion can be accurately understood. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or it may be performed without a generative AI. For example, the analysis unit can input the target's audio data into a generative AI, and the generative AI can analyze changes in behavior and emotion.

[0046] The analysis unit can analyze the target's location information during analysis to understand its range of activity and movement patterns. For example, if the target is an animal, the analysis unit can analyze its movement path to understand its range of activity. If the target is a human, the analysis unit can also analyze the location information of its learning location to understand its learning environment. Furthermore, if the target is a pet, the analysis unit can analyze its play area to understand its behavior patterns. For example, the analysis unit can analyze an animal's movement path to understand its range of activity. It can analyze the location information of a human's learning location to understand its learning environment. It can analyze a pet's play area to understand its behavior patterns. In this way, by analyzing the target's location information, its range of activity and movement patterns can be accurately understood. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or it may be performed without using a generative AI. For example, the analysis unit can input the target's location information data into a generative AI, which can then analyze its range of activity and movement patterns.

[0047] The generation unit can generate an optimal program by referring to the target's past learning and behavioral data during generation. For example, if the target is an animal, the generation unit can generate an optimal behavioral modification program by referring to past behavioral data. Furthermore, if the target is a human, the generation unit can generate an optimal learning plan by referring to past learning data. Additionally, if the target is a pet, the generation unit can generate a program that suggests the optimal way to play by referring to past play data. For example, the generation unit can refer to an animal's past behavioral data to generate an optimal behavioral modification program, a human's past learning data to generate an optimal learning plan, and a pet's past play data to generate a program that suggests the optimal way to play. This allows the generation of an optimal program by referring to the target's past learning and behavioral data. Some or all of the above-described processes in the generation unit may be performed using, for example, a generation AI, or without using a generation AI. For example, the generation unit can input the target's past learning and behavioral data into a generation AI, which can then generate an optimal program.

[0048] The generation unit can generate a customized program that takes into account the target's environment and living conditions during the generation process. For example, if the target is an animal, the generation unit can generate an optimal behavioral modification program considering its living environment. Furthermore, if the target is a human, the generation unit can generate an optimal learning plan considering its learning environment. Additionally, if the target is a pet, the generation unit can generate a program that suggests the most appropriate way to play, taking its living environment into account. For example, the generation unit can generate an optimal behavioral modification program considering the animal's living environment; generate an optimal learning plan considering the human's learning environment; and generate a program that suggests the most appropriate way to play considering the pet's living environment. This allows for the provision of more appropriate programs by considering the target's environment and living conditions. Some or all of the above-described processes in the generation unit may be performed using, for example, a generation AI, or without using a generation AI. For example, the generation unit can input data on the target's environment and living conditions into a generation AI, which can then generate a customized program.

[0049] The generation unit can generate a program that includes voice instructions using the target audio data during generation. For example, if the target is an animal, the generation unit can generate a behavior modification program that includes voice instructions based on the analysis results of its vocalizations. The generation unit can also generate a learning plan that includes voice instructions based on the content of a conversation if the target is a human. Furthermore, if the target is a pet, the generation unit can generate a program that includes voice instructions based on the sounds of play. For example, the generation unit can generate a behavior modification program that includes voice instructions based on the analysis results of an animal's vocalizations. It can generate a learning plan that includes voice instructions based on the content of a human conversation. It can generate a program that includes voice instructions based on the sounds of a pet playing. In this way, a program that includes voice instructions can be provided by utilizing the target's audio data. Some or all of the above processing in the generation unit may be performed using a generation AI, for example, or without using a generation AI. For example, the generation unit can input the target's audio data into a generation AI, and the generation AI can generate a program that includes voice instructions.

[0050] The generation unit can generate programs based on movement and range of action by utilizing the target's location information during generation. For example, if the target is an animal, the generation unit can generate a behavioral modification program based on its movement range. Furthermore, if the target is a human, the generation unit can generate a learning plan based on the location information of the learning location. Additionally, if the target is a pet, the generation unit can generate a program that suggests ways to play based on the play location. For example, the generation unit can generate a behavioral modification program based on the animal's movement range, a learning plan based on the location information of the human's learning location, and a program that suggests ways to play based on the pet's play location. This allows the generation unit to provide programs based on movement and range of action by utilizing the target's location information. Some or all of the above-described processes in the generation unit may be performed using, for example, a generation AI, or without using a generation AI. For example, the generation unit can input the target's location data into a generation AI, which can then generate a program based on movement and range of action.

[0051] The information delivery unit can select the optimal information delivery method by referring to the user's past usage history at the time of delivery. For example, the information delivery unit can prioritize displaying information delivery methods that the user has frequently used in the past. The information delivery unit can also select an information delivery method suitable for a specific time period based on the user's past usage history. Furthermore, the information delivery unit can analyze the user's past usage history and select the most efficient information delivery method. For example, the information delivery unit can prioritize displaying information delivery methods that the user has frequently used in the past. It can select an information delivery method suitable for a specific time period based on the user's past usage history. It can analyze the user's past usage history and select the most efficient information delivery method. In this way, the optimal information delivery method can be selected by referring to the user's past usage history. Some or all of the above processing in the information delivery unit may be performed using, for example, a generation AI, or without a generation AI. For example, the information delivery unit can input the user's past usage history data into a generation AI, and the generation AI can select the optimal information delivery method.

[0052] The service provider can select the optimal display format by considering the user's device information at the time of delivery. For example, if the user is using a smartphone, the service provider will provide a display format that matches the screen size. Furthermore, if the user is using a tablet, the service provider can also provide a display format optimized for a larger screen. In addition, if the user is using a smartwatch, the service provider can provide a concise and highly visible display format. This allows the service provider to provide the optimal display format by considering the user's device information. Some or all of the above processing in the service provider may be performed using, for example, a generation AI, or without a generation AI. For example, the service provider can input user device information data into a generation AI, which can then select the optimal display format.

[0053] The information provider can provide information in response to the user's voice instructions at the time of provision. For example, if the user requests information by voice, the information provider will provide the information based on the voice instructions. The information provider can also provide information related to a specific keyword if the user inputs that keyword by voice. Furthermore, the information provider can provide answers to questions if the user asks them by voice. For example, if the information provider requests information by voice, the information provider will provide the information based on the voice instructions. If the user inputs a specific keyword by voice, the information provider will provide information related to that keyword. If the user asks a question by voice, the information provider will provide answers to that question. This makes it possible to provide information more efficiently by providing information in response to the user's voice instructions. Some or all of the above processing in the information provider may be performed using, for example, a generative AI, or without a generative AI. For example, the information provider can input the user's voice data into a generative AI, and the generative AI can provide the information.

[0054] The information provider can provide optimal information by considering the user's location information at the time of provision. For example, if the user is in a specific location, the information provider can provide information related to that location. The information provider can also provide information based on the user's current location if the user is on the move. Furthermore, if the user is in a specific region, the information provider can provide information related to that region. For example, if the information provider is in a specific location, it can provide information related to that location. If the user is on the move, it can provide information based on the user's current location. If the user is in a specific region, it can provide information related to that region. This allows for the provision of more appropriate information by considering the user's location information. Some or all of the above processing in the information provider may be performed using, for example, a generative AI, or without a generative AI. For example, the information provider can input the user's location data into a generative AI, and the generative AI can provide optimal information.

[0055] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0056] The analysis unit can improve the accuracy of its analysis by referring to the subject's past behavioral data during the analysis process. For example, if the subject is an animal, it can analyze its current behavior by referring to its past behavioral patterns. If the subject is a human, it can also analyze its current learning status by referring to its past learning history. Furthermore, if the subject is a pet, it can analyze its current behavior by referring to its past play data. This improves the accuracy of the analysis by referring to the subject's past behavioral data. Some or all of the above-described processes in the analysis unit may be performed using, for example, a generative AI, or without a generative AI. For example, the analysis unit can input the subject's past behavioral data into a generative AI, which can then improve the accuracy of the analysis.

[0057] The camera unit can predict the subject's movements and behavior patterns during shooting and automatically select the optimal shooting angle. For example, if the subject is an animal, it can automatically adjust the camera angle based on the animal's predicted movements. If the subject is a human, it can analyze their behavior patterns and select the optimal shooting position. Furthermore, if the subject is a pet, it can predict the pet's play movements and shoot accordingly. By predicting the subject's movements and behavior patterns and selecting the optimal shooting angle, more effective shooting becomes possible. Some or all of the above processing in the camera unit may be performed using, for example, a generative AI, or it may be performed without a generative AI. For example, the camera unit can input data on the subject's movements and behavior patterns into a generative AI, which can then select the optimal shooting angle.

[0058] The information delivery unit can select the most suitable information delivery method by referring to the user's past usage history at the time of delivery. For example, it can prioritize displaying information delivery methods that the user has frequently used in the past. It can also select an information delivery method suitable for a specific time period based on the user's past usage history. Furthermore, it can analyze the user's past usage history and select the most efficient information delivery method. In this way, the optimal information delivery method can be selected by referring to the user's past usage history. Some or all of the above processing in the information delivery unit may be performed using, for example, a generation AI, or without a generation AI. For example, the information delivery unit can input the user's past usage history data into a generation AI, and the generation AI can select the most suitable information delivery method.

[0059] The analysis unit can analyze the target's audio data during analysis to understand changes in behavior and emotion. For example, if the target is an animal, it can analyze changes in vocalizations to understand changes in behavior and emotion. If the target is a human, it can analyze the content of conversations to understand learning progress and changes in emotion. Furthermore, if the target is a pet, it can analyze play sounds to understand changes in behavior and emotion. In this way, changes in behavior and emotion can be accurately understood by analyzing the target's audio data. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or it may be performed without a generative AI. For example, the analysis unit can input the target's audio data into a generative AI, which can then analyze changes in behavior and emotion.

[0060] The generation unit can generate an optimal program by referring to the target's past learning and behavioral data during the generation process. For example, if the target is an animal, it can generate an optimal behavioral modification program by referring to past behavioral data. If the target is a human, it can also generate an optimal learning plan by referring to past learning data. Furthermore, if the target is a pet, it can generate a program that suggests the optimal way to play by referring to past play data. In this way, an optimal program can be generated by referring to the target's past learning and behavioral data. Some or all of the above processing in the generation unit may be performed using a generation AI, for example, or without a generation AI. For example, the generation unit can input the target's past learning and behavioral data into a generation AI, and the generation AI can generate an optimal program.

[0061] The following briefly describes the processing flow for example form 1.

[0062] Step 1: The photography team takes photos and videos of the subject's daily activities. The photography team can, for example, use smartphones or digital cameras to film the subject's behavior. They can also use surveillance cameras to film for extended periods. For example, they can film a pet's behavior with a smartphone and upload the data to the cloud. Furthermore, they can film the behavior of animals in a zoo with surveillance cameras and collect data in real time. Step 2: The analysis unit analyzes the information captured by the shooting unit to understand the subject's learning and behavioral status. The analysis unit uses generative AI and image recognition technology to analyze the subject's behavior, facial expressions, and physical condition. For example, it can analyze a pet's behavior to determine if it is playing energetically or appears unwell. It can also analyze the behavior of animals in a zoo to understand their health and stress levels. Furthermore, it can analyze a person's learning situation to understand their learning progress and level of comprehension. Step 3: The generation unit generates an optimal learning plan and behavior modification program based on the situation grasped by the analysis unit. The generation unit uses generation AI to generate a program that is appropriate to the target situation. For example, if a pet is unwell, it will generate information on how to deal with the situation and a veterinary hospital. If the pet is playing happily, it can also generate a program that suggests new ways to play. Furthermore, it can also generate an optimal learning plan according to the human's learning progress. Step 4: The provider unit provides the program generated by the generator unit to the user. The provider unit can provide the generated program to the user, for example, through a smartphone app or a web app. It can also notify the user of the generated program using email or a messaging app. Furthermore, it can provide the generated program as a printed document.

[0063] (Example of form 2) The learning and behavioral status monitoring system according to an embodiment of the present invention is a system that uses a generative AI to understand the individual learning and behavioral status and provides an optimal learning plan and behavioral modification program. This system is applicable not only to humans but also to pets and zoo animals. The learning and behavioral status monitoring system works by having the user take photos or videos of the target human or animal's daily behavior. For example, the user can take photos of a pet's behavior and learning status with a smartphone. This information is input into the generative AI. Next, the generative AI analyzes the input photos and videos to understand the target's learning and behavioral status. The generative AI uses image recognition technology to analyze the target's behavior, facial expressions, physical condition, etc. For example, it can determine whether the pet is playing energetically or appears unwell. Based on the results of the generative AI's analysis, it generates an optimal learning plan and behavioral modification program. For example, if the pet is unwell, it provides information on how to deal with the situation and information on veterinary hospitals. If the pet is playing energetically, it suggests new ways to play that will make it even more enjoyable. The generated learning plan and behavioral modification program are provided to the user. The user can modify the target's learning and behavior based on the information provided by the generative AI. For example, if a pet is unwell, the system can implement the suggested solutions provided by the AI ​​and take the pet to a veterinary hospital. This system allows users to understand the subject's learning and behavioral status and take the most appropriate action. For instance, even if a pet is unwell, users can quickly learn how to deal with the situation and provide appropriate care. Furthermore, even if a pet is playing happily, the system can suggest new ways to play, making the experience even more enjoyable. This system is applicable to a variety of subjects, including humans, pets, and zoo animals. For example, by filming the behavior of zoo animals and having the AI ​​analyze the footage, the system can understand the animal's health and stress levels, enabling the provision of optimal care. It can also support efficient learning by understanding a person's learning progress and providing an optimal learning plan. Thus, the learning and behavioral status monitoring system can automatically understand the subject's learning and behavioral status and provide optimal learning plans and behavioral modification programs.

[0064] The learning and behavioral status monitoring system according to this embodiment comprises a shooting unit, an analysis unit, a generation unit, and a provision unit. The shooting unit captures the subject's daily behavior in photographs and videos. The shooting unit can, for example, use a smartphone or digital camera to capture the subject's behavior. The shooting unit can also perform long-term shooting using a surveillance camera. For example, the shooting unit can photograph a pet's behavior with a smartphone and upload the data to the cloud. Furthermore, the shooting unit can also photograph the behavior of animals in a zoo with a surveillance camera and collect data in real time. The analysis unit analyzes the information captured by the shooting unit to understand the subject's learning and behavioral status. The analysis unit uses generation AI and image recognition technology to analyze the subject's behavior, expressions, and physical condition. For example, the analysis unit can analyze a pet's behavior to determine if it is playing energetically or appears unwell. The analysis unit can also analyze the behavior of animals in a zoo to understand their health status and stress levels. Furthermore, the analysis unit can analyze a person's learning situation to understand their learning progress and level of comprehension. The generation unit generates an optimal learning plan and behavior modification program based on the situation grasped by the analysis unit. The generation unit uses a generation AI to generate a program that is appropriate to the target situation. For example, if a pet is unwell, the generation unit generates information on how to deal with the situation and information on veterinary hospitals. The generation unit can also generate a program that suggests new ways to play if the pet is playing happily. Furthermore, the generation unit can also generate an optimal learning plan according to the human's learning progress. The provision unit provides the program generated by the generation unit to the user. The provision unit can provide the generated program to the user, for example, through a smartphone app or a web app. The provision unit can also notify the user of the generated program using email or a messaging app. Furthermore, the provision unit can also provide the generated program as printed material. As a result, the learning and behavior situation awareness system according to this embodiment can capture and analyze the target's daily behavior and provide an optimal learning plan and behavior modification program.

[0065] The photography department captures the subject's daily activities in photos and videos. For example, the photography department can use smartphones or digital cameras to film the subject's behavior. Specifically, high-resolution smartphone cameras can clearly capture the subject's subtle movements and expressions. Digital cameras, utilizing zoom and continuous shooting functions, can capture detailed footage even from long distances. Furthermore, the photography department can use surveillance cameras for long-term recording. Multiple surveillance cameras are installed to cover a wide area, recording the subject's behavior 24 hours a day. For example, the photography department can film a pet's behavior with a smartphone and upload the data to the cloud. The data uploaded to the cloud is managed so that the analysis and generation departments can access it. Additionally, the photography department can use surveillance cameras to film the behavior of animals in zoos and collect data in real time. Surveillance cameras allow for long-term observation without disrupting the animals' natural behavior. This allows the photography department to use a variety of devices to record the subject's behavior in detail and continuously, improving the overall data collection capability of the system.

[0066] The analysis unit analyzes information captured by the shooting unit to understand the subject's learning and behavioral status. Using generative AI and advanced image recognition technology, the analysis unit analyzes the subject's behavior, facial expressions, and physical condition. Specifically, the generative AI uses deep learning algorithms to automatically identify the subject's movements and facial expressions from image and video data. For example, the analysis unit analyzes a pet's behavior to determine if it is playing energetically or appears unwell. The generative AI learns the pet's movement patterns and facial expression changes to detect abnormal behavior or changes in physical condition. The analysis unit can also analyze the behavior of animals in zoos to understand their health and stress levels. Based on animal behavioral data, the generative AI predicts signs of stress and changes in health. Furthermore, the analysis unit can analyze human learning to understand learning progress and comprehension. The generative AI analyzes the learner's facial expressions and behavior to evaluate concentration and comprehension. This allows the analysis unit to quickly and accurately analyze collected data and understand the subject's learning and behavioral status in real time. Furthermore, the analysis unit can utilize historical data and statistical information to analyze long-term behavioral patterns and learning trends. This allows the analysis unit to comprehensively evaluate the target behavior and learning status, improving the overall accuracy and reliability of the system.

[0067] The generation unit generates optimal learning plans and behavioral modification programs based on the situation grasped by the analysis unit. The generation unit uses a generation AI to generate programs tailored to the target situation. Specifically, the generation AI automatically designs the optimal program for the target's needs and situation based on the analysis results. For example, if a pet is unwell, the generation unit generates information on how to deal with the situation and information on veterinary hospitals. The generation AI analyzes the pet's symptoms and behavioral data to provide appropriate treatment methods and information on the nearest veterinary hospital. The generation unit can also generate a program that suggests new ways to play if the pet is playing happily. The generation AI learns the pet's preferences and behavioral patterns and suggests new ways to play and training methods that will pique their interest. Furthermore, the generation unit can also generate an optimal learning plan according to the human's learning progress. The generation AI creates an individually customized learning plan based on the learner's progress and understanding, supporting effective learning. This allows the generation unit to quickly and accurately generate the optimal program tailored to the target situation, maximizing the overall system's effectiveness. In addition, the generation unit continuously evaluates the effectiveness of the generated programs and can modify and improve them as needed. This allows the generation unit to consistently provide high-quality programs based on the latest information and technology, supporting the improvement of the target's learning and behavioral status.

[0068] The service provider delivers the programs generated by the generation unit to the user. The service provider can deliver the generated programs to users, for example, through smartphone apps or web apps. Specifically, a smartphone app provides an easily accessible interface for users, visually displaying the generated programs. A web app provides a platform accessible from anywhere via the internet, allowing users to view and run the generated programs. Furthermore, the service provider can also notify users of the generated programs using email or messaging apps. For example, important notices and urgent instructions can be quickly communicated to users via email or messaging apps. The service provider can also provide the generated programs as printed materials. For example, learning plans and behavioral modification programs can be printed and distributed to users, making them available offline. This allows the service provider to deliver generated programs to users in diverse ways, improving convenience and accessibility. Furthermore, the service provider can collect user feedback and use it to improve delivery methods and program content. This enables the service provider to respond flexibly to user needs, improving overall system satisfaction and effectiveness.

[0069] The analysis unit can analyze the subject's behavior, facial expressions, and physical condition using image recognition technology. For example, the analysis unit can analyze the subject's facial expressions using face recognition technology and estimate emotions. The analysis unit can also analyze the subject's behavior using motion recognition technology and understand behavioral patterns. Furthermore, the analysis unit can analyze the subject's physical condition using physical condition analysis technology and understand its health status. For example, the analysis unit can analyze the subject's facial expressions using face recognition technology and estimate emotions such as smiles or anger. Motion recognition technology analyzes the subject's movements and understands behavioral patterns such as walking or running. Physical condition analysis technology analyzes data such as the subject's body temperature and heart rate and evaluates its health status. In this way, by using image recognition technology, the subject's behavior, facial expressions, and physical condition can be accurately analyzed. Some or all of the above-described processes in the analysis unit may be performed using, for example, a generative AI, or without a generative AI. For example, the analysis unit can input data on the subject's behavior, facial expressions, and physical condition into a generative AI, which can then perform the analysis.

[0070] The generation unit can generate information on how to treat a pet and information on veterinary hospitals when the pet is unwell. For example, if the pet is unwell, the generation unit can generate first aid methods. The generation unit can also generate information such as the location, hours of operation, and specialty of veterinary hospitals. Furthermore, the generation unit can generate methods for administering medication according to the pet's condition. For example, if the pet is unwell, the generation unit can suggest methods of cooling or warming as first aid. The veterinary hospital information provides information on specialists appropriate to the pet's symptoms. The medication administration methods suggest appropriate dosages based on the pet's weight and age. This allows for the provision of appropriate treatment methods and information on veterinary hospitals when a pet is unwell. Some or all of the above processing in the generation unit may be performed using, for example, a generation AI, or without using a generation AI. For example, the generation unit can input pet health data into a generation AI, and the generation AI can generate treatment methods and information on veterinary hospitals.

[0071] The generation unit can generate a program that suggests new ways to play when the pet is playing energetically. For example, the generation unit can suggest new ways to play that the pet can enjoy. The generation unit can also suggest ways to play that increase the pet's exercise level. Furthermore, the generation unit can suggest ways to play that promote the pet's intellectual development. For example, the generation unit can suggest ball games or hide-and-seek as new ways to play that the pet can enjoy. For ways to increase exercise, it can suggest playing in a large space where the pet can run around. For ways to promote intellectual development, it can suggest puzzles and tricks that cultivate the pet's thinking skills. In this way, by suggesting new ways to play when the pet is playing energetically, it can provide even more enjoyable time. Some or all of the above processing in the generation unit may be performed using a generation AI, for example, or without a generation AI. For example, the generation unit can input pet behavior data into a generation AI, and the generation AI can generate a program that suggests new ways to play.

[0072] The service provider can provide users with generated learning plans and behavioral modification programs. The service provider can provide the generated programs to users, for example, through a smartphone app or a web app. The service provider can also notify users of the generated programs via email or messaging apps. Furthermore, the service provider can provide the generated programs as printed materials. For example, the service provider can display the generated learning plan to the user through a smartphone app. A web app is a platform accessible to users via the internet and provides the generated programs. Email and messaging apps send notifications directly to users and provide the generated programs. Printed materials are provided in a format that allows users to receive the programs on paper. This allows users to take appropriate action by providing them with generated learning plans and behavioral modification programs. Some or all of the above processing in the service provider may be performed using, for example, a generation AI, or without a generation AI. For example, the service provider can input the generated programs into a generation AI, and the generation AI can determine how to provide them to the user.

[0073] The analysis unit can analyze the behavior of zoo animals to understand their health status and stress levels. For example, the analysis unit can analyze the animals' behavior patterns to assess their health status. It can also analyze the animals' facial expressions to estimate their stress levels. Furthermore, the analysis unit can analyze the animals' physical condition data to understand their health status. For example, the analysis unit can analyze the animals' behavior patterns to detect abnormal behavior that differs from normal behavior. Facial expression analysis identifies facial expressions that indicate stress and estimates the stress level. Physical condition data analysis analyzes biometric data such as the animals' body temperature and heart rate to assess their health status. This allows for the provision of appropriate care by understanding the health status and stress levels of zoo animals. Some or all of the above-described processes in the analysis unit may be performed using, for example, a generative AI, or not using a generative AI. For example, the analysis unit can input animal behavior data into a generative AI, which can then analyze the health status and stress levels.

[0074] The camera unit can estimate the user's emotions and adjust the shooting timing based on the estimated emotions. For example, if the user is relaxed, the camera unit can shoot for a long time to capture natural behavior. If the user is in a hurry, the camera unit can also shoot continuously to obtain the necessary information in a short time. Furthermore, if the user is stressed, the camera unit can temporarily suspend shooting and resume when the user has calmed down. For example, the camera unit analyzes the user's facial expressions to determine if they are relaxed. If the user is in a hurry, it shoots continuously to obtain a lot of information in a short time. If the user is stressed, it observes the user's facial expressions and behavior and resumes shooting at an appropriate time. By adjusting the shooting timing according to the user's emotions, more natural behavior can be captured. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the camera unit may be performed using, for example, generative AI, or not using generative AI. For example, the camera unit can input the user's facial expression data into a generating AI, which can then estimate emotions and adjust the timing of the shot.

[0075] The camera unit can predict the subject's movements and behavior patterns during shooting and automatically select the optimal shooting angle. For example, if the subject is an animal, the camera unit automatically adjusts the camera angle based on the animal's predicted movements. If the subject is a human, the camera unit can also analyze their behavior patterns and select the optimal shooting position. Furthermore, if the subject is a pet, the camera unit can predict the pet's play movements and shoot accordingly. For example, the camera unit predicts animal movements and automatically adjusts the camera angle. It analyzes human behavior patterns and selects the optimal shooting position. It predicts pet play movements and shoots accordingly. By predicting the subject's movements and behavior patterns and selecting the optimal shooting angle, more effective shooting becomes possible. Some or all of the above processing in the camera unit may be performed using, for example, a generative AI, or without a generative AI. For example, the camera unit can input data on the subject's movements and behavior patterns into a generative AI, which can then select the optimal shooting angle.

[0076] The camera unit can automatically adjust the optimal shooting settings during shooting, taking into account the ambient light and background of the subject. For example, if the ambient light is strong, the camera unit can automatically adjust the exposure to capture the image at the optimal brightness. Furthermore, if the background is complex, the camera unit can blur the background to emphasize the subject. Additionally, if the ambient light is weak, the camera unit can automatically adjust the ISO sensitivity to capture a brighter image. For example, if the ambient light is strong, the camera unit can automatically adjust the exposure to capture the image at the optimal brightness. If the background is complex, it can blur the background to emphasize the subject. If the ambient light is weak, it can automatically adjust the ISO sensitivity to capture a brighter image. This allows for higher quality photography by adjusting the optimal shooting settings considering the ambient light and background of the subject. Some or all of the above processing in the camera unit may be performed using, for example, a generative AI, or without a generative AI. For example, the camera unit can input ambient light and background data into a generative AI, which can then adjust the optimal shooting settings.

[0077] The camera unit can estimate the user's emotions and determine the priority of subjects to photograph based on the estimated emotions. For example, if the user is excited, the camera unit will prioritize photographing subjects with a lot of movement. If the user is relaxed, the camera unit can also prioritize photographing quiet scenes. Furthermore, if the user is stressed, the camera unit can also prioritize photographing relaxing scenes. For example, if the user is excited, the camera unit will prioritize photographing subjects with a lot of movement. If the user is relaxed, it will prioritize photographing quiet scenes. If the user is stressed, it will prioritize photographing relaxing scenes. This allows for more appropriate photography by determining the priority of subjects to photograph according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the camera unit may be performed using, for example, generative AI, or not using generative AI. For example, the shooting unit can input user emotion data into a generating AI, which can then determine the priority of what to photograph.

[0078] The recording unit can simultaneously record the subject's voice and sound environment during recording and use this information for analysis. For example, if the subject is an animal, the recording unit can simultaneously record the animal's vocalizations and use this information for behavioral analysis. If the subject is a human, the recording unit can also record the content of conversations and use this information for analyzing learning progress. Furthermore, if the subject is a pet, the recording unit can record the sounds of play and use this information for analyzing behavioral patterns. For example, the recording unit can simultaneously record animal vocalizations and use this information for behavioral analysis. It can record the content of human conversations and use this information for analyzing learning progress. It can record the sounds of pets playing and use this information for analyzing behavioral patterns. By simultaneously recording the subject's voice and sound environment, more detailed analysis becomes possible. Some or all of the above processing in the recording unit may be performed using, for example, a generative AI, or it may be performed without a generative AI. For example, the recording unit can input the subject's voice data into a generative AI, which can then analyze the voice data.

[0079] The camera unit can acquire location information of the subject during shooting and use it for analysis. For example, if the subject is an animal, the camera unit can record the animal's movement path and use it to analyze its range of activity. If the subject is a human, the camera unit can also record location information of learning locations and use it to analyze the learning environment. Furthermore, if the subject is a pet, the camera unit can record play locations and use it to analyze behavioral patterns. For example, the camera unit can record the movement path of an animal and use it to analyze its range of activity. It can record location information of a human's learning locations and use it to analyze the learning environment. It can record play locations of a pet and use it to analyze behavioral patterns. This makes it possible to analyze the range of activity and movement patterns by acquiring the subject's location information. Some or all of the above processing in the camera unit may be performed using, for example, a generative AI, or it may be performed without a generative AI. For example, the camera unit can input the subject's location information data into a generative AI, and the generative AI can analyze the location information.

[0080] The analysis unit can estimate the user's emotions and adjust the display method of the analysis results based on the estimated emotions. For example, if the user is relaxed, the analysis unit will display detailed analysis results. If the user is in a hurry, the analysis unit can also display concise analysis results that get straight to the point. Furthermore, if the user is stressed, the analysis unit can display the analysis results using visually easy-to-understand graphs or charts. For example, if the user is relaxed, the analysis unit will display detailed analysis results. If the user is in a hurry, it will display concise analysis results that get straight to the point. If the user is stressed, it will display the analysis results using visually easy-to-understand graphs or charts. This allows for a more easily understandable display by adjusting the display method of the analysis results according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generative AI. The generative AI is a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to these examples. Some or all of the above-described processing in the analysis unit may be performed using a generative AI, for example, or without a generative AI. For example, the analysis unit can input user emotion data into a generating AI, which can then adjust how the analysis results are displayed.

[0081] The analysis unit can improve the accuracy of its analysis by referring to the subject's past behavioral data during the analysis. For example, if the subject is an animal, the analysis unit can analyze the current behavior by referring to past behavioral patterns. If the subject is a human, the analysis unit can also analyze the current learning status by referring to past learning history. Furthermore, if the subject is a pet, the analysis unit can analyze the current behavior by referring to past play data. For example, the analysis unit can refer to the animal's past behavioral patterns to analyze the current behavior. It can refer to the human's past learning history to analyze the current learning status. It can refer to the pet's past play data to analyze the current behavior. By referring to the subject's past behavioral data, the accuracy of the analysis is improved. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or it may be performed without using a generative AI. For example, the analysis unit can input the subject's past behavioral data into a generative AI, which can then improve the accuracy of the analysis.

[0082] The analysis unit can monitor the target's behavioral patterns in real time during analysis and detect anomalies. For example, if the target is an animal, the analysis unit can detect movements that deviate from normal behavioral patterns in real time. Furthermore, if the target is a human, the analysis unit can also detect abnormal behavior during learning in real time. Additionally, if the target is a pet, the analysis unit can detect behavior that deviates from normal play patterns in real time. For example, the analysis unit can detect movements that deviate from the normal behavioral patterns of animals in real time. It can detect abnormal behavior during learning in humans in real time. It can detect behavior that deviates from normal play patterns of pets in real time. This allows for rapid detection of anomalies by monitoring the target's behavioral patterns in real time. Some or all of the above-described processes in the analysis unit may be performed using, for example, a generative AI, or without a generative AI. For example, the analysis unit can input the target's behavioral pattern data into a generative AI, which can then detect anomalies.

[0083] The analysis unit can estimate the user's emotions and determine the priority of analysis results based on the estimated emotions. For example, if the user is relaxed, the analysis unit will prioritize displaying detailed analysis results. It can also prioritize displaying important analysis results if the user is in a hurry. Furthermore, if the user is stressed, the analysis unit can prioritize displaying visually easy-to-understand analysis results. For example, if the user is relaxed, the analysis unit will prioritize displaying detailed analysis results. If the user is in a hurry, it will prioritize displaying important analysis results. If the user is stressed, it will prioritize displaying visually easy-to-understand analysis results. This allows for the prioritization of more important information by determining the priority of analysis results according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processing in the analysis unit may be performed using, for example, generative AI, or without generative AI. For example, the analysis unit can input user emotion data into a generating AI, which can then determine the priority of the analysis results.

[0084] The analysis unit can analyze the target's audio data during analysis to understand changes in behavior and emotion. For example, if the target is an animal, the analysis unit can analyze changes in its vocalizations to understand changes in behavior and emotion. If the target is a human, the analysis unit can also analyze the content of conversations to understand learning progress and changes in emotion. Furthermore, if the target is a pet, the analysis unit can analyze the sounds of play to understand changes in behavior and emotion. For example, the analysis unit can analyze changes in animal vocalizations to understand changes in behavior and emotion. It can analyze the content of human conversations to understand learning progress and changes in emotion. It can analyze the sounds of pets playing to understand changes in behavior and emotion. In this way, by analyzing the target's audio data, changes in behavior and emotion can be accurately understood. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or it may be performed without a generative AI. For example, the analysis unit can input the target's audio data into a generative AI, and the generative AI can analyze changes in behavior and emotion.

[0085] The analysis unit can analyze the target's location information during analysis to understand its range of activity and movement patterns. For example, if the target is an animal, the analysis unit can analyze its movement path to understand its range of activity. If the target is a human, the analysis unit can also analyze the location information of its learning location to understand its learning environment. Furthermore, if the target is a pet, the analysis unit can analyze its play area to understand its behavior patterns. For example, the analysis unit can analyze an animal's movement path to understand its range of activity. It can analyze the location information of a human's learning location to understand its learning environment. It can analyze a pet's play area to understand its behavior patterns. In this way, by analyzing the target's location information, its range of activity and movement patterns can be accurately understood. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or it may be performed without using a generative AI. For example, the analysis unit can input the target's location information data into a generative AI, which can then analyze its range of activity and movement patterns.

[0086] The generation unit can estimate the user's emotions and adjust the content of the program it generates based on the estimated emotions. For example, if the user is relaxed, the generation unit can generate a detailed learning plan. If the user is in a hurry, the generation unit can also generate a concise behavioral modification program. Furthermore, if the user is stressed, the generation unit can generate a program that helps them relax. For example, if the user is relaxed, the generation unit generates a detailed learning plan. If the user is in a hurry, it generates a concise behavioral modification program. If the user is stressed, it generates a program that helps them relax. This allows for the provision of more appropriate programs by adjusting the content of the program generated according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the generation unit may be performed using a generation AI, for example, or without a generation AI. For example, the generation unit can input user emotion data into a generation AI, which can then adjust the program content.

[0087] The generation unit can generate an optimal program by referring to the target's past learning and behavioral data during generation. For example, if the target is an animal, the generation unit can generate an optimal behavioral modification program by referring to past behavioral data. Furthermore, if the target is a human, the generation unit can generate an optimal learning plan by referring to past learning data. Additionally, if the target is a pet, the generation unit can generate a program that suggests the optimal way to play by referring to past play data. For example, the generation unit can refer to an animal's past behavioral data to generate an optimal behavioral modification program, a human's past learning data to generate an optimal learning plan, and a pet's past play data to generate a program that suggests the optimal way to play. This allows the generation of an optimal program by referring to the target's past learning and behavioral data. Some or all of the above-described processes in the generation unit may be performed using, for example, a generation AI, or without using a generation AI. For example, the generation unit can input the target's past learning and behavioral data into a generation AI, which can then generate an optimal program.

[0088] The generation unit can generate a customized program that takes into account the target's environment and living conditions during the generation process. For example, if the target is an animal, the generation unit can generate an optimal behavioral modification program considering its living environment. Furthermore, if the target is a human, the generation unit can generate an optimal learning plan considering its learning environment. Additionally, if the target is a pet, the generation unit can generate a program that suggests the most appropriate way to play, taking its living environment into account. For example, the generation unit can generate an optimal behavioral modification program considering the animal's living environment; generate an optimal learning plan considering the human's learning environment; and generate a program that suggests the most appropriate way to play considering the pet's living environment. This allows for the provision of more appropriate programs by considering the target's environment and living conditions. Some or all of the above-described processes in the generation unit may be performed using, for example, a generation AI, or without using a generation AI. For example, the generation unit can input data on the target's environment and living conditions into a generation AI, which can then generate a customized program.

[0089] The generation unit can estimate the user's emotions and determine the priority of the programs to generate based on the estimated emotions. For example, if the user is relaxed, the generation unit will prioritize generating detailed programs. If the user is in a hurry, the generation unit can also prioritize generating concise programs. Furthermore, if the user is stressed, the generation unit can also prioritize generating relaxing programs. For example, if the user is relaxed, the generation unit will prioritize generating detailed programs. If the user is in a hurry, it will prioritize generating concise programs. If the user is stressed, it will prioritize generating relaxing programs. This allows for the prioritization of more important programs by determining the program priority according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the generation unit may be performed using a generation AI, or not. For example, the generation unit can input user emotion data into a generation AI, which can then determine the program priority.

[0090] The generation unit can generate a program that includes voice instructions using the target audio data during generation. For example, if the target is an animal, the generation unit can generate a behavior modification program that includes voice instructions based on the analysis results of its vocalizations. The generation unit can also generate a learning plan that includes voice instructions based on the content of a conversation if the target is a human. Furthermore, if the target is a pet, the generation unit can generate a program that includes voice instructions based on the sounds of play. For example, the generation unit can generate a behavior modification program that includes voice instructions based on the analysis results of an animal's vocalizations. It can generate a learning plan that includes voice instructions based on the content of a human conversation. It can generate a program that includes voice instructions based on the sounds of a pet playing. In this way, a program that includes voice instructions can be provided by utilizing the target's audio data. Some or all of the above processing in the generation unit may be performed using a generation AI, for example, or without using a generation AI. For example, the generation unit can input the target's audio data into a generation AI, and the generation AI can generate a program that includes voice instructions.

[0091] The generation unit can generate programs based on movement and range of action by utilizing the target's location information during generation. For example, if the target is an animal, the generation unit can generate a behavioral modification program based on its movement range. Furthermore, if the target is a human, the generation unit can generate a learning plan based on the location information of the learning location. Additionally, if the target is a pet, the generation unit can generate a program that suggests ways to play based on the play location. For example, the generation unit can generate a behavioral modification program based on the animal's movement range, a learning plan based on the location information of the human's learning location, and a program that suggests ways to play based on the pet's play location. This allows the generation unit to provide programs based on movement and range of action by utilizing the target's location information. Some or all of the above-described processes in the generation unit may be performed using, for example, a generation AI, or without using a generation AI. For example, the generation unit can input the target's location data into a generation AI, which can then generate a program based on movement and range of action.

[0092] The information provider can estimate the user's emotions and adjust how the information is displayed based on the estimated emotions. For example, if the user is relaxed, the provider can display detailed information. If the user is in a hurry, the provider can also display concise information to the point. Furthermore, if the user is stressed, the provider can display information using visually easy-to-understand graphs and charts. For example, if the user is relaxed, the provider can display detailed information. If the user is in a hurry, it can display concise information to the point. If the user is stressed, it can display information using visually easy-to-understand graphs and charts. In this way, by adjusting how information is displayed according to the user's emotions, it is possible to provide information that is easier to understand. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the information provider may be performed using a generative AI, for example, or without a generative AI. For example, the service provider can input user emotion data into a generating AI, which can then adjust how the information is displayed.

[0093] The information delivery unit can select the optimal information delivery method by referring to the user's past usage history at the time of delivery. For example, the information delivery unit can prioritize displaying information delivery methods that the user has frequently used in the past. The information delivery unit can also select an information delivery method suitable for a specific time period based on the user's past usage history. Furthermore, the information delivery unit can analyze the user's past usage history and select the most efficient information delivery method. For example, the information delivery unit can prioritize displaying information delivery methods that the user has frequently used in the past. It can select an information delivery method suitable for a specific time period based on the user's past usage history. It can analyze the user's past usage history and select the most efficient information delivery method. In this way, the optimal information delivery method can be selected by referring to the user's past usage history. Some or all of the above processing in the information delivery unit may be performed using, for example, a generation AI, or without a generation AI. For example, the information delivery unit can input the user's past usage history data into a generation AI, and the generation AI can select the optimal information delivery method.

[0094] The service provider can select the optimal display format by considering the user's device information at the time of delivery. For example, if the user is using a smartphone, the service provider will provide a display format that matches the screen size. Furthermore, if the user is using a tablet, the service provider can also provide a display format optimized for a larger screen. In addition, if the user is using a smartwatch, the service provider can provide a concise and highly visible display format. This allows the service provider to provide the optimal display format by considering the user's device information. Some or all of the above processing in the service provider may be performed using, for example, a generation AI, or without a generation AI. For example, the service provider can input user device information data into a generation AI, which can then select the optimal display format.

[0095] The service provider can estimate the user's emotions and determine the priority of the information to be provided based on the estimated emotions. For example, if the user is relaxed, the service provider may prioritize providing detailed information. If the user is in a hurry, the service provider may also prioritize providing important information. Furthermore, if the user is stressed, the service provider may also prioritize providing visually easy-to-understand information. For example, if the user is relaxed, the service provider may prioritize providing detailed information. If the user is in a hurry, it may prioritize providing important information. If the user is stressed, it may prioritize providing visually easy-to-understand information. This allows for the prioritization of more important information by determining the priority of information according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the service provider may be performed using, for example, a generative AI, or not using a generative AI. For example, the service provider can input user emotion data into a generative AI, which can then determine the priority of the information.

[0096] The information provider can provide information in response to the user's voice instructions at the time of provision. For example, if the user requests information by voice, the information provider will provide the information based on the voice instructions. The information provider can also provide information related to a specific keyword if the user inputs that keyword by voice. Furthermore, the information provider can provide answers to questions if the user asks them by voice. For example, if the information provider requests information by voice, the information provider will provide the information based on the voice instructions. If the user inputs a specific keyword by voice, the information provider will provide information related to that keyword. If the user asks a question by voice, the information provider will provide answers to that question. This makes it possible to provide information more efficiently by providing information in response to the user's voice instructions. Some or all of the above processing in the information provider may be performed using, for example, a generative AI, or without a generative AI. For example, the information provider can input the user's voice data into a generative AI, and the generative AI can provide the information.

[0097] The information provider can provide optimal information by considering the user's location information at the time of provision. For example, if the user is in a specific location, the information provider can provide information related to that location. The information provider can also provide information based on the user's current location if the user is on the move. Furthermore, if the user is in a specific region, the information provider can provide information related to that region. For example, if the information provider is in a specific location, it can provide information related to that location. If the user is on the move, it can provide information based on the user's current location. If the user is in a specific region, it can provide information related to that region. This allows for the provision of more appropriate information by considering the user's location information. Some or all of the above processing in the information provider may be performed using, for example, a generative AI, or without a generative AI. For example, the information provider can input the user's location data into a generative AI, and the generative AI can provide optimal information.

[0098] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0099] The analysis unit can estimate the user's emotions and adjust the display method of the analysis results based on the estimated emotions. For example, if the user is relaxed, detailed analysis results can be displayed. If the user is in a hurry, concise analysis results that get straight to the point can be displayed. Furthermore, if the user is stressed, the analysis results can be displayed using visually easy-to-understand graphs or charts. In this way, by adjusting the display method of the analysis results according to the user's emotions, a more easily understandable display becomes possible. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the analysis unit may be performed using a generative AI, or not using a generative AI. For example, the analysis unit can input user emotion data into a generative AI, and the generative AI can adjust the display method of the analysis results.

[0100] The service provider can estimate the user's emotions and adjust how the information is displayed based on the estimated emotions. For example, if the user is relaxed, detailed information can be displayed. If the user is in a hurry, concise information can be displayed. Furthermore, if the user is stressed, information can be displayed using visually easy-to-understand graphs or charts. In this way, by adjusting how information is displayed according to the user's emotions, more easily understandable information can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the service provider may be performed using a generative AI, or not using a generative AI. For example, the service provider can input user emotion data into a generative AI, and the generative AI can adjust how the information is displayed.

[0101] The camera unit can estimate the user's emotions and adjust the shooting timing based on the estimated emotions. For example, if the user is relaxed, it can shoot for a long time to capture natural behavior. If the user is in a hurry, it can shoot continuously to obtain the necessary information in a short time. Furthermore, if the user is stressed, it can temporarily suspend shooting and resume when the user has calmed down. By adjusting the shooting timing according to the user's emotions, it is possible to capture more natural behavior. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the camera unit may be performed using a generative AI, or not. For example, the camera unit can input user facial expression data into a generative AI, which can estimate emotions and adjust the shooting timing.

[0102] The generation unit can estimate the user's emotions and adjust the content of the generated program based on the estimated user emotions. For example, if the user is relaxed, it can generate a detailed learning plan. If the user is in a hurry, it can also generate a concise behavioral modification program. Furthermore, if the user is stressed, it can generate a program that helps them relax. In this way, by adjusting the content of the generated program according to the user's emotions, a more appropriate program can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the generation unit may be performed using a generative AI, or not using a generative AI. For example, the generation unit can input user emotion data into a generative AI, and the generative AI can adjust the content of the program.

[0103] The service provider can estimate the user's emotions and determine the priority of the information to be provided based on the estimated emotions. For example, if the user is relaxed, detailed information may be prioritized. If the user is in a hurry, important information may be prioritized. Furthermore, if the user is stressed, visually easy-to-understand information may be prioritized. In this way, by prioritizing information according to the user's emotions, more important information can be provided preferentially. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the service provider may be performed using a generative AI, or not using a generative AI. For example, the service provider can input user emotion data into a generative AI, and the generative AI can determine the priority of the information.

[0104] The analysis unit can improve the accuracy of its analysis by referring to the subject's past behavioral data during the analysis process. For example, if the subject is an animal, it can analyze its current behavior by referring to its past behavioral patterns. If the subject is a human, it can also analyze its current learning status by referring to its past learning history. Furthermore, if the subject is a pet, it can analyze its current behavior by referring to its past play data. This improves the accuracy of the analysis by referring to the subject's past behavioral data. Some or all of the above-described processes in the analysis unit may be performed using, for example, a generative AI, or without a generative AI. For example, the analysis unit can input the subject's past behavioral data into a generative AI, which can then improve the accuracy of the analysis.

[0105] The camera unit can predict the subject's movements and behavior patterns during shooting and automatically select the optimal shooting angle. For example, if the subject is an animal, it can automatically adjust the camera angle based on the animal's predicted movements. If the subject is a human, it can analyze their behavior patterns and select the optimal shooting position. Furthermore, if the subject is a pet, it can predict the pet's play movements and shoot accordingly. By predicting the subject's movements and behavior patterns and selecting the optimal shooting angle, more effective shooting becomes possible. Some or all of the above processing in the camera unit may be performed using, for example, a generative AI, or it may be performed without a generative AI. For example, the camera unit can input data on the subject's movements and behavior patterns into a generative AI, which can then select the optimal shooting angle.

[0106] The information delivery unit can select the most suitable information delivery method by referring to the user's past usage history at the time of delivery. For example, it can prioritize displaying information delivery methods that the user has frequently used in the past. It can also select an information delivery method suitable for a specific time period based on the user's past usage history. Furthermore, it can analyze the user's past usage history and select the most efficient information delivery method. In this way, the optimal information delivery method can be selected by referring to the user's past usage history. Some or all of the above processing in the information delivery unit may be performed using, for example, a generation AI, or without a generation AI. For example, the information delivery unit can input the user's past usage history data into a generation AI, and the generation AI can select the most suitable information delivery method.

[0107] The analysis unit can analyze the target's audio data during analysis to understand changes in behavior and emotion. For example, if the target is an animal, it can analyze changes in vocalizations to understand changes in behavior and emotion. If the target is a human, it can analyze the content of conversations to understand learning progress and changes in emotion. Furthermore, if the target is a pet, it can analyze play sounds to understand changes in behavior and emotion. In this way, changes in behavior and emotion can be accurately understood by analyzing the target's audio data. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or it may be performed without a generative AI. For example, the analysis unit can input the target's audio data into a generative AI, which can then analyze changes in behavior and emotion.

[0108] The generation unit can generate an optimal program by referring to the target's past learning and behavioral data during the generation process. For example, if the target is an animal, it can generate an optimal behavioral modification program by referring to past behavioral data. If the target is a human, it can also generate an optimal learning plan by referring to past learning data. Furthermore, if the target is a pet, it can generate a program that suggests the optimal way to play by referring to past play data. In this way, an optimal program can be generated by referring to the target's past learning and behavioral data. Some or all of the above processing in the generation unit may be performed using a generation AI, for example, or without a generation AI. For example, the generation unit can input the target's past learning and behavioral data into a generation AI, and the generation AI can generate an optimal program.

[0109] The following briefly describes the processing flow for example form 2.

[0110] Step 1: The photography team takes photos and videos of the subject's daily activities. The photography team can, for example, use smartphones or digital cameras to film the subject's behavior. They can also use surveillance cameras to film for extended periods. For example, they can film a pet's behavior with a smartphone and upload the data to the cloud. Furthermore, they can film the behavior of animals in a zoo with surveillance cameras and collect data in real time. Step 2: The analysis unit analyzes the information captured by the shooting unit to understand the subject's learning and behavioral status. The analysis unit uses generative AI and image recognition technology to analyze the subject's behavior, facial expressions, and physical condition. For example, it can analyze a pet's behavior to determine if it is playing energetically or appears unwell. It can also analyze the behavior of animals in a zoo to understand their health and stress levels. Furthermore, it can analyze a person's learning situation to understand their learning progress and level of comprehension. Step 3: The generation unit generates an optimal learning plan and behavior modification program based on the situation grasped by the analysis unit. The generation unit uses generation AI to generate a program that is appropriate to the target situation. For example, if a pet is unwell, it will generate information on how to deal with the situation and a veterinary hospital. If the pet is playing happily, it can also generate a program that suggests new ways to play. Furthermore, it can also generate an optimal learning plan according to the human's learning progress. Step 4: The provider unit provides the program generated by the generator unit to the user. The provider unit can provide the generated program to the user, for example, through a smartphone app or a web app. It can also notify the user of the generated program using email or a messaging app. Furthermore, it can provide the generated program as a printed document.

[0111] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0112] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0113] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0114] Each of the multiple elements described above, including the imaging unit, analysis unit, generation unit, and provision unit, is implemented, for example, by at least one of the smart device 14 and the data processing unit 12. For example, the imaging unit can capture images of the subject's daily behavior using the camera 42 of the smart device 14. The analysis unit is implemented by the specific processing unit 290 of the data processing unit 12 and analyzes the captured information using a generation AI. The generation unit is implemented by the specific processing unit 290 of the data processing unit 12 and generates an optimal learning plan and behavior modification program based on the analysis results. The provision unit is implemented by the control unit 46A of the smart device 14 and provides the generated program to the user. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0115] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0116] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0117] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0118] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0119] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0120] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0121] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0122] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0123] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0124] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0125] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0126] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0127] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0128] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0129] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0130] Each of the multiple elements described above, including the imaging unit, analysis unit, generation unit, and provision unit, is implemented in at least one of the smart glasses 214 and the data processing unit 12. For example, the imaging unit can capture the subject's daily behavior using the camera 42 of the smart glasses 214. The analysis unit is implemented by the specific processing unit 290 of the data processing unit 12 and analyzes the captured information using the generation AI. The generation unit is implemented by the specific processing unit 290 of the data processing unit 12 and generates an optimal learning plan and behavior modification program based on the analysis results. The provision unit is implemented by the control unit 46A of the smart glasses 214 and provides the generated program to the user. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0131] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0132] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0133] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0134] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0135] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0136] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0137] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0138] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0139] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0140] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0141] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0142] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0143] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0144] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0145] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0146] Each of the multiple elements described above, including the imaging unit, analysis unit, generation unit, and provision unit, is implemented in at least one of the headset terminal 314 and the data processing unit 12. For example, the imaging unit can capture images of the subject's daily behavior using the camera 42 of the headset terminal 314. The analysis unit is implemented by the specific processing unit 290 of the data processing unit 12 and analyzes the captured information using a generation AI. The generation unit is implemented by the specific processing unit 290 of the data processing unit 12 and generates an optimal learning plan and behavior modification program based on the analysis results. The provision unit is implemented by the control unit 46A of the headset terminal 314 and provides the generated program to the user. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0147] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0148] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0149] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0150] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0151] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0152] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0153] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0154] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0155] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0156] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0157] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0158] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0159] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0160] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0161] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0162] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0163] Each of the multiple elements described above, including the imaging unit, analysis unit, generation unit, and provision unit, is implemented in at least one of the robot 414 and the data processing unit 12. For example, the imaging unit can use the camera 42 of the robot 414 to capture images of the subject's daily behavior. The analysis unit is implemented by the specific processing unit 290 of the data processing unit 12 and analyzes the captured information using the generation AI. The generation unit is implemented by the specific processing unit 290 of the data processing unit 12 and generates an optimal learning plan and behavior modification program based on the analysis results. The provision unit is implemented by the control unit 46A of the robot 414 and provides the generated program to the user. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0164] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0165] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0166] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0167] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0168] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0169] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0170] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0171] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0172] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0173] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0174] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0175] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0176] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0177] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0178] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0179] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0180] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0181] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0182] (Note 1) The photography team takes photos and videos of the subject's daily life, An analysis unit analyzes the information captured by the aforementioned imaging unit to understand the subject's learning and behavioral status, Based on the situation grasped by the analysis unit, a generation unit generates an optimal learning plan and behavioral modification program. The system comprises a providing unit that provides the program generated by the generation unit. A system characterized by the following features. (Note 2) The aforementioned analysis unit, Using image recognition technology, we analyze the subject's behavior, facial expressions, and physical condition. The system described in Appendix 1, characterized by the features described herein. (Note 3) The generating unit is If your pet is unwell, this tool will generate information on how to treat it and recommend a veterinary clinic. The system described in Appendix 1, characterized by the features described herein. (Note 4) The generating unit is If the pet is playing energetically, it generates a program that suggests new ways to play. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned supply unit is, Provides users with generated learning plans and behavioral modification programs. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned analysis unit, Analyzing the behavior of zoo animals to understand their health status and stress levels. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned imaging unit is It estimates the user's emotions and adjusts the shooting timing based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned imaging unit is During shooting, the system predicts the subject's movements and behavior patterns to automatically select the optimal shooting angle. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned imaging unit is During shooting, the system automatically adjusts the optimal shooting settings, taking into account the ambient light and background of the subject. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned imaging unit is It estimates the user's emotions and determines the priority of subjects to photograph based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned imaging unit is During shooting, the subject's voice and sound environment are simultaneously recorded and used for analysis. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned imaging unit is During shooting, the location information of the target is acquired and used for analysis. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned analysis unit, It estimates the user's emotions and adjusts how the analysis results are displayed based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned analysis unit, During analysis, past behavioral data of the subject is referenced to improve the accuracy of the analysis. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned analysis unit, During analysis, the target's behavioral patterns are monitored in real time to detect anomalies. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned analysis unit, It estimates the user's emotions and prioritizes the analysis results based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned analysis unit, During the analysis, the target's audio data is analyzed to understand changes in behavior and emotions. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned analysis unit, During the analysis, the target's location information is analyzed to understand its range of activity and movement patterns. The system described in Appendix 1, characterized by the features described herein. (Note 19) The generating unit is It estimates the user's emotions and adjusts the program content generated based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 20) The generating unit is During generation, the program is optimized by referring to the target's past learning and behavioral data. The system described in Appendix 1, characterized by the features described herein. (Note 21) The generating unit is During generation, a customized program is created that takes into account the target environment and living situation. The system described in Appendix 1, characterized by the features described herein. (Note 22) The generating unit is It estimates the user's emotions and determines the priority of programs to generate based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 23) The generating unit is During generation, the program, including voice commands, is generated using the target audio data. The system described in Appendix 1, characterized by the features described herein. (Note 24) The generating unit is During generation, the program is generated using the target's location information, based on its movement and range of action. The system described in Appendix 1, characterized by the features described herein. (Note 25) The aforementioned supply unit is, It estimates the user's emotions and adjusts how information is displayed based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 26) The aforementioned supply unit is, When providing information, the system will select the most suitable method of information delivery by referring to the user's past usage history. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned supply unit is, When providing the service, the optimal display format is selected considering the user's device information. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned supply unit is, It estimates the user's emotions and prioritizes the information provided based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned supply unit is, When providing information, it will be delivered in response to the user's voice commands. The system described in Appendix 1, characterized by the features described herein. (Note 30) The aforementioned supply unit is, When providing information, the system will take the user's location into consideration to provide the most relevant information. The system described in Appendix 1, characterized by the features described herein. [Explanation of Symbols]

[0183] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. The photography team takes photos and videos of the subject's daily life, An analysis unit analyzes the information captured by the aforementioned imaging unit to understand the subject's learning and behavioral status, Based on the situation grasped by the analysis unit, a generation unit generates an optimal learning plan and behavioral modification program. The system comprises a providing unit that provides the program generated by the generation unit. A system characterized by the following features.

2. The aforementioned analysis unit, Using image recognition technology, we analyze the subject's behavior, facial expressions, and physical condition. The system according to feature 1.

3. The generating unit is If your pet is unwell, this tool will generate information on how to treat it and recommend a veterinary clinic. The system according to feature 1.

4. The generating unit is If the pet is playing energetically, it generates a program that suggests new ways to play. The system according to feature 1.

5. The aforementioned supply unit is, Provides users with generated learning plans and behavioral modification programs. The system according to feature 1.

6. The aforementioned analysis unit, Analyzing the behavior of zoo animals to understand their health status and stress levels. The system according to feature 1.

7. The aforementioned imaging unit is It estimates the user's emotions and adjusts the shooting timing based on the estimated user emotions. The system according to feature 1.

8. The aforementioned imaging unit is During shooting, the system predicts the subject's movements and behavior patterns to automatically select the optimal shooting angle. The system according to feature 1.

Images