system

Abstract

We provide the system. [Solution] Using multiple miniature autonomous devices controlled by the generated artificial intelligence, A means for generating control signals to search for human lives at a disaster site, A means for monitoring the progress of the search and adjusting commands via a user interface, A means for tracking the current location and operating status of each miniature autonomous device in real time, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern disaster sites, rescue operations are required to be rapid and efficient. However, with conventional methods, it is difficult to search in narrow spaces or unstable environments where humans cannot directly access, resulting in delays in rescue operations and limitations in the search range. Under such circumstances, new technologies are needed to effectively rescue many lives.

Means for Solving the Problems

[0005] This invention provides a system that utilizes generated artificial intelligence to simultaneously control multiple miniature autonomous devices, enabling rapid search for people at disaster sites. This system includes means for generating control signals and allows monitoring of the search progress and adjustment of commands via a user interface. Furthermore, it improves the efficiency of the search by tracking the current location and operating status of each miniature autonomous device in real time. In addition, its biomimetic structure allows it to enter confined spaces, enabling operations in areas that were previously inaccessible.

[0006] "Generated artificial intelligence" refers to an artificial intelligence system trained to autonomously make decisions and generate appropriate control signals for a given task.

[0007] A "miniature autonomous device" is a small autonomous robot that mimics the characteristics of insects and small organisms, enabling movement and exploration in confined spaces.

[0008] A "control signal" is a series of command information generated to cause a miniature autonomous device to perform a specific action.

[0009] A "user interface" is a set of interaction mechanisms, including input and display devices, that a user uses to operate and monitor a system.

[0010] "Exploration progress" refers to information indicating the progress of exploration activities at a disaster site, and includes various states such as the current location and unexplored areas.

[0011] A "biomimic structure" is a structure designed to mimic the characteristics and movements of living organisms, and in particular, it refers to a design that possesses the flexibility and mobility of insects. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention relates to a system for supporting search and rescue operations at disaster sites using artificial intelligence. Specifically, it can simultaneously control multiple miniature autonomous devices, enabling rapid searching even in confined spaces where human access is difficult.

[0034] First, the server receives map data and sensor information from the disaster site and analyzes it to generate the optimal search route. At this time, the generated artificial intelligence considers past data and the characteristics of the site to identify important search points. Based on this information, the server generates control signals to instruct the actions of each miniature autonomous device.

[0035] The terminal is operated by the user conducting the rescue operation and has the function of receiving and visually displaying the generated control signals. The terminal also monitors the current location and operating status of each miniature autonomous device in real time, allowing the user to supervise and adjust the search operation as needed. This function enables the user to make rapid decisions in response to the situation they face.

[0036] Users manage the progress of the exploration through their terminal and adjust the exploration strategy as needed. For example, if an important discovery is made, the user can reroute the exploration route and redeploy more autonomous devices to a specific area.

[0037] A concrete example is the search for survivors inside collapsed buildings during natural disasters such as earthquakes. The server considers the changes in the building structure after the earthquake and plans the optimal search route for the miniature autonomous device. Through a terminal, the user can monitor the ongoing search, make manual adjustments as needed, and ultimately take swift and accurate measures to save many lives.

[0038] Thus, by combining the generated artificial intelligence with a miniature autonomous device, this invention efficiently supports search and rescue activities at disaster sites, which were difficult with conventional methods, and greatly expands the possibility of saving lives.

[0039] The following describes the processing flow.

[0040] Step 1:

[0041] The server receives map data of the disaster site and sensor information acquired in real time. This creates a data base based on the latest situation on the ground.

[0042] Step 2:

[0043] The server uses generated artificial intelligence to analyze the received map data and understand the structure of the disaster site. This identifies specific search points and danger areas, which are then mapped according to their importance.

[0044] Step 3:

[0045] The server determines the search route for the miniature autonomous device based on the identified search points. The device's functions and surrounding environment are considered in formulating the optimal path.

[0046] Step 4:

[0047] The server generates control signals for each miniature autonomous device, instructing its operation. These signals include routing settings and operating mode specifications.

[0048] Step 5:

[0049] The terminal receives control signals transmitted from the server and provides them to the rescue team as visual information. The screen displays the device's location and operating status in real time.

[0050] Step 6:

[0051] Users monitor the information displayed through their devices and adjust their search activities as needed. For example, they might reset their search route based on new information or deploy additional equipment to specific areas.

[0052] Step 7:

[0053] The server continuously collects data from each autonomous device and analyzes the search results and the status of the devices. This allows for updating the area assessment and optimizing commands for the next step.

[0054] Step 8:

[0055] The terminal provides users with the latest progress and analysis results, forming an information infrastructure to maximize the effectiveness of search and rescue operations. Based on this, users can make strategic decisions and improve the efficiency of rescue operations.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] Rapid and effective search and rescue operations are required at disaster sites, but conventional technologies have problems such as low search efficiency in areas difficult for humans to access and difficulty in real-time situation assessment and rapid response. This invention aims to solve these problems and increase the possibility of saving lives.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for analyzing generated data and calculating the optimal search route based on past disaster data; means for generating control signals for searching for targets at a disaster site using multiple small autonomous mechanisms controlled by the generated artificial intelligence; and means for monitoring the progress of the search and adjusting commands via a user interface. This enables rapid and effective search activities at disaster sites.

[0061] "Generated data" refers to information obtained from disaster sites and includes various source data such as sensor data and geographical information.

[0062] The "optimal search route" is a path calculated based on analyzed historical disaster data and on-site information to maximize the efficiency of search activities.

[0063] A "miniature autonomous mechanism" is a small mechanical device controlled by artificial intelligence, with a structure that allows it to enter confined spaces and operate autonomously.

[0064] A "control signal" is a signal generated to instruct a small autonomous mechanism on its operation, and it contains command information necessary for carrying out search activities.

[0065] A "user interface" is a screen or device that provides a means for the user to monitor their exploration activities and adjust commands.

[0066] "Progress status" refers to the current state of the exploration activity and indicates the extent to which the plan has been implemented.

[0067] The following system configuration and operation are provided as embodiments for carrying out this invention.

[0068] The server plays a role in integrating and managing geographic data acquired from disaster sites and information from sensors. Using this data, the server utilizes a generated AI model to perform analysis that takes into account past disaster data and site characteristics. This allows for the calculation of the optimal search route. The server requires computing hardware with powerful processing capabilities and data analysis functions, and the software used consists of a platform on which data analysis algorithms and AI models operate. A concrete example is real-time analysis using cloud computing services.

[0069] The terminal receives control signals from the server at the disaster site and presents the search status to the user through a visual interface. The terminal is a crucial tool for users to monitor search activities and adjust commands as needed. The user interface is designed to be intuitive and easy to use, and operates on tablets and dedicated monitors. The terminal is equipped with a communication module for real-time data exchange with the server.

[0070] The user uses a device to supervise search and rescue operations at the disaster site. The user manages the progress of the search based on real-time information and controls the search strategy as new information becomes available. For example, in a search within a building whose structure has collapsed due to an earthquake, the user might instruct areas to be searched intensively while avoiding dangerous zones.

[0071] Furthermore, an example of a prompt statement in this invention is, "Explain how to use the sensor data to allow the generating AI model to identify search points." This indicates how the AI ​​model processes the data and extracts important points.

[0072] Thus, the embodiment for carrying out the present invention provides a system that efficiently supports search and rescue activities at disaster sites and enables life-saving efforts.

[0073] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0074] Step 1:

[0075] The server receives real-time geographic and sensor data from disaster sites. Inputs are data acquired from various sensors and drones deployed at the site. The server integrates and organizes this data, preparing it for analysis. Outputs are normalized datasets used in subsequent analysis stages.

[0076] Step 2:

[0077] The server analyzes incoming data using a generative AI model and compares it against a database of past disasters. The input is the normalized data processed in step 1. At this point, the server utilizes the AI ​​model to perform analysis to calculate the optimal route for search activities. The output is a dataset containing recommended routes for the search. This allows the server to identify areas where search efforts should focus.

[0078] Step 3:

[0079] The server generates control signals for each miniature autonomous device based on the analysis results. The input is the search route information generated in step 2. Based on this, the server determines a specific action plan for each device and forms control signals as data packets. The output is the control signals to be sent to the terminal and the miniature autonomous devices.

[0080] Step 4:

[0081] The terminal receives control signals sent from the server and displays them on the user interface. The input is the control signals from the server. The terminal analyzes and visualizes the received signals, making it easy for the user to understand the situation. The output is an intuitive GUI display of progress and control instructions.

[0082] Step 5:

[0083] The user monitors the progress in real time via the terminal and makes decisions regarding the exploration's progress. Input is the visualized information displayed by the terminal. Based on this information, the user can change the direction of the exploration or give additional instructions. Output is feedback signals as a result of the user's commands and adjustments.

[0084] Step 6:

[0085] If necessary, the user instructs the server to reset the search route or resend control signals, and the server performs the corresponding update processing. The inputs are feedback signals and new field information. The server uses these inputs to re-execute the analysis process and, if necessary, generate a new route. The outputs are the updated control signals and search plan. In this way, the system continues to adapt flexibly to changes.

[0086] (Application Example 1)

[0087] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0088] In emergencies such as disasters, there is a need for technologies that can conduct rapid and accurate searches and efficiently support rescue operations for living beings. However, conventional search methods struggle to efficiently gather information and make appropriate decisions continuously in narrow and complex spaces that are difficult for humans to enter directly. Furthermore, the lack of systems that enable real-time location tracking and instruction changes hinders more effective search activities.

[0089] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0090] In this invention, the server includes means for generating control commands to search for living organisms at a specific site using multiple mobile autonomous devices controlled by a generated numerical analysis model; means for monitoring the progress of the search and adjusting instructions via a user interface; means for tracking the current position and operating status of each mobile autonomous device in real time; means for inputting emergency response commands according to the situation via voice; and means for dynamically calculating the optimal route based on received environmental data. This enables rapid and accurate searching in confined and complex spaces, and realizes efficient rescue of living organisms.

[0091] A "numerical analysis model" is a computational method designed to analyze data, extract patterns and trends, and perform a specific task.

[0092] A "mobile autonomous device" is a device that can make decisions and act on its own, and is a device that enables movement to explore a specific location.

[0093] A "control command" refers to an instruction issued to control the operation of equipment according to a specific purpose.

[0094] A "user interface" is the point of contact that enables the exchange of information between a system and a user, and it reflects the user's intentions through operation and display.

[0095] "Exploration progress" refers to status information indicating the stage of the exploration activity, and is data that can be monitored in real time.

[0096] An "emergency response order" defines an order that should be taken immediately in response to an emergency.

[0097] The "optimal route" is the chosen travel path that efficiently reaches a specific destination, and it is derived through calculation.

[0098] This invention is a system for enabling rapid and accurate searching in disaster areas and other similar situations. This system controls multiple mobile autonomous devices, enabling efficient search activities in confined spaces.

[0099] The server uses the generated numerical analysis model to analyze received environmental data and dynamically calculate the optimal search route for a specific site. The server also generates control commands and transmits them to mobile autonomous devices. In this process, the server uses cloud data streaming services such as Google FI® rebase and AWS® IoT to achieve real-time data processing.

[0100] The terminal is provided to the user, visually displaying the progress of the exploration via a user interface and tracking the current location and operational status of each mobile autonomous device in real time. This interface uses cross-platform app development frameworks such as React Native and Flutter® to display a visually easy-to-understand map. In addition, a voice input function is implemented using API.AI and Dialogflow, allowing users to input emergency response commands according to the situation.

[0101] The user monitors the progress of the search through their device, inputs commands via voice or manual, and recalculates the search route as needed. For example, when searching inside a building that has collapsed due to an earthquake, the user can input the following prompt into the AI ​​model to support the search: "Generate a route to search for survivors at the site of the collapsed Tremor Mansion."

[0102] As a concrete example, a user carrying a terminal enters a facility that has collapsed due to an earthquake and operates a mobile autonomous device according to a search route indicated by a server. This device updates the route in real time according to the data on site, enters even narrow gaps to collect information, and transmits the search results to the server. This allows for efficient and safe exploration.

[0103] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0104] Step 1:

[0105] The server receives environmental data (sensor data and map data) from disaster sites via cloud services. Inputs are data from field sensors and geographic information systems. The server preprocesses this data, removing noise and normalizing it to convert it into a format easily processed by the model. The output is clean, analyzable data.

[0106] Step 2:

[0107] The server runs a numerical analysis model using pre-processed data. This model utilizes a generative AI model to plan the optimal search route based on past disaster cases and site characteristics. The input is clean, analyzable data, and based on this, Kalman filters and path planning algorithms are used. The output is command data for the search route.

[0108] Step 3:

[0109] The server transmits the search route command data as a control signal to the mobile autonomous device. The input is the search route command data, which is converted into a communication protocol that the device can understand. In terms of specific operation, remote instructions are given using a communication module. The output is a control signal to the device.

[0110] Step 4:

[0111] The terminal displays the progress of the search route sent from the server in real time on the user interface. The input is the current status information of the device. The terminal updates a dynamic map using the received data, visually indicating the progress. Specifically, the UI is rendered using React Native or similar. The output is the visually displayed progress information.

[0112] Step 5:

[0113] The user directs the search route and emergency response via the terminal using voice commands or direct input. Input consists of the user's voice commands or touch operation data. The terminal analyzes this data and sends commands to the server as needed. Speech recognition using API.AI or Dialogflow is employed. Output is the analyzed user command data.

[0114] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0115] This invention is an advanced system that utilizes artificial intelligence to efficiently search for people in disaster areas. The system consists of multiple miniature autonomous devices and, by incorporating an emotion engine, can recognize the user's emotions and enhance the effectiveness of search activities.

[0116] Specifically, the server receives map data from the disaster site and analyzes it to identify the search area. Next, it generates a search route and sends control signals to each miniature autonomous device. These control signals enable the devices to adapt to confined spaces with biomimetic, flexible movements.

[0117] The terminal is a user-operated device that visually displays information sent from the server. Furthermore, the terminal is equipped with an emotion engine that recognizes the user's emotional state in real time. For example, if the user's stress level is high, it can automatically suggest taking a break or provide operational support. In addition, the interface's color scheme and operation patterns dynamically change according to the user's emotions, helping the user operate the system in the most optimal state.

[0118] The user has the authority to use this system to monitor exploration activities and adjust exploration commands as needed. Because the user can monitor the operation status of the miniature autonomous device via a terminal, rapid decision-making is possible based on the situation on site. The emotion engine analyzes the user's psychological state and helps adjust the human interface to reduce unnecessary stress.

[0119] For example, in the event of a building collapse caused by an earthquake, this system will quickly provide a route for searching for survivors and assist the user in their actions. In situations where the user's tension is high, the terminal display will be adjusted to be easier to understand, and only the most essential information will be highlighted to avoid information overload.

[0120] Thus, the present invention is a system that utilizes user emotion recognition to provide superior functionality in terms of both the efficiency of exploration activities and the reduction of the user's psychological burden.

[0121] The following describes the processing flow.

[0122] Step 1:

[0123] The server receives map data and real-time sensor information from the disaster site. Based on this, the generated artificial intelligence analyzes the data and identifies areas that should be prioritized for search.

[0124] Step 2:

[0125] The server constructs the optimal search route from the analysis results and generates and transmits control signals suitable for each micro-autonomous device. This enables the devices to perform biomimetic, flexible movements.

[0126] Step 3:

[0127] The terminal receives control signals from the server and displays the search route and the device's current location on the user interface. The terminal allows for real-time monitoring of its operating status.

[0128] Step 4:

[0129] The device's built-in emotion engine recognizes the user's emotional state in real time. For example, it measures stress levels through facial expressions and voice analysis.

[0130] Step 5:

[0131] Users can check the progress of their exploration through their device and adjust instructions as needed. Based on the emotion engine's judgment, if the user's stress level is high, suggestions for breaks or operational assistance will be automatically offered.

[0132] Step 6:

[0133] The device dynamically adjusts its operating interface according to the user's emotional state. For example, it might change the color scheme to a more subdued one or highlight necessary information.

[0134] Step 7:

[0135] The server continuously collects feedback from each autonomous device, evaluating exploration results and optimizing commands. This improves the efficiency of exploration and allows for timely strategic adjustments.

[0136] Step 8:

[0137] The latest progress information and analysis results are provided to the user via the terminal, and the user then issues further search commands based on this information. The entire system reduces the burden on the user while enabling effective life-saving efforts.

[0138] (Example 2)

[0139] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0140] Searching for lost lives at disaster sites is a race against time, requiring rapid and efficient operations. Furthermore, it's essential to reduce the burden and psychological stress on operators involved in the search. However, current technology is insufficient for optimizing search routes and addressing users' emotional states, making immediate response at the scene and reducing operator burden challenging.

[0141] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0142] In this invention, the server includes means for generating control signals for searching for human lives at a disaster site using multiple autonomous machines controlled by generated artificial intelligence; means for monitoring the progress of the search and adjusting commands via a user interface; and means for recognizing the user's emotional state in real time and dynamically adjusting the content displayed on the user interface. This makes it possible to increase the efficiency of the search and reduce the psychological burden on the operator.

[0143] "Generated artificial intelligence" refers to a machine learning model that possesses optimized algorithms necessary to achieve a specific objective and has the ability to make situation-appropriate decisions.

[0144] An "autonomous machine" is a device that can perform operations independently with minimal human intervention, and by adopting a biomimetic structure, it can move flexibly and adaptively.

[0145] A "disaster site" is a specific location in an area that has suffered damage due to a natural or man-made disaster, where human lives and supplies are at risk.

[0146] A "control signal" is command data that is sent to an autonomous machine to cause it to act according to a desired operation or route.

[0147] A "user interface" is a means of interaction that allows a user to interact with a system, visually confirm system information, and perform operations.

[0148] "Biomimic structures" are design forms that mimic the natural movements and functions of living organisms to give machines flexibility and adaptability.

[0149] "Map data" refers to a collection of digital or analog information that includes geographical information and provides detailed location information for a specific area.

[0150] "Emotional state" refers to the psychological and emotional state of a user, encompassing a variety of psychological indicators including stress levels and concentration levels.

[0151] This invention is a system that utilizes generated artificial intelligence to streamline the search for people at disaster sites. This system is realized through three main elements: a server, terminals, and users.

[0152] The server receives and analyzes map data acquired from disaster sites. The hardware used includes a server computer with high processing power, and map analysis software and a generative AI model are employed for the analysis. The AI ​​model optimizes the search route based on the received map data. The analyzed route information is generated as control signals for each autonomous machine, which then acts accordingly.

[0153] The terminal is a device used by operators, displaying information supplied from the server in real time through a user interface. The terminal is equipped with an emotion engine that recognizes the user's emotional state by analyzing their heart rate and facial expressions. This allows the terminal's interface display to dynamically adjust, reducing user stress and enabling efficient operation. For example, if the system detects that the user is in a state of tension, only the minimum necessary information will be highlighted on the interface.

[0154] Users have the authority to monitor the progress of search activities via their terminals and adjust commands to the server as needed. For example, when searching for survivors in a building collapsed due to an earthquake, the system will support the user by suggesting the optimal entry and exit routes. This system assists the user's judgment and enables quicker decision-making.

[0155] An example of a prompt message is: "Please explain in detail how to generate efficient search routes in a disaster area. Also, please explain how to dynamically change the device interface according to the user's emotional state." This system is equipped with advanced features to enhance the effectiveness of search operations and reduce the psychological burden on operators.

[0156] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0157] Step 1:

[0158] The server receives map data from the disaster site. The input is map data including GPS coordinates and satellite imagery. Based on this data, the server uses map analysis software to identify search areas. As part of data processing, it analyzes obstacle placement and priority areas for searching for people, and evaluates the risk level and accessibility of specific areas. The output is foundational information for generating search routes.

[0159] Step 2:

[0160] The server generates the optimal search route from map data analyzed using a generative AI model. The input is the foundational information obtained as a result of the analysis. The AI ​​model calculates an efficient route considering terrain characteristics and the location of obstacles. As part of the data calculation, it simulates multiple search routes and selects the shortest and safest route from among them. The output is the specific route data of the control signals sent to each autonomous machine.

[0161] Step 3:

[0162] The server transmits control signals to each autonomous machine based on the generated search route. The input is optimized route data. The device receives this signal and responds with biomimetic, flexible movements. As part of data processing, the operation instructions are converted into machine language and supplied to the device as specific operation commands. The output is a specific operation pattern that allows the autonomous machine to operate even in confined spaces.

[0163] Step 4:

[0164] The terminal visually provides information sent from the server through a user interface. Input consists of search information and control signal data from the server. The terminal organizes this information and displays it in an easy-to-understand format for the user. Specifically, the screen displays a map and search status that are updated in real time. Output is search data displayed visually in a user-friendly format.

[0165] Step 5:

[0166] The device uses an emotion engine to recognize the user's emotional state in real time. Inputs include biometric information such as the user's heart rate and facial expressions. The emotion engine analyzes this information to determine the user's stress level and concentration level. As a data calculation, it dynamically adjusts the user interface based on the emotional state. The output is an interface display optimized for the user's psychological state.

[0167] Step 6:

[0168] The user monitors their exploration activity based on the information displayed on the terminal and issues commands to the server as needed. Input consists of visual information and navigation instructions provided by the terminal. The user analyzes the information and sends commands to the server to prioritize exploring specific areas. Specifically, they adjust their exploration plan using touch input and voice commands. Output is the new exploration command after the operation.

[0169] (Application Example 2)

[0170] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0171] Efficient search and work support are necessary in work environments such as logistics centers and collapse sites. However, conventional autonomous devices and work support systems have shortcomings in terms of dynamic interface adjustments that take into account the psychological state of workers and insufficient stress reduction measures. As a result, problems arise such as decreased work efficiency and accuracy, and increased mental burden on workers.

[0172] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0173] In this invention, the server includes means for generating control signals for searching for human life or objects using multiple miniature autonomous devices controlled by generated artificial intelligence; means for monitoring the progress of the search or work and adjusting commands via a user interface; means for tracking the current position and operating status of each autonomous device; means for sensing the user's psychological state and dynamically adjusting the interface between visual information and work support; and means for providing operational support and break suggestions based on the worker's emotional state. This enables efficient search and work support and reduces the mental burden on the worker.

[0174] Artificial intelligence is a computer program that has the ability to analyze data and make decisions and learn based on the situation.

[0175] A "miniature autonomous device" is a small, independently operating mechanical device designed to function according to a specific purpose.

[0176] A "control signal" is a command transmitted to adjust the operation and functions of an autonomous device.

[0177] A "user interface" is a means that allows users to interact with digital systems and devices.

[0178] "Progress status" refers to information that indicates the current stage and degree of completion of a particular process or task.

[0179] "Adjusting instructions" is the process of modifying or optimizing instructions according to the situation in order to achieve the objective.

[0180] "Emotional state" is a concept that refers to an individual's internal psychological and emotional condition.

[0181] "Dynamic interface adjustment" is a technique that optimizes the display and operation methods in real time according to the user's situation and environment.

[0182] "Work support" refers to actions and systems designed to help workers perform their tasks efficiently and effectively.

[0183] A "suggestion for a break" is an instruction that encourages workers to take a break, taking into account their fatigue level and stress levels.

[0184] The server uses the generated artificial intelligence to create control signals for miniature autonomous devices designed to streamline operations within the logistics center. These control signals analyze the logistics center's map data, acquire the location information of specific products, and instruct the devices to move along the optimal route. The server also considers environmental data and dynamically adjusts the route as needed.

[0185] The terminal functions as smart glasses worn by the worker, providing visual information transmitted from the server. The terminal incorporates an emotion engine that analyzes the worker's psychological state in real time. The emotion engine detects the worker's stress and fatigue levels and adjusts the display interface accordingly to help maximize work efficiency. For example, it can display suggestions for breaks or instructions to adjust the work pace.

[0186] Users have the authority to monitor the overall work progress via a terminal and adjust instructions as needed. Using this system, users can operate the system to ensure work is carried out at a sustainable pace with minimal wear and tear, providing an optimal working environment. In this way, the system enables effective logistics center management and improved worker capabilities.

[0187] As a concrete example, when a worker at a logistics center begins to feel tired, the smart glasses can display a message such as, "You seem tired. Let's try stretching a little," allowing for health management while maintaining work efficiency. An example of a prompt message from the generated AI model in this invention is, "Please propose a method for dealing with worker stress using an emotion engine."

[0188] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0189] Step 1:

[0190] The server receives map data and picking lists for the logistics center in real time. It analyzes this input data to generate the optimal picking route for each product. The server retrieves the location information of the products from the database and uses this to calculate the most efficient travel route.

[0191] Step 2:

[0192] The server sends a control signal to the miniature autonomous device based on the picking route generated in step 1. The control signal includes specific movement instructions indicating which path the device should take. The device receives this signal and automatically moves along the specified route.

[0193] Step 3:

[0194] The terminal functions as smart glasses for the worker, visually displaying the location information of items and the picking route transmitted from the server to the worker. At the same time, specific instructions such as "Move to the location of item A after this item" are displayed to the worker.

[0195] Step 4:

[0196] The emotion engine installed in the terminal acquires biometric data from the worker (e.g., heart rate, skin potential) and analyzes the worker's psychological state in real time. Based on this input data, data processing is performed to evaluate stress and fatigue levels.

[0197] Step 5:

[0198] Based on the emotional data analyzed in step 4, the terminal adjusts the interface according to the worker's psychological state. For example, if the worker is fatigued, the interface is simplified to highlight only important information and a break suggestion is added.

[0199] Step 6:

[0200] Based on the terminal display, the user monitors the progress of the task and requests changes to the search route or control signals from the server as needed. The changes requested by the user are sent to the server and reflected in the operation of the autonomous device in the next step.

[0201] Step 7:

[0202] The server receives a command change request from the user and recalculates the path of the miniature autonomous device based on it. It generates a new control signal and resends the readjusted instructions to the autonomous device. This allows for flexible responses depending on the situation.

[0203] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0204] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0205] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0206] [Second Embodiment]

[0207] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0208] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0209] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0210] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0211] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0212] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0213] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0214] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0215] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0216] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0217] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0218] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0219] This invention relates to a system for supporting search and rescue operations at disaster sites using artificial intelligence that has been generated. Specifically, it can simultaneously control multiple miniature autonomous devices and conduct rapid searches even in confined spaces that are difficult for humans to enter.

[0220] First, the server receives map data and sensor information from the disaster site and analyzes it to generate the optimal search route. At this time, the generated artificial intelligence considers past data and the characteristics of the site to identify important search points. Based on this information, the server generates control signals to instruct the actions of each miniature autonomous device.

[0221] The terminal is operated by the user conducting the rescue operation and has the function of receiving and visually displaying the generated control signals. The terminal also monitors the current location and operating status of each miniature autonomous device in real time, allowing the user to supervise and adjust the search operation as needed. This function enables the user to make rapid decisions in response to the situation they face.

[0222] Users manage the progress of the exploration through their terminal and adjust the exploration strategy as needed. For example, if an important discovery is made, the user can reroute the exploration route and redeploy more autonomous devices to a specific area.

[0223] A concrete example is the search for survivors inside collapsed buildings during natural disasters such as earthquakes. The server considers the changes in the building structure after the earthquake and plans the optimal search route for the miniature autonomous device. Through a terminal, the user can monitor the ongoing search, make manual adjustments as needed, and ultimately take swift and accurate measures to save many lives.

[0224] Thus, by combining the generated artificial intelligence with a miniature autonomous device, this invention efficiently supports search and rescue activities at disaster sites, which were difficult with conventional methods, and greatly expands the possibility of saving lives.

[0225] The following describes the processing flow.

[0226] Step 1:

[0227] The server receives map data of the disaster site and sensor information acquired in real time. This creates a data base based on the latest situation on the ground.

[0228] Step 2:

[0229] The server uses generated artificial intelligence to analyze the received map data and understand the structure of the disaster site. This identifies specific search points and danger areas, which are then mapped according to their importance.

[0230] Step 3:

[0231] The server determines the search route for the miniature autonomous device based on the identified search points. The device's functions and surrounding environment are considered in formulating the optimal path.

[0232] Step 4:

[0233] The server generates control signals for each miniature autonomous device, instructing its operation. These signals include routing settings and operating mode specifications.

[0234] Step 5:

[0235] The terminal receives control signals transmitted from the server and provides them to the rescue team as visual information. The screen displays the device's location and operating status in real time.

[0236] Step 6:

[0237] Users monitor the information displayed through their devices and adjust their search activities as needed. For example, they might reset their search route based on new information or deploy additional equipment to specific areas.

[0238] Step 7:

[0239] The server continuously collects data from each autonomous device and analyzes the exploration results and the status of the devices. This allows for updating the area assessment and optimizing commands for the next step.

[0240] Step 8:

[0241] The terminal provides users with the latest progress and analysis results, forming an information infrastructure to maximize the effectiveness of search and rescue operations. Based on this, users can make strategic decisions and improve the efficiency of rescue operations.

[0242] (Example 1)

[0243] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0244] Rapid and effective search and rescue operations are required at disaster sites, but conventional technologies have problems such as low search efficiency in areas difficult for humans to access and difficulty in real-time situation assessment and rapid response. This invention aims to solve these problems and increase the possibility of saving lives.

[0245] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0246] In this invention, the server includes means for analyzing generated data and calculating the optimal search route based on past disaster data; means for generating control signals for searching for targets at a disaster site using multiple small autonomous mechanisms controlled by the generated artificial intelligence; and means for monitoring the progress of the search and adjusting commands via a user interface. This enables rapid and effective search activities at disaster sites.

[0247] "Generated data" refers to information obtained from disaster sites and includes various source data such as sensor data and geographical information.

[0248] The "optimal search route" is a path calculated based on analyzed historical disaster data and on-site information to maximize the efficiency of search activities.

[0249] A "miniature autonomous mechanism" is a small mechanical device controlled by artificial intelligence, with a structure that allows it to enter confined spaces and operate autonomously.

[0250] A "control signal" is a signal generated to instruct a small autonomous mechanism on its operation, and it contains command information necessary for carrying out search activities.

[0251] A "user interface" is a screen or device that provides a means for the user to monitor their exploration activities and adjust commands.

[0252] "Progress status" refers to the current state of the exploration activity and indicates the extent to which the plan has been implemented.

[0253] The following system configuration and operation are provided as embodiments for carrying out this invention.

[0254] The server plays a role in integrating and managing geographic data acquired from disaster sites and information from sensors. Using this data, the server utilizes a generated AI model to perform analysis that takes into account past disaster data and site characteristics. This allows for the calculation of the optimal search route. The server requires computing hardware with powerful processing capabilities and data analysis functions, and the software used consists of a platform on which data analysis algorithms and AI models operate. A concrete example is real-time analysis using cloud computing services.

[0255] The terminal receives control signals from the server at the disaster site and presents the search status to the user through a visual interface. The terminal is a crucial tool for users to monitor search activities and adjust commands as needed. The user interface is designed to be intuitive and easy to use, and operates on tablets and dedicated monitors. The terminal is equipped with a communication module for real-time data exchange with the server.

[0256] The user uses a device to supervise search and rescue operations at the disaster site. The user manages the progress of the search based on real-time information and controls the search strategy as new information becomes available. For example, in a search within a building whose structure has collapsed due to an earthquake, the user might instruct areas to be searched intensively while avoiding dangerous zones.

[0257] Furthermore, an example of a prompt statement in this invention is, "Explain how to use the sensor data to allow the generating AI model to identify search points." This indicates how the AI ​​model processes the data and extracts important points.

[0258] Thus, the embodiment for carrying out the present invention provides a system that efficiently supports search and rescue activities at disaster sites and enables life-saving efforts.

[0259] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0260] Step 1:

[0261] The server receives real-time geographic and sensor data from disaster sites. Inputs are data acquired from various sensors and drones deployed at the site. The server integrates and organizes this data, preparing it for analysis. Outputs are normalized datasets used in subsequent analysis stages.

[0262] Step 2:

[0263] The server analyzes incoming data using a generative AI model and compares it against a database of past disasters. The input is the normalized data processed in step 1. At this point, the server utilizes the AI ​​model to perform analysis to calculate the optimal route for search activities. The output is a dataset containing recommended routes for the search. This allows the server to identify areas where search efforts should focus.

[0264] Step 3:

[0265] The server generates control signals for each miniature autonomous device based on the analysis results. The input is the search route information generated in step 2. Based on this, the server determines a specific action plan for each device and forms control signals as data packets. The output is the control signals to be sent to the terminal and the miniature autonomous devices.

[0266] Step 4:

[0267] The terminal receives control signals sent from the server and displays them on the user interface. The input is the control signals from the server. The terminal analyzes and visualizes the received signals, making it easy for the user to understand the situation. The output is an intuitive GUI display of progress and control instructions.

[0268] Step 5:

[0269] The user monitors the progress in real time via the terminal and makes decisions regarding the exploration's progress. Input is the visualized information displayed by the terminal. Based on this information, the user can change the direction of the exploration or give additional instructions. Output is feedback signals as a result of the user's commands and adjustments.

[0270] Step 6:

[0271] If necessary, the user instructs the server to reset the search route or resend control signals, and the server performs the corresponding update processing. The inputs are feedback signals and new field information. The server uses these inputs to re-execute the analysis process and, if necessary, generate a new route. The outputs are the updated control signals and search plan. In this way, the system continues to adapt flexibly to changes.

[0272] (Application Example 1)

[0273] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0274] In emergencies such as disasters, there is a need for technologies that can conduct rapid and accurate searches and efficiently support rescue operations for living beings. However, conventional search methods struggle to efficiently gather information and make appropriate decisions continuously in narrow and complex spaces that are difficult for humans to enter directly. Furthermore, the lack of systems that enable real-time location tracking and instruction changes hinders more effective search activities.

[0275] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0276] In this invention, the server includes means for generating control commands to search for living organisms at a specific site using multiple mobile autonomous devices controlled by a generated numerical analysis model; means for monitoring the progress of the search and adjusting instructions via a user interface; means for tracking the current position and operating status of each mobile autonomous device in real time; means for inputting emergency response commands according to the situation via voice; and means for dynamically calculating the optimal route based on received environmental data. This enables rapid and accurate searching in confined and complex spaces, and realizes efficient rescue of living organisms.

[0277] A "numerical analysis model" is a computational method designed to analyze data, extract patterns and trends, and perform a specific task.

[0278] A "mobile autonomous device" is a device that can make decisions and act on its own, and is a device that enables movement to explore a specific location.

[0279] A "control command" refers to an instruction issued to control the operation of equipment according to a specific purpose.

[0280] The "user interface" is a contact point that enables the exchange of information between the system and the user, and reflects the user's intention through operations and displays.

[0281] The "progress of search" is status information indicating at what stage the search activity is, and is data that can be monitored in real time.

[0282] The "emergency response command" defines an instruction that should be immediately executed in case of an emergency.

[0283] The "optimal route" is a passage route selected to efficiently reach a specific destination, and is derived by calculation.

[0284] The present invention is a system for realizing quick and accurate search at a disaster site or the like. This system controls a plurality of mobile autonomous devices and enables efficient search activities in a narrow space.

[0285] The server uses the generated numerical analysis model to analyze the received environmental data and dynamically calculate the optimal search route at a specific site. The server also has the role of generating control commands and sending commands to the mobile autonomous devices. At this time, the server uses cloud data streaming services such as Google (registered trademark) Firebase and AWS IoT to realize real-time processing of data.

[0286] The terminal is provided to the user, visually displays the progress of the search via the user interface, and tracks the current position and operating status of each mobile autonomous device in real time. A cross-platform app development framework such as React Native or Flutter is used for this interface, and a visually easy-to-understand map is displayed. Also, a voice input function is implemented using API.AI or Dialogflow so that the user can input an emergency response command according to the situation by voice.

[0287] The user monitors the progress of the search through their device, inputs commands via voice or manual, and recalculates the search route as needed. For example, when searching inside a building that has collapsed due to an earthquake, the user can input the following prompt into the AI ​​model to support the search: "Generate a route to search for survivors at the site of the collapsed Tremor Mansion."

[0288] As a concrete example, a user carrying a terminal enters a facility that has collapsed due to an earthquake and operates a mobile autonomous device according to a search route indicated by a server. This device updates the route in real time according to the data on site, enters even narrow gaps to collect information, and transmits the search results to the server. This allows for efficient and safe exploration.

[0289] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0290] Step 1:

[0291] The server receives environmental data (sensor data and map data) from disaster sites via cloud services. Inputs are data from field sensors and geographic information systems. The server preprocesses this data, removing noise and normalizing it to convert it into a format easily processed by the model. The output is clean, analyzable data.

[0292] Step 2:

[0293] The server runs a numerical analysis model using pre-processed data. This model utilizes a generative AI model to plan the optimal search route based on past disaster cases and site characteristics. The input is clean, analyzable data, and based on this, Kalman filters and path planning algorithms are used. The output is command data for the search route.

[0294] Step 3:

[0295] The server transmits the search route command data as a control signal to the mobile autonomous device. The input is the search route command data, which is converted into a communication protocol that the device can understand. In terms of specific operation, remote instructions are given using a communication module. The output is a control signal to the device.

[0296] Step 4:

[0297] The terminal displays the progress of the search route sent from the server in real time on the user interface. The input is the current status information of the device. The terminal updates a dynamic map using the received data, visually indicating the progress. Specifically, the UI is rendered using React Native or similar. The output is the visually displayed progress information.

[0298] Step 5:

[0299] The user directs the search route and emergency response via the terminal using voice commands or direct input. Input consists of the user's voice commands or touch operation data. The terminal analyzes this data and sends commands to the server as needed. Speech recognition using API.AI or Dialogflow is employed. Output is the analyzed user command data.

[0300] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0301] This invention is an advanced system that utilizes artificial intelligence to efficiently search for people in disaster areas. The system consists of multiple miniature autonomous devices and, by incorporating an emotion engine, can recognize the user's emotions and enhance the effectiveness of search activities.

[0302] Specifically, the server receives map data from the disaster site and analyzes it to identify the search area. Subsequently, it generates a search route and transmits a control signal to each micro autonomous device. This control signal enables the device to adapt to narrow spaces with biomimetic flexible movements.

[0303] The terminal is a device operated by the user and visually provides the information sent from the server. Furthermore, the terminal is equipped with an emotion engine that recognizes the user's emotional state in real time. For example, when the user has a high stress level, it can automatically propose breaks or provide operation assistance. Also, according to the user's emotions, the color scheme and operation pattern of the interface dynamically change to assist the user in operating the system in an optimal state.

[0304] The user has the authority to monitor the search activity using this system and adjust the search commands as needed. Since the user can monitor the operating status of the micro autonomous device via the terminal, quick decision-making is possible according to the on-site situation. The emotion engine analyzes the user's psychological state and assists in adjusting the human interface to reduce unnecessary stress.

[0305] For example, at the site of a building collapse due to an earthquake, this system quickly presents a route for searching for survivors and provides operation assistance to the user. In a situation where the user's sense of tension increases, the display of the terminal is adjusted to be clearer, and only the most necessary information is emphasized to avoid information overload.

[0306] Thus, the present invention is a system that utilizes the user's emotion recognition and provides excellent functions in both the efficiency of the search activity and the reduction of the user's psychological burden.

[0307] The following describes the processing flow.

[0308] Step 1:

[0309] The server receives map data and real-time sensor information from the disaster site. Based on this, the generated artificial intelligence analyzes the data and identifies areas that should be prioritized for search.

[0310] Step 2:

[0311] The server constructs the optimal search route from the analysis results and generates and transmits control signals suitable for each micro-autonomous device. This enables the devices to perform biomimetic, flexible movements.

[0312] Step 3:

[0313] The terminal receives control signals from the server and displays the search route and the device's current location on the user interface. The terminal allows for real-time monitoring of its operating status.

[0314] Step 4:

[0315] The device's built-in emotion engine recognizes the user's emotional state in real time. For example, it measures stress levels through facial expressions and voice analysis.

[0316] Step 5:

[0317] Users can check the progress of their exploration through their device and adjust instructions as needed. Based on the emotion engine's judgment, if the user's stress level is high, suggestions for breaks or operational assistance will be automatically offered.

[0318] Step 6:

[0319] The device dynamically adjusts its operating interface according to the user's emotional state. For example, it might change the color scheme to a more subdued one or highlight necessary information.

[0320] Step 7:

[0321] The server continuously collects feedback from each autonomous device, evaluating exploration results and optimizing commands. This improves the efficiency of exploration and allows for timely strategic adjustments.

[0322] Step 8:

[0323] The latest progress information and analysis results are provided to the user via the terminal, and the user then issues further search commands based on this information. The entire system reduces the burden on the user while enabling effective life-saving efforts.

[0324] (Example 2)

[0325] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0326] Searching for lost lives at disaster sites is a race against time, requiring rapid and efficient operations. Furthermore, it's essential to reduce the burden and psychological stress on operators involved in the search. However, current technology is insufficient for optimizing search routes and addressing users' emotional states, making immediate response at the scene and reducing operator burden challenging.

[0327] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0328] In this invention, the server includes means for generating control signals for searching for human lives at a disaster site using multiple autonomous machines controlled by generated artificial intelligence; means for monitoring the progress of the search and adjusting commands via a user interface; and means for recognizing the user's emotional state in real time and dynamically adjusting the content displayed on the user interface. This makes it possible to increase the efficiency of the search and reduce the psychological burden on the operator.

[0329] "Generated artificial intelligence" refers to a machine learning model that possesses optimized algorithms necessary to achieve a specific objective and has the ability to make situation-appropriate decisions.

[0330] An "autonomous machine" is a device that can perform operations independently with minimal human intervention, and by adopting a biomimetic structure, it can move flexibly and adaptively.

[0331] A "disaster site" is a specific location in an area that has suffered damage due to a natural or man-made disaster, where human lives and supplies are at risk.

[0332] A "control signal" is command data that is sent to an autonomous machine to cause it to act according to a desired operation or route.

[0333] A "user interface" is a means of interaction that allows a user to interact with a system, visually confirm system information, and perform operations.

[0334] "Biomimic structures" are design forms that mimic the natural movements and functions of living organisms to give machines flexibility and adaptability.

[0335] "Map data" refers to a collection of digital or analog information that includes geographical information and provides detailed location information for a specific area.

[0336] "Emotional state" refers to the psychological and emotional state of a user, encompassing a variety of psychological indicators including stress levels and concentration levels.

[0337] This invention is a system that utilizes generated artificial intelligence to streamline the search for people at disaster sites. This system is realized through three main elements: a server, terminals, and users.

[0338] The server receives and analyzes map data acquired from disaster sites. The hardware used includes a server computer with high processing power, and map analysis software and a generative AI model are employed for the analysis. The AI ​​model optimizes the search route based on the received map data. The analyzed route information is generated as control signals for each autonomous machine, which then acts accordingly.

[0339] The terminal is a device used by operators, displaying information supplied from the server in real time through a user interface. The terminal is equipped with an emotion engine that recognizes the user's emotional state by analyzing their heart rate and facial expressions. This allows the terminal's interface display to dynamically adjust, reducing user stress and enabling efficient operation. For example, if the system detects that the user is in a state of tension, only the minimum necessary information will be highlighted on the interface.

[0340] Users have the authority to monitor the progress of search activities via their terminals and adjust commands to the server as needed. For example, when searching for survivors in a building collapsed due to an earthquake, the system will support the user by suggesting the optimal entry and exit routes. This system assists the user's judgment and enables quicker decision-making.

[0341] An example of a prompt message is: "Please explain in detail how to generate efficient search routes in a disaster area. Also, please explain how to dynamically change the device interface according to the user's emotional state." This system is equipped with advanced features to enhance the effectiveness of search operations and reduce the psychological burden on operators.

[0342] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0343] Step 1:

[0344] The server receives map data from the disaster site. The input is map data including GPS coordinates and satellite imagery. Based on this data, the server uses map analysis software to identify search areas. As part of data processing, it analyzes obstacle placement and priority areas for searching for people, and evaluates the risk level and accessibility of specific areas. The output is foundational information for generating search routes.

[0345] Step 2:

[0346] The server generates the optimal search route from map data analyzed using a generative AI model. The input is the foundational information obtained as a result of the analysis. The AI ​​model calculates an efficient route considering terrain characteristics and the location of obstacles. As part of the data calculation, it simulates multiple search routes and selects the shortest and safest route from among them. The output is the specific route data of the control signals sent to each autonomous machine.

[0347] Step 3:

[0348] The server transmits control signals to each autonomous machine based on the generated search route. The input is optimized route data. The device receives this signal and responds with biomimetic, flexible movements. As part of data processing, the operation instructions are converted into machine language and supplied to the device as specific operation commands. The output is a specific operation pattern that allows the autonomous machine to operate even in confined spaces.

[0349] Step 4:

[0350] The terminal visually provides information sent from the server through a user interface. Input consists of search information and control signal data from the server. The terminal organizes this information and displays it in an easy-to-understand format for the user. Specifically, the screen displays a map and search status that are updated in real time. Output is search data displayed visually in a user-friendly format.

[0351] Step 5:

[0352] The device uses an emotion engine to recognize the user's emotional state in real time. Inputs include biometric information such as the user's heart rate and facial expressions. The emotion engine analyzes this information to determine the user's stress level and concentration level. As a data calculation, it dynamically adjusts the user interface based on the emotional state. The output is an interface display optimized for the user's psychological state.

[0353] Step 6:

[0354] The user monitors their exploration activity based on the information displayed on the terminal and issues commands to the server as needed. Input consists of visual information and navigation instructions provided by the terminal. The user analyzes the information and sends commands to the server to prioritize exploring specific areas. Specifically, they adjust their exploration plan using touch input and voice commands. Output is the new exploration command after the operation.

[0355] (Application Example 2)

[0356] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0357] Efficient search and work support are necessary in work environments such as logistics centers and collapse sites. However, conventional autonomous devices and work support systems have shortcomings in terms of dynamic interface adjustments that take into account the psychological state of workers and insufficient stress reduction measures. As a result, problems arise such as decreased work efficiency and accuracy, and increased mental burden on workers.

[0358] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0359] In this invention, the server includes means for generating control signals for searching for human life or objects using multiple miniature autonomous devices controlled by generated artificial intelligence; means for monitoring the progress of the search or work and adjusting commands via a user interface; means for tracking the current position and operating status of each autonomous device; means for sensing the user's psychological state and dynamically adjusting the interface between visual information and work support; and means for providing operational support and break suggestions based on the worker's emotional state. This enables efficient search and work support and reduces the mental burden on the worker.

[0360] Artificial intelligence is a computer program that has the ability to analyze data and make decisions and learn based on the situation.

[0361] A "miniature autonomous device" is a small, independently operating mechanical device designed to function according to a specific purpose.

[0362] A "control signal" is a command transmitted to adjust the operation and functions of an autonomous device.

[0363] A "user interface" is a means that allows users to interact with digital systems and devices.

[0364] "Progress status" refers to information that indicates the current stage and degree of completion of a particular process or task.

[0365] "Adjusting instructions" is the process of modifying or optimizing instructions according to the situation in order to achieve the objective.

[0366] "Emotional state" is a concept that refers to an individual's internal psychological and emotional condition.

[0367] "Dynamic interface adjustment" is a technique that optimizes the display and operation methods in real time according to the user's situation and environment.

[0368] "Work support" refers to actions and systems designed to help workers perform their tasks efficiently and effectively.

[0369] A "suggestion for a break" is an instruction that encourages workers to take a break, taking into account their fatigue level and stress levels.

[0370] The server uses the generated artificial intelligence to create control signals for miniature autonomous devices designed to streamline operations within the logistics center. These control signals analyze the logistics center's map data, acquire the location information of specific products, and instruct the devices to move along the optimal route. The server also considers environmental data and dynamically adjusts the route as needed.

[0371] The terminal functions as smart glasses worn by the worker, providing visual information transmitted from the server. The terminal incorporates an emotion engine that analyzes the worker's psychological state in real time. The emotion engine detects the worker's stress and fatigue levels and adjusts the display interface accordingly to help maximize work efficiency. For example, it can display suggestions for breaks or instructions to adjust the work pace.

[0372] Users have the authority to monitor the overall work progress via a terminal and adjust instructions as needed. Using this system, users can operate the system to ensure work is carried out at a sustainable pace with minimal wear and tear, providing an optimal working environment. In this way, the system enables effective logistics center management and improved worker capabilities.

[0373] As a concrete example, when a worker at a logistics center begins to feel tired, the smart glasses can display a message such as, "You seem tired. Let's try stretching a little," allowing for health management while maintaining work efficiency. An example of a prompt message from the generated AI model in this invention is, "Please propose a method for dealing with worker stress using an emotion engine."

[0374] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0375] Step 1:

[0376] The server receives map data and picking lists for the logistics center in real time. It analyzes this input data to generate the optimal picking route for each product. The server retrieves the location information of the products from the database and uses this to calculate the most efficient travel route.

[0377] Step 2:

[0378] The server sends a control signal to the miniature autonomous device based on the picking route generated in step 1. The control signal includes specific movement instructions indicating which path the device should take. The device receives this signal and automatically moves along the specified route.

[0379] Step 3:

[0380] The terminal functions as smart glasses for the worker, visually displaying the location information of items and the picking route transmitted from the server to the worker. At the same time, specific instructions such as "Move to the location of item A after this item" are displayed to the worker.

[0381] Step 4:

[0382] The emotion engine installed in the terminal acquires biometric data from the worker (e.g., heart rate, skin potential) and analyzes the worker's psychological state in real time. Based on this input data, data processing is performed to evaluate stress and fatigue levels.

[0383] Step 5:

[0384] Based on the emotional data analyzed in step 4, the terminal adjusts the interface according to the worker's psychological state. For example, if the worker is fatigued, the interface is simplified to highlight only important information and a break suggestion is added.

[0385] Step 6:

[0386] Based on the terminal display, the user monitors the progress of the task and requests changes to the search route or control signals from the server as needed. The changes requested by the user are sent to the server and reflected in the operation of the autonomous device in the next step.

[0387] Step 7:

[0388] The server receives a command change request from the user and recalculates the path of the miniature autonomous device based on it. It generates a new control signal and resends the readjusted instructions to the autonomous device. This allows for flexible responses depending on the situation.

[0389] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0390] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0391] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0392] [Third Embodiment]

[0393] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0394] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0395] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0396] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0397] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0398] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0399] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0400] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0401] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0402] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0403] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0404] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0405] This invention relates to a system for supporting search and rescue operations at disaster sites using artificial intelligence that has been generated. Specifically, it can simultaneously control multiple miniature autonomous devices and conduct rapid searches even in confined spaces that are difficult for humans to enter.

[0406] First, the server receives map data and sensor information from the disaster site and analyzes it to generate the optimal search route. At this time, the generated artificial intelligence considers past data and the characteristics of the site to identify important search points. Based on this information, the server generates control signals to instruct the actions of each miniature autonomous device.

[0407] The terminal is operated by the user conducting the rescue operation and has the function of receiving and visually displaying the generated control signals. The terminal also monitors the current location and operating status of each miniature autonomous device in real time, allowing the user to supervise and adjust the search operation as needed. This function enables the user to make rapid decisions in response to the situation they face.

[0408] Users manage the progress of the exploration through their terminal and adjust the exploration strategy as needed. For example, if an important discovery is made, the user can reroute the exploration route and redeploy more autonomous devices to a specific area.

[0409] A concrete example is the search for survivors inside collapsed buildings during natural disasters such as earthquakes. The server considers the changes in the building structure after the earthquake and plans the optimal search route for the miniature autonomous device. Through a terminal, the user can monitor the ongoing search, make manual adjustments as needed, and ultimately take swift and accurate measures to save many lives.

[0410] Thus, by combining the generated artificial intelligence with a miniature autonomous device, this invention efficiently supports search and rescue activities at disaster sites, which were difficult with conventional methods, and greatly expands the possibility of saving lives.

[0411] The following describes the processing flow.

[0412] Step 1:

[0413] The server receives map data of the disaster site and sensor information acquired in real time. This creates a data base based on the latest situation on the ground.

[0414] Step 2:

[0415] The server uses generated artificial intelligence to analyze the received map data and understand the structure of the disaster site. This identifies specific search points and danger areas, which are then mapped according to their importance.

[0416] Step 3:

[0417] The server determines the search route for the miniature autonomous device based on the identified search points. The device's functions and surrounding environment are considered in formulating the optimal path.

[0418] Step 4:

[0419] The server generates control signals for each miniature autonomous device, instructing its operation. These signals include routing settings and operating mode specifications.

[0420] Step 5:

[0421] The terminal receives control signals transmitted from the server and provides them to the rescue team as visual information. The screen displays the device's location and operating status in real time.

[0422] Step 6:

[0423] Users monitor the information displayed through their devices and adjust their search activities as needed. For example, they might reset their search route based on new information or deploy additional equipment to specific areas.

[0424] Step 7:

[0425] The server continuously collects data from each autonomous device and analyzes the exploration results and the status of the devices. This allows for updating the area assessment and optimizing commands for the next step.

[0426] Step 8:

[0427] The terminal provides users with the latest progress and analysis results, forming an information infrastructure to maximize the effectiveness of search and rescue operations. Based on this, users can make strategic decisions and improve the efficiency of rescue operations.

[0428] (Example 1)

[0429] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0430] Rapid and effective search and rescue operations are required at disaster sites, but conventional technologies have problems such as low search efficiency in areas difficult for humans to access and difficulty in real-time situation assessment and rapid response. This invention aims to solve these problems and increase the possibility of saving lives.

[0431] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0432] In this invention, the server includes means for analyzing generated data and calculating the optimal search route based on past disaster data; means for generating control signals for searching for targets at a disaster site using multiple small autonomous mechanisms controlled by the generated artificial intelligence; and means for monitoring the progress of the search and adjusting commands via a user interface. This enables rapid and effective search activities at disaster sites.

[0433] "Generated data" refers to information obtained from disaster sites and includes various source data such as sensor data and geographical information.

[0434] The "optimal search route" is a path calculated based on analyzed historical disaster data and on-site information to maximize the efficiency of search activities.

[0435] A "miniature autonomous mechanism" is a small mechanical device controlled by artificial intelligence, with a structure that allows it to enter confined spaces and operate autonomously.

[0436] A "control signal" is a signal generated to instruct a small autonomous mechanism on its operation, and it contains command information necessary for carrying out search activities.

[0437] A "user interface" is a screen or device that provides a means for the user to monitor their exploration activities and adjust commands.

[0438] "Progress status" refers to the current state of the exploration activity and indicates the extent to which the plan has been implemented.

[0439] The following system configuration and operation are provided as embodiments for carrying out this invention.

[0440] The server plays a role in integrating and managing geographic data acquired from disaster sites and information from sensors. Using this data, the server utilizes a generated AI model to perform analysis that takes into account past disaster data and site characteristics. This allows for the calculation of the optimal search route. The server requires computing hardware with powerful processing capabilities and data analysis functions, and the software used consists of a platform on which data analysis algorithms and AI models operate. A concrete example is real-time analysis using cloud computing services.

[0441] The terminal receives control signals from the server at the disaster site and presents the search status to the user through a visual interface. The terminal is a crucial tool for users to monitor search activities and adjust commands as needed. The user interface is designed to be intuitive and easy to use, and operates on tablets and dedicated monitors. The terminal is equipped with a communication module for real-time data exchange with the server.

[0442] The user uses a device to supervise search and rescue operations at the disaster site. The user manages the progress of the search based on real-time information and controls the search strategy as new information becomes available. For example, in a search within a building whose structure has collapsed due to an earthquake, the user might instruct areas to be searched intensively while avoiding dangerous zones.

[0443] Furthermore, an example of a prompt statement in this invention is, "Explain how to use the sensor data to allow the generating AI model to identify search points." This indicates how the AI ​​model processes the data and extracts important points.

[0444] Thus, the embodiment for carrying out the present invention provides a system that efficiently supports search and rescue activities at disaster sites and enables life-saving efforts.

[0445] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0446] Step 1:

[0447] The server receives real-time geographic and sensor data from disaster sites. Inputs are data acquired from various sensors and drones deployed at the site. The server integrates and organizes this data, preparing it for analysis. Outputs are normalized datasets used in subsequent analysis stages.

[0448] Step 2:

[0449] The server analyzes incoming data using a generative AI model and compares it against a database of past disasters. The input is the normalized data processed in step 1. At this point, the server utilizes the AI ​​model to perform analysis to calculate the optimal route for search activities. The output is a dataset containing recommended routes for the search. This allows the server to identify areas where search efforts should focus.

[0450] Step 3:

[0451] The server generates control signals for each miniature autonomous device based on the analysis results. The input is the search route information generated in step 2. Based on this, the server determines a specific action plan for each device and forms control signals as data packets. The output is the control signals to be sent to the terminal and the miniature autonomous devices.

[0452] Step 4:

[0453] The terminal receives control signals sent from the server and displays them on the user interface. The input is the control signals from the server. The terminal analyzes and visualizes the received signals, making it easy for the user to understand the situation. The output is an intuitive GUI display of progress and control instructions.

[0454] Step 5:

[0455] The user monitors the progress in real time via the terminal and makes decisions regarding the exploration's progress. Input is the visualized information displayed by the terminal. Based on this information, the user can change the direction of the exploration or give additional instructions. Output is feedback signals as a result of the user's commands and adjustments.

[0456] Step 6:

[0457] If necessary, the user instructs the server to reset the search route or resend control signals, and the server performs the corresponding update processing. The inputs are feedback signals and new field information. The server uses these inputs to re-execute the analysis process and, if necessary, generate a new route. The outputs are the updated control signals and search plan. In this way, the system continues to adapt flexibly to changes.

[0458] (Application Example 1)

[0459] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0460] In emergencies such as disasters, there is a need for technologies that can conduct rapid and accurate searches and efficiently support rescue operations for living beings. However, conventional search methods struggle to efficiently gather information and make appropriate decisions continuously in narrow and complex spaces that are difficult for humans to enter directly. Furthermore, the lack of systems that enable real-time location tracking and instruction changes hinders more effective search activities.

[0461] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0462] In this invention, the server includes means for generating control commands to search for living organisms at a specific site using multiple mobile autonomous devices controlled by a generated numerical analysis model; means for monitoring the progress of the search and adjusting instructions via a user interface; means for tracking the current position and operating status of each mobile autonomous device in real time; means for inputting emergency response commands according to the situation via voice; and means for dynamically calculating the optimal route based on received environmental data. This enables rapid and accurate searching in confined and complex spaces, and realizes efficient rescue of living organisms.

[0463] A "numerical analysis model" is a computational method designed to analyze data, extract patterns and trends, and perform a specific task.

[0464] A "mobile autonomous device" is a device that can make decisions and act on its own, and is a device that enables movement to explore a specific location.

[0465] A "control command" refers to an instruction issued to control the operation of equipment according to a specific purpose.

[0466] A "user interface" is the point of contact that enables the exchange of information between a system and a user, and it reflects the user's intentions through operation and display.

[0467] "Exploration progress" refers to status information indicating the stage of the exploration activity, and is data that can be monitored in real time.

[0468] An "emergency response order" defines an order that should be taken immediately in response to an emergency.

[0469] The "optimal route" is the chosen travel path that efficiently reaches a specific destination, and it is derived through calculation.

[0470] This invention is a system for enabling rapid and accurate searching in disaster areas and other similar situations. This system controls multiple mobile autonomous devices, enabling efficient search activities in confined spaces.

[0471] The server uses the generated numerical analysis model to analyze the received environmental data and dynamically calculate the optimal search route for a specific site. The server also generates control commands and transmits them to mobile autonomous devices. In this process, the server uses cloud data streaming services such as Google Firebase and AWS IoT to achieve real-time data processing.

[0472] The terminal is provided to the user, visually displaying the progress of the exploration via a user interface and tracking the current location and operational status of each mobile autonomous device in real time. This interface uses cross-platform app development frameworks such as React Native and Flutter, and displays a visually easy-to-understand map. In addition, voice input functionality is implemented using API.AI and Dialogflow, allowing users to input emergency response commands according to the situation.

[0473] The user monitors the progress of the search through their device, inputs commands via voice or manual, and recalculates the search route as needed. For example, when searching inside a building that has collapsed due to an earthquake, the user can input the following prompt into the AI ​​model to support the search: "Generate a route to search for survivors at the site of the collapsed Tremor Mansion."

[0474] As a concrete example, a user carrying a terminal enters a facility that has collapsed due to an earthquake and operates a mobile autonomous device according to a search route indicated by a server. This device updates the route in real time according to the data on site, enters even narrow gaps to collect information, and transmits the search results to the server. This allows for efficient and safe exploration.

[0475] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0476] Step 1:

[0477] The server receives environmental data (sensor data and map data) from disaster sites via cloud services. Inputs are data from field sensors and geographic information systems. The server preprocesses this data, removing noise and normalizing it to convert it into a format easily processed by the model. The output is clean, analyzable data.

[0478] Step 2:

[0479] The server runs a numerical analysis model using pre-processed data. This model utilizes a generative AI model to plan the optimal search route based on past disaster cases and site characteristics. The input is clean, analyzable data, and based on this, Kalman filters and path planning algorithms are used. The output is command data for the search route.

[0480] Step 3:

[0481] The server transmits the search route command data as a control signal to the mobile autonomous device. The input is the search route command data, which is converted into a communication protocol that the device can understand. In terms of specific operation, remote instructions are given using a communication module. The output is a control signal to the device.

[0482] Step 4:

[0483] The terminal displays the progress of the search route sent from the server in real time on the user interface. The input is the current status information of the device. The terminal updates a dynamic map using the received data, visually indicating the progress. Specifically, the UI is rendered using React Native or similar. The output is the visually displayed progress information.

[0484] Step 5:

[0485] The user directs the search route and emergency response via the terminal using voice commands or direct input. Input consists of the user's voice commands or touch operation data. The terminal analyzes this data and sends commands to the server as needed. Speech recognition using API.AI or Dialogflow is employed. Output is the analyzed user command data.

[0486] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0487] This invention is an advanced system that utilizes artificial intelligence to efficiently search for people in disaster areas. The system consists of multiple miniature autonomous devices and, by incorporating an emotion engine, can recognize the user's emotions and enhance the effectiveness of search activities.

[0488] Specifically, the server receives map data from the disaster site and analyzes it to identify the search area. Next, it generates a search route and sends control signals to each miniature autonomous device. These control signals enable the devices to adapt to confined spaces with biomimetic, flexible movements.

[0489] The terminal is a user-operated device that visually displays information sent from the server. Furthermore, the terminal is equipped with an emotion engine that recognizes the user's emotional state in real time. For example, if the user's stress level is high, it can automatically suggest taking a break or provide operational support. In addition, the interface's color scheme and operation patterns dynamically change according to the user's emotions, helping the user operate the system in the most optimal state.

[0490] The user has the authority to use this system to monitor exploration activities and adjust exploration commands as needed. Because the user can monitor the operation status of the miniature autonomous device via a terminal, rapid decision-making is possible based on the situation on site. The emotion engine analyzes the user's psychological state and helps adjust the human interface to reduce unnecessary stress.

[0491] For example, in the event of a building collapse caused by an earthquake, this system will quickly provide a route for searching for survivors and assist the user in their actions. In situations where the user's tension is high, the terminal display will be adjusted to be easier to understand, and only the most essential information will be highlighted to avoid information overload.

[0492] Thus, the present invention is a system that utilizes user emotion recognition to provide superior functionality in terms of both the efficiency of exploration activities and the reduction of the user's psychological burden.

[0493] The following describes the processing flow.

[0494] Step 1:

[0495] The server receives map data and real-time sensor information from the disaster site. Based on this, the generated artificial intelligence analyzes the data and identifies areas that should be prioritized for search.

[0496] Step 2:

[0497] The server constructs the optimal search route from the analysis results and generates and transmits control signals suitable for each micro-autonomous device. This enables the devices to perform biomimetic, flexible movements.

[0498] Step 3:

[0499] The terminal receives control signals from the server and displays the search route and the device's current location on the user interface. The terminal allows for real-time monitoring of its operating status.

[0500] Step 4:

[0501] The device's built-in emotion engine recognizes the user's emotional state in real time. For example, it measures stress levels through facial expressions and voice analysis.

[0502] Step 5:

[0503] Users can check the progress of their exploration through their device and adjust instructions as needed. Based on the emotion engine's judgment, if the user's stress level is high, suggestions for breaks or operational assistance will be automatically offered.

[0504] Step 6:

[0505] The device dynamically adjusts its operating interface according to the user's emotional state. For example, it might change the color scheme to a more subdued one or highlight necessary information.

[0506] Step 7:

[0507] The server continuously collects feedback from each autonomous device, evaluating exploration results and optimizing commands. This improves the efficiency of exploration and allows for timely strategic adjustments.

[0508] Step 8:

[0509] The latest progress information and analysis results are provided to the user via the terminal, and the user then issues further search commands based on this information. The entire system reduces the burden on the user while enabling effective life-saving efforts.

[0510] (Example 2)

[0511] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0512] Searching for lost lives at disaster sites is a race against time, requiring rapid and efficient operations. Furthermore, it's essential to reduce the burden and psychological stress on operators involved in the search. However, current technology is insufficient for optimizing search routes and addressing users' emotional states, making immediate response at the scene and reducing operator burden challenging.

[0513] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0514] In this invention, the server includes means for generating control signals for searching for human lives at a disaster site using multiple autonomous machines controlled by generated artificial intelligence; means for monitoring the progress of the search and adjusting commands via a user interface; and means for recognizing the user's emotional state in real time and dynamically adjusting the content displayed on the user interface. This makes it possible to increase the efficiency of the search and reduce the psychological burden on the operator.

[0515] "Generated artificial intelligence" refers to a machine learning model that possesses optimized algorithms necessary to achieve a specific objective and has the ability to make situation-appropriate decisions.

[0516] An "autonomous machine" is a device that can perform operations independently with minimal human intervention, and by adopting a biomimetic structure, it can move flexibly and adaptively.

[0517] A "disaster site" is a specific location in an area that has suffered damage due to a natural or man-made disaster, where human lives and supplies are at risk.

[0518] A "control signal" is command data that is sent to an autonomous machine to cause it to act according to a desired operation or route.

[0519] A "user interface" is a means of interaction that allows a user to interact with a system, visually confirm system information, and perform operations.

[0520] "Biomimic structures" are design forms that mimic the natural movements and functions of living organisms to give machines flexibility and adaptability.

[0521] "Map data" refers to a collection of digital or analog information that includes geographical information and provides detailed location information for a specific area.

[0522] "Emotional state" refers to the psychological and emotional state of a user, encompassing a variety of psychological indicators including stress levels and concentration levels.

[0523] This invention is a system that utilizes generated artificial intelligence to streamline the search for people at disaster sites. This system is realized through three main elements: a server, terminals, and users.

[0524] The server receives and analyzes map data acquired from disaster sites. The hardware used includes a server computer with high processing power, and map analysis software and a generative AI model are employed for the analysis. The AI ​​model optimizes the search route based on the received map data. The analyzed route information is generated as control signals for each autonomous machine, which then acts accordingly.

[0525] The terminal is a device used by operators, displaying information supplied from the server in real time through a user interface. The terminal is equipped with an emotion engine that recognizes the user's emotional state by analyzing their heart rate and facial expressions. This allows the terminal's interface display to dynamically adjust, reducing user stress and enabling efficient operation. For example, if the system detects that the user is in a state of tension, only the minimum necessary information will be highlighted on the interface.

[0526] Users have the authority to monitor the progress of search activities via their terminals and adjust commands to the server as needed. For example, when searching for survivors in a building collapsed due to an earthquake, the system will support the user by suggesting the optimal entry and exit routes. This system assists the user's judgment and enables quicker decision-making.

[0527] An example of a prompt message is: "Please explain in detail how to generate efficient search routes in a disaster area. Also, please explain how to dynamically change the device interface according to the user's emotional state." This system is equipped with advanced features to enhance the effectiveness of search operations and reduce the psychological burden on operators.

[0528] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0529] Step 1:

[0530] The server receives map data from the disaster site. The input is map data including GPS coordinates and satellite imagery. Based on this data, the server uses map analysis software to identify search areas. As part of data processing, it analyzes obstacle placement and priority areas for searching for people, and evaluates the risk level and accessibility of specific areas. The output is foundational information for generating search routes.

[0531] Step 2:

[0532] The server generates the optimal search route from map data analyzed using a generative AI model. The input is the foundational information obtained as a result of the analysis. The AI ​​model calculates an efficient route considering terrain characteristics and the location of obstacles. As part of the data calculation, it simulates multiple search routes and selects the shortest and safest route from among them. The output is the specific route data of the control signals sent to each autonomous machine.

[0533] Step 3:

[0534] The server transmits control signals to each autonomous machine based on the generated search route. The input is optimized route data. The device receives this signal and responds with biomimetic, flexible movements. As part of data processing, the operation instructions are converted into machine language and supplied to the device as specific operation commands. The output is a specific operation pattern that allows the autonomous machine to operate even in confined spaces.

[0535] Step 4:

[0536] The terminal visually provides information sent from the server through a user interface. Input consists of search information and control signal data from the server. The terminal organizes this information and displays it in an easy-to-understand format for the user. Specifically, the screen displays a map and search status that are updated in real time. Output is search data displayed visually in a user-friendly format.

[0537] Step 5:

[0538] The device uses an emotion engine to recognize the user's emotional state in real time. Inputs include biometric information such as the user's heart rate and facial expressions. The emotion engine analyzes this information to determine the user's stress level and concentration level. As a data calculation, it dynamically adjusts the user interface based on the emotional state. The output is an interface display optimized for the user's psychological state.

[0539] Step 6:

[0540] The user monitors their exploration activity based on the information displayed on the terminal and issues commands to the server as needed. Input consists of visual information and navigation instructions provided by the terminal. The user analyzes the information and sends commands to the server to prioritize exploring specific areas. Specifically, they adjust their exploration plan using touch input and voice commands. Output is the new exploration command after the operation.

[0541] (Application Example 2)

[0542] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0543] Efficient search and work support are necessary in work environments such as logistics centers and collapse sites. However, conventional autonomous devices and work support systems have shortcomings in terms of dynamic interface adjustments that take into account the psychological state of workers and insufficient stress reduction measures. As a result, problems arise such as decreased work efficiency and accuracy, and increased mental burden on workers.

[0544] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0545] In this invention, the server includes means for generating control signals for searching for human life or objects using multiple miniature autonomous devices controlled by generated artificial intelligence; means for monitoring the progress of the search or work and adjusting commands via a user interface; means for tracking the current position and operating status of each autonomous device; means for sensing the user's psychological state and dynamically adjusting the interface between visual information and work support; and means for providing operational support and break suggestions based on the worker's emotional state. This enables efficient search and work support and reduces the mental burden on the worker.

[0546] Artificial intelligence is a computer program that has the ability to analyze data and make decisions and learn based on the situation.

[0547] A "miniature autonomous device" is a small, independently operating mechanical device designed to function according to a specific purpose.

[0548] A "control signal" is a command transmitted to adjust the operation and functions of an autonomous device.

[0549] A "user interface" is a means that allows users to interact with digital systems and devices.

[0550] "Progress status" refers to information that indicates the current stage and degree of completion of a particular process or task.

[0551] "Adjusting instructions" is the process of modifying or optimizing instructions according to the situation in order to achieve the objective.

[0552] "Emotional state" is a concept that refers to an individual's internal psychological and emotional condition.

[0553] "Dynamic interface adjustment" is a technique that optimizes the display and operation methods in real time according to the user's situation and environment.

[0554] "Work support" refers to actions and systems designed to help workers perform their tasks efficiently and effectively.

[0555] A "suggestion for a break" is an instruction that encourages workers to take a break, taking into account their fatigue level and stress levels.

[0556] The server uses the generated artificial intelligence to create control signals for miniature autonomous devices designed to streamline operations within the logistics center. These control signals analyze the logistics center's map data, acquire the location information of specific products, and instruct the devices to move along the optimal route. The server also considers environmental data and dynamically adjusts the route as needed.

[0557] The terminal functions as smart glasses worn by the worker, providing visual information transmitted from the server. The terminal incorporates an emotion engine that analyzes the worker's psychological state in real time. The emotion engine detects the worker's stress and fatigue levels and adjusts the display interface accordingly to help maximize work efficiency. For example, it can display suggestions for breaks or instructions to adjust the work pace.

[0558] Users have the authority to monitor the overall work progress via a terminal and adjust instructions as needed. Using this system, users can operate the system to ensure work is carried out at a sustainable pace with minimal wear and tear, providing an optimal working environment. In this way, the system enables effective logistics center management and improved worker capabilities.

[0559] As a concrete example, when a worker at a logistics center begins to feel tired, the smart glasses can display a message such as, "You seem tired. Let's try stretching a little," allowing for health management while maintaining work efficiency. An example of a prompt message from the generated AI model in this invention is, "Please propose a method for dealing with worker stress using an emotion engine."

[0560] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0561] Step 1:

[0562] The server receives map data and picking lists for the logistics center in real time. It analyzes this input data to generate the optimal picking route for each product. The server retrieves the location information of the products from the database and uses this to calculate the most efficient travel route.

[0563] Step 2:

[0564] The server sends a control signal to the miniature autonomous device based on the picking route generated in step 1. The control signal includes specific movement instructions indicating which path the device should take. The device receives this signal and automatically moves along the specified route.

[0565] Step 3:

[0566] The terminal functions as smart glasses for the worker, visually displaying the location information of items and the picking route transmitted from the server to the worker. At the same time, specific instructions such as "Move to the location of item A after this item" are displayed to the worker.

[0567] Step 4:

[0568] The emotion engine installed in the terminal acquires biometric data from the worker (e.g., heart rate, skin potential) and analyzes the worker's psychological state in real time. Based on this input data, data processing is performed to evaluate stress and fatigue levels.

[0569] Step 5:

[0570] Based on the emotional data analyzed in step 4, the terminal adjusts the interface according to the worker's psychological state. For example, if the worker is fatigued, the interface is simplified to highlight only important information and a break suggestion is added.

[0571] Step 6:

[0572] Based on the terminal display, the user monitors the progress of the task and requests changes to the search route or control signals from the server as needed. The changes requested by the user are sent to the server and reflected in the operation of the autonomous device in the next step.

[0573] Step 7:

[0574] The server receives a command change request from the user and recalculates the path of the miniature autonomous device based on it. It generates a new control signal and resends the readjusted instructions to the autonomous device. This allows for flexible responses depending on the situation.

[0575] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0576] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0577] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0578] [Fourth Embodiment]

[0579] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0580] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0581] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0582] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0583] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0584] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0585] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0586] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0587] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0588] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0589] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0590] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0591] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0592] This invention relates to a system for supporting search and rescue operations at disaster sites using artificial intelligence that has been generated. Specifically, it can simultaneously control multiple miniature autonomous devices and conduct rapid searches even in confined spaces that are difficult for humans to enter.

[0593] First, the server receives map data and sensor information from the disaster site and analyzes it to generate the optimal search route. At this time, the generated artificial intelligence considers past data and the characteristics of the site to identify important search points. Based on this information, the server generates control signals to instruct the actions of each miniature autonomous device.

[0594] The terminal is operated by the user conducting the rescue operation and has the function of receiving and visually displaying the generated control signals. The terminal also monitors the current location and operating status of each miniature autonomous device in real time, allowing the user to supervise and adjust the search operation as needed. This function enables the user to make rapid decisions in response to the situation they face.

[0595] Users manage the progress of the exploration through their terminal and adjust the exploration strategy as needed. For example, if an important discovery is made, the user can reroute the exploration route and redeploy more autonomous devices to a specific area.

[0596] A concrete example is the search for survivors inside collapsed buildings during natural disasters such as earthquakes. The server considers the changes in the building structure after the earthquake and plans the optimal search route for the miniature autonomous device. Through a terminal, the user can monitor the ongoing search, make manual adjustments as needed, and ultimately take swift and accurate measures to save many lives.

[0597] Thus, by combining the generated artificial intelligence with a miniature autonomous device, this invention efficiently supports search and rescue activities at disaster sites, which were difficult with conventional methods, and greatly expands the possibility of saving lives.

[0598] The following describes the processing flow.

[0599] Step 1:

[0600] The server receives map data of the disaster site and sensor information acquired in real time. This creates a data base based on the latest situation on the ground.

[0601] Step 2:

[0602] The server uses generated artificial intelligence to analyze the received map data and understand the structure of the disaster site. This identifies specific search points and danger areas, which are then mapped according to their importance.

[0603] Step 3:

[0604] The server determines the search route for the miniature autonomous device based on the identified search points. The device's functions and surrounding environment are considered in formulating the optimal path.

[0605] Step 4:

[0606] The server generates control signals for each miniature autonomous device, instructing its operation. These signals include routing settings and operating mode specifications.

[0607] Step 5:

[0608] The terminal receives control signals transmitted from the server and provides them to the rescue team as visual information. The screen displays the device's location and operating status in real time.

[0609] Step 6:

[0610] Users monitor the information displayed through their devices and adjust their search activities as needed. For example, they might reset their search route based on new information or deploy additional equipment to specific areas.

[0611] Step 7:

[0612] The server continuously collects data from each autonomous device and analyzes the exploration results and the status of the devices. This allows for updating the area assessment and optimizing commands for the next step.

[0613] Step 8:

[0614] The terminal provides users with the latest progress and analysis results, forming an information infrastructure to maximize the effectiveness of search and rescue operations. Based on this, users can make strategic decisions and improve the efficiency of rescue operations.

[0615] (Example 1)

[0616] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0617] Rapid and effective search and rescue operations are required at disaster sites, but conventional technologies have problems such as low search efficiency in areas difficult for humans to access and difficulty in real-time situation assessment and rapid response. This invention aims to solve these problems and increase the possibility of saving lives.

[0618] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0619] In this invention, the server includes means for analyzing generated data and calculating the optimal search route based on past disaster data; means for generating control signals for searching for targets at a disaster site using multiple small autonomous mechanisms controlled by the generated artificial intelligence; and means for monitoring the progress of the search and adjusting commands via a user interface. This enables rapid and effective search activities at disaster sites.

[0620] "Generated data" refers to information obtained from disaster sites and includes various source data such as sensor data and geographical information.

[0621] The "optimal search route" is a path calculated based on analyzed historical disaster data and on-site information to maximize the efficiency of search activities.

[0622] A "miniature autonomous mechanism" is a small mechanical device controlled by artificial intelligence, with a structure that allows it to enter confined spaces and operate autonomously.

[0623] A "control signal" is a signal generated to instruct a small autonomous mechanism on its operation, and it contains command information necessary for carrying out search activities.

[0624] A "user interface" is a screen or device that provides a means for the user to monitor their exploration activities and adjust commands.

[0625] "Progress status" refers to the current state of the exploration activity and indicates the extent to which the plan has been implemented.

[0626] The following system configuration and operation are provided as embodiments for carrying out this invention.

[0627] The server plays a role in integrating and managing geographic data acquired from disaster sites and information from sensors. Using this data, the server utilizes a generated AI model to perform analysis that takes into account past disaster data and site characteristics. This allows for the calculation of the optimal search route. The server requires computing hardware with powerful processing capabilities and data analysis functions, and the software used consists of a platform on which data analysis algorithms and AI models operate. A concrete example is real-time analysis using cloud computing services.

[0628] The terminal receives control signals from the server at the disaster site and presents the search status to the user through a visual interface. The terminal is a crucial tool for users to monitor search activities and adjust commands as needed. The user interface is designed to be intuitive and easy to use, and operates on tablets and dedicated monitors. The terminal is equipped with a communication module for real-time data exchange with the server.

[0629] The user uses a device to supervise search and rescue operations at the disaster site. The user manages the progress of the search based on real-time information and controls the search strategy as new information becomes available. For example, in a search within a building whose structure has collapsed due to an earthquake, the user might instruct areas to be searched intensively while avoiding dangerous zones.

[0630] Furthermore, an example of a prompt statement in this invention is, "Explain how to use the sensor data to allow the generating AI model to identify search points." This indicates how the AI ​​model processes the data and extracts important points.

[0631] Thus, the embodiment for carrying out the present invention provides a system that efficiently supports search and rescue activities at disaster sites and enables life-saving efforts.

[0632] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0633] Step 1:

[0634] The server receives real-time geographic and sensor data from disaster sites. Inputs are data acquired from various sensors and drones deployed at the site. The server integrates and organizes this data, preparing it for analysis. Outputs are normalized datasets used in subsequent analysis stages.

[0635] Step 2:

[0636] The server analyzes incoming data using a generative AI model and compares it against a database of past disasters. The input is the normalized data processed in step 1. At this point, the server utilizes the AI ​​model to perform analysis to calculate the optimal route for search activities. The output is a dataset containing recommended routes for the search. This allows the server to identify areas where search efforts should focus.

[0637] Step 3:

[0638] The server generates control signals for each miniature autonomous device based on the analysis results. The input is the search route information generated in step 2. Based on this, the server determines a specific action plan for each device and forms control signals as data packets. The output is the control signals to be sent to the terminal and the miniature autonomous devices.

[0639] Step 4:

[0640] The terminal receives control signals sent from the server and displays them on the user interface. The input is the control signals from the server. The terminal analyzes and visualizes the received signals, making it easy for the user to understand the situation. The output is an intuitive GUI display of progress and control instructions.

[0641] Step 5:

[0642] The user monitors the progress in real time via the terminal and makes decisions regarding the exploration's progress. Input is the visualized information displayed by the terminal. Based on this information, the user can change the direction of the exploration or give additional instructions. Output is feedback signals as a result of the user's commands and adjustments.

[0643] Step 6:

[0644] If necessary, the user instructs the server to reset the search route or resend control signals, and the server performs the corresponding update processing. The inputs are feedback signals and new field information. The server uses these inputs to re-execute the analysis process and, if necessary, generate a new route. The outputs are the updated control signals and search plan. In this way, the system continues to adapt flexibly to changes.

[0645] (Application Example 1)

[0646] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0647] In emergencies such as disasters, there is a need for technologies that can conduct rapid and accurate searches and efficiently support rescue operations for living beings. However, conventional search methods struggle to efficiently gather information and make appropriate decisions continuously in narrow and complex spaces that are difficult for humans to enter directly. Furthermore, the lack of systems that enable real-time location tracking and instruction changes hinders more effective search activities.

[0648] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0649] In this invention, the server includes means for generating control commands to search for living organisms at a specific site using multiple mobile autonomous devices controlled by a generated numerical analysis model; means for monitoring the progress of the search and adjusting instructions via a user interface; means for tracking the current position and operating status of each mobile autonomous device in real time; means for inputting emergency response commands according to the situation via voice; and means for dynamically calculating the optimal route based on received environmental data. This enables rapid and accurate searching in confined and complex spaces, and realizes efficient rescue of living organisms.

[0650] A "numerical analysis model" is a computational method designed to analyze data, extract patterns and trends, and perform a specific task.

[0651] A "mobile autonomous device" is a device that can make decisions and act on its own, and is a device that enables movement to explore a specific location.

[0652] A "control command" refers to an instruction issued to control the operation of equipment according to a specific purpose.

[0653] A "user interface" is the point of contact that enables the exchange of information between a system and a user, and it reflects the user's intentions through operation and display.

[0654] "Exploration progress" refers to status information indicating the stage of the exploration activity, and is data that can be monitored in real time.

[0655] An "emergency response order" defines an order that should be taken immediately in response to an emergency.

[0656] The "optimal route" is the chosen travel path that efficiently reaches a specific destination, and it is derived through calculation.

[0657] This invention is a system for enabling rapid and accurate searching in disaster areas and other similar situations. This system controls multiple mobile autonomous devices, enabling efficient search activities in confined spaces.

[0658] The server uses the generated numerical analysis model to analyze the received environmental data and dynamically calculate the optimal search route for a specific site. The server also generates control commands and transmits them to mobile autonomous devices. In this process, the server uses cloud data streaming services such as Google Firebase and AWS IoT to achieve real-time data processing.

[0659] The terminal is provided to the user, visually displaying the progress of the exploration via a user interface and tracking the current location and operational status of each mobile autonomous device in real time. This interface uses cross-platform app development frameworks such as React Native and Flutter, and displays a visually easy-to-understand map. In addition, voice input functionality is implemented using API.AI and Dialogflow, allowing users to input emergency response commands according to the situation.

[0660] The user monitors the progress of the search through their device, inputs commands via voice or manual, and recalculates the search route as needed. For example, when searching inside a building that has collapsed due to an earthquake, the user can input the following prompt into the AI ​​model to support the search: "Generate a route to search for survivors at the site of the collapsed Tremor Mansion."

[0661] As a concrete example, a user carrying a terminal enters a facility that has collapsed due to an earthquake and operates a mobile autonomous device according to a search route indicated by a server. This device updates the route in real time according to the data on site, enters even narrow gaps to collect information, and transmits the search results to the server. This allows for efficient and safe exploration.

[0662] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0663] Step 1:

[0664] The server receives environmental data (sensor data and map data) from disaster sites via cloud services. Inputs are data from field sensors and geographic information systems. The server preprocesses this data, removing noise and normalizing it to convert it into a format easily processed by the model. The output is clean, analyzable data.

[0665] Step 2:

[0666] The server runs a numerical analysis model using pre-processed data. This model utilizes a generative AI model to plan the optimal search route based on past disaster cases and site characteristics. The input is clean, analyzable data, and based on this, Kalman filters and path planning algorithms are used. The output is command data for the search route.

[0667] Step 3:

[0668] The server transmits the search route command data as a control signal to the mobile autonomous device. The input is the search route command data, which is converted into a communication protocol that the device can understand. In terms of specific operation, remote instructions are given using a communication module. The output is a control signal to the device.

[0669] Step 4:

[0670] The terminal displays the progress of the search route sent from the server in real time on the user interface. The input is the current status information of the device. The terminal updates a dynamic map using the received data, visually indicating the progress. Specifically, the UI is rendered using React Native or similar. The output is the visually displayed progress information.

[0671] Step 5:

[0672] The user directs the search route and emergency response via the terminal using voice commands or direct input. Input consists of the user's voice commands or touch operation data. The terminal analyzes this data and sends commands to the server as needed. Speech recognition using API.AI or Dialogflow is employed. Output is the analyzed user command data.

[0673] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0674] This invention is an advanced system that utilizes artificial intelligence to efficiently search for people in disaster areas. The system consists of multiple miniature autonomous devices and, by incorporating an emotion engine, can recognize the user's emotions and enhance the effectiveness of search activities.

[0675] Specifically, the server receives map data from the disaster site and analyzes it to identify the search area. Next, it generates a search route and sends control signals to each miniature autonomous device. These control signals enable the devices to adapt to confined spaces with biomimetic, flexible movements.

[0676] The terminal is a user-operated device that visually displays information sent from the server. Furthermore, the terminal is equipped with an emotion engine that recognizes the user's emotional state in real time. For example, if the user's stress level is high, it can automatically suggest taking a break or provide operational support. In addition, the interface's color scheme and operation patterns dynamically change according to the user's emotions, helping the user operate the system in the most optimal state.

[0677] The user has the authority to use this system to monitor exploration activities and adjust exploration commands as needed. Because the user can monitor the operation status of the miniature autonomous device via a terminal, rapid decision-making is possible based on the situation on site. The emotion engine analyzes the user's psychological state and helps adjust the human interface to reduce unnecessary stress.

[0678] For example, in the event of a building collapse caused by an earthquake, this system will quickly provide a route for searching for survivors and assist the user in their actions. In situations where the user's tension is high, the terminal display will be adjusted to be easier to understand, and only the most essential information will be highlighted to avoid information overload.

[0679] Thus, the present invention is a system that utilizes user emotion recognition to provide superior functionality in terms of both the efficiency of exploration activities and the reduction of the user's psychological burden.

[0680] The following describes the processing flow.

[0681] Step 1:

[0682] The server receives map data and real-time sensor information from the disaster site. Based on this, the generated artificial intelligence analyzes the data and identifies areas that should be prioritized for search.

[0683] Step 2:

[0684] The server constructs the optimal search route from the analysis results and generates and transmits control signals suitable for each micro-autonomous device. This enables the devices to perform biomimetic, flexible movements.

[0685] Step 3:

[0686] The terminal receives control signals from the server and displays the search route and the device's current location on the user interface. The terminal allows for real-time monitoring of its operating status.

[0687] Step 4:

[0688] The device's built-in emotion engine recognizes the user's emotional state in real time. For example, it measures stress levels through facial expressions and voice analysis.

[0689] Step 5:

[0690] Users can check the progress of their exploration through their device and adjust instructions as needed. Based on the emotion engine's judgment, if the user's stress level is high, suggestions for breaks or operational assistance will be automatically offered.

[0691] Step 6:

[0692] The device dynamically adjusts its operating interface according to the user's emotional state. For example, it might change the color scheme to a more subdued one or highlight necessary information.

[0693] Step 7:

[0694] The server continuously collects feedback from each autonomous device, evaluating exploration results and optimizing commands. This improves the efficiency of exploration and allows for timely strategic adjustments.

[0695] Step 8:

[0696] The latest progress information and analysis results are provided to the user via the terminal, and the user then issues further search commands based on this information. The entire system reduces the burden on the user while enabling effective life-saving efforts.

[0697] (Example 2)

[0698] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0699] Searching for lost lives at disaster sites is a race against time, requiring rapid and efficient operations. Furthermore, it's essential to reduce the burden and psychological stress on operators involved in the search. However, current technology is insufficient for optimizing search routes and addressing users' emotional states, making immediate response at the scene and reducing operator burden challenging.

[0700] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0701] In this invention, the server includes means for generating control signals for searching for human lives at a disaster site using multiple autonomous machines controlled by generated artificial intelligence; means for monitoring the progress of the search and adjusting commands via a user interface; and means for recognizing the user's emotional state in real time and dynamically adjusting the content displayed on the user interface. This makes it possible to increase the efficiency of the search and reduce the psychological burden on the operator.

[0702] "Generated artificial intelligence" refers to a machine learning model that possesses optimized algorithms necessary to achieve a specific objective and has the ability to make situation-appropriate decisions.

[0703] An "autonomous machine" is a device that can perform operations independently with minimal human intervention, and by adopting a biomimetic structure, it can move flexibly and adaptively.

[0704] A "disaster site" is a specific location in an area that has suffered damage due to a natural or man-made disaster, where human lives and supplies are at risk.

[0705] A "control signal" is command data that is sent to an autonomous machine to cause it to act according to a desired operation or route.

[0706] A "user interface" is a means of interaction that allows a user to interact with a system, visually confirm system information, and perform operations.

[0707] "Biomimic structures" are design forms that mimic the natural movements and functions of living organisms to give machines flexibility and adaptability.

[0708] "Map data" refers to a collection of digital or analog information that includes geographical information and provides detailed location information for a specific area.

[0709] "Emotional state" refers to the psychological and emotional state of a user, encompassing a variety of psychological indicators including stress levels and concentration levels.

[0710] This invention is a system that utilizes generated artificial intelligence to streamline the search for people at disaster sites. This system is realized through three main elements: a server, terminals, and users.

[0711] The server receives and analyzes map data acquired from disaster sites. The hardware used includes a server computer with high processing power, and map analysis software and a generative AI model are employed for the analysis. The AI ​​model optimizes the search route based on the received map data. The analyzed route information is generated as control signals for each autonomous machine, which then acts accordingly.

[0712] The terminal is a device used by operators, displaying information supplied from the server in real time through a user interface. The terminal is equipped with an emotion engine that recognizes the user's emotional state by analyzing their heart rate and facial expressions. This allows the terminal's interface display to dynamically adjust, reducing user stress and enabling efficient operation. For example, if the system detects that the user is in a state of tension, only the minimum necessary information will be highlighted on the interface.

[0713] Users have the authority to monitor the progress of search activities via their terminals and adjust commands to the server as needed. For example, when searching for survivors in a building collapsed due to an earthquake, the system will support the user by suggesting the optimal entry and exit routes. This system assists the user's judgment and enables quicker decision-making.

[0714] An example of a prompt message is: "Please explain in detail how to generate efficient search routes in a disaster area. Also, please explain how to dynamically change the device interface according to the user's emotional state." This system is equipped with advanced features to enhance the effectiveness of search operations and reduce the psychological burden on operators.

[0715] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0716] Step 1:

[0717] The server receives map data from the disaster site. The input is map data including GPS coordinates and satellite imagery. Based on this data, the server uses map analysis software to identify search areas. As part of data processing, it analyzes obstacle placement and priority areas for searching for people, and evaluates the risk level and accessibility of specific areas. The output is foundational information for generating search routes.

[0718] Step 2:

[0719] The server generates the optimal search route from map data analyzed using a generative AI model. The input is the foundational information obtained as a result of the analysis. The AI ​​model calculates an efficient route considering terrain characteristics and the location of obstacles. As part of the data calculation, it simulates multiple search routes and selects the shortest and safest route from among them. The output is the specific route data of the control signals sent to each autonomous machine.

[0720] Step 3:

[0721] The server transmits control signals to each autonomous machine based on the generated search route. The input is optimized route data. The device receives this signal and responds with biomimetic, flexible movements. As part of data processing, the operation instructions are converted into machine language and supplied to the device as specific operation commands. The output is a specific operation pattern that allows the autonomous machine to operate even in confined spaces.

[0722] Step 4:

[0723] The terminal visually provides information sent from the server through a user interface. Input consists of search information and control signal data from the server. The terminal organizes this information and displays it in an easy-to-understand format for the user. Specifically, the screen displays a map and search status that are updated in real time. Output is search data displayed visually in a user-friendly format.

[0724] Step 5:

[0725] The device uses an emotion engine to recognize the user's emotional state in real time. Inputs include biometric information such as the user's heart rate and facial expressions. The emotion engine analyzes this information to determine the user's stress level and concentration level. As a data calculation, it dynamically adjusts the user interface based on the emotional state. The output is an interface display optimized for the user's psychological state.

[0726] Step 6:

[0727] The user monitors their exploration activity based on the information displayed on the terminal and issues commands to the server as needed. Input consists of visual information and navigation instructions provided by the terminal. The user analyzes the information and sends commands to the server to prioritize exploring specific areas. Specifically, they adjust their exploration plan using touch input and voice commands. Output is the new exploration command after the operation.

[0728] (Application Example 2)

[0729] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0730] Efficient search and work support are necessary in work environments such as logistics centers and collapse sites. However, conventional autonomous devices and work support systems have shortcomings in terms of dynamic interface adjustments that take into account the psychological state of workers and insufficient stress reduction measures. As a result, problems arise such as decreased work efficiency and accuracy, and increased mental burden on workers.

[0731] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0732] In this invention, the server includes means for generating control signals for searching for human life or objects using multiple miniature autonomous devices controlled by generated artificial intelligence; means for monitoring the progress of the search or work and adjusting commands via a user interface; means for tracking the current position and operating status of each autonomous device; means for sensing the user's psychological state and dynamically adjusting the interface between visual information and work support; and means for providing operational support and break suggestions based on the worker's emotional state. This enables efficient search and work support and reduces the mental burden on the worker.

[0733] Artificial intelligence is a computer program that has the ability to analyze data and make decisions and learn based on the situation.

[0734] A "miniature autonomous device" is a small, independently operating mechanical device designed to function according to a specific purpose.

[0735] A "control signal" is a command transmitted to adjust the operation and functions of an autonomous device.

[0736] A "user interface" is a means that allows users to interact with digital systems and devices.

[0737] "Progress status" refers to information that indicates the current stage and degree of completion of a particular process or task.

[0738] "Adjusting instructions" is the process of modifying or optimizing instructions according to the situation in order to achieve the objective.

[0739] "Emotional state" is a concept that refers to an individual's internal psychological and emotional condition.

[0740] "Dynamic interface adjustment" is a technique that optimizes the display and operation methods in real time according to the user's situation and environment.

[0741] "Work support" refers to actions and systems designed to help workers perform their tasks efficiently and effectively.

[0742] A "suggestion for a break" is an instruction that encourages workers to take a break, taking into account their fatigue level and stress levels.

[0743] The server uses the generated artificial intelligence to create control signals for miniature autonomous devices designed to streamline operations within the logistics center. These control signals analyze the logistics center's map data, acquire the location information of specific products, and instruct the devices to move along the optimal route. The server also considers environmental data and dynamically adjusts the route as needed.

[0744] The terminal functions as smart glasses worn by the worker, providing visual information transmitted from the server. The terminal incorporates an emotion engine that analyzes the worker's psychological state in real time. The emotion engine detects the worker's stress and fatigue levels and adjusts the display interface accordingly to help maximize work efficiency. For example, it can display suggestions for breaks or instructions to adjust the work pace.

[0745] Users have the authority to monitor the overall work progress via a terminal and adjust instructions as needed. Using this system, users can operate the system to ensure work is carried out at a sustainable pace with minimal wear and tear, providing an optimal working environment. In this way, the system enables effective logistics center management and improved worker capabilities.

[0746] As a concrete example, when a worker at a logistics center begins to feel tired, the smart glasses can display a message such as, "You seem tired. Let's try stretching a little," allowing for health management while maintaining work efficiency. An example of a prompt message from the generated AI model in this invention is, "Please propose a method for dealing with worker stress using an emotion engine."

[0747] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0748] Step 1:

[0749] The server receives map data and picking lists for the logistics center in real time. It analyzes this input data to generate the optimal picking route for each product. The server retrieves the location information of the products from the database and uses this to calculate the most efficient travel route.

[0750] Step 2:

[0751] The server sends a control signal to the miniature autonomous device based on the picking route generated in step 1. The control signal includes specific movement instructions indicating which path the device should take. The device receives this signal and automatically moves along the specified route.

[0752] Step 3:

[0753] The terminal functions as smart glasses for the worker, visually displaying the location information of items and the picking route transmitted from the server to the worker. At the same time, specific instructions such as "Move to the location of item A after this item" are displayed to the worker.

[0754] Step 4:

[0755] The emotion engine installed in the terminal acquires biometric data from the worker (e.g., heart rate, skin potential) and analyzes the worker's psychological state in real time. Based on this input data, data processing is performed to evaluate stress and fatigue levels.

[0756] Step 5:

[0757] Based on the emotional data analyzed in step 4, the terminal adjusts the interface according to the worker's psychological state. For example, if the worker is fatigued, the interface is simplified to highlight only important information and a break suggestion is added.

[0758] Step 6:

[0759] Based on the terminal display, the user monitors the progress of the task and requests changes to the search route or control signals from the server as needed. The changes requested by the user are sent to the server and reflected in the operation of the autonomous device in the next step.

[0760] Step 7:

[0761] The server receives a command change request from the user and recalculates the path of the miniature autonomous device based on it. It generates a new control signal and resends the readjusted instructions to the autonomous device. This allows for flexible responses depending on the situation.

[0762] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0763] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0764] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0765] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0766] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0767] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0768] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0769] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0770] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0771] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0772] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0773] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0774] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0775] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0776] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0777] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0778] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0779] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0780] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0781] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0782] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0783] The following is further disclosed regarding the embodiments described above.

[0784] (Claim 1)

[0785] Using multiple miniature autonomous devices controlled by the generated artificial intelligence,

[0786] A means for generating control signals to search for human lives at a disaster site,

[0787] A means for monitoring the progress of the search and adjusting commands via a user interface,

[0788] A means for tracking the current location and operating status of each miniature autonomous device in real time,

[0789] A system that includes this.

[0790] (Claim 2)

[0791] The system according to claim 1, wherein each miniature autonomous device is constructed using a biomimetic structure and designed to enter even narrow spaces.

[0792] (Claim 3)

[0793] The system according to claim 1, further comprising means for analyzing map data of a disaster site and identifying areas that should be searched intensively.

[0794] "Example 1"

[0795] (Claim 1)

[0796] A method for analyzing the generated data and calculating the optimal search route based on past disaster data,

[0797] A means for generating control signals for searching for targets in a disaster area, using multiple small autonomous mechanisms controlled by the generated artificial intelligence,

[0798] A means for monitoring the progress of the search and adjusting commands via a user interface,

[0799] A means for tracking the current position and operating status of each small autonomous device in real time,

[0800] A system that includes this.

[0801] (Claim 2)

[0802] The system according to claim 1, wherein each small autonomous mechanism is constructed using a biomimetic structure and designed to be able to enter even narrow areas.

[0803] (Claim 3)

[0804] The system according to claim 1, further comprising means for analyzing geographic information of a disaster site and identifying areas that should be searched intensively.

[0805] "Application Example 1"

[0806] (Claim 1)

[0807] Using multiple mobile autonomous devices controlled by the generated numerical analysis model,

[0808] A means for generating control commands to search for living organisms at a specific site,

[0809] A means for monitoring the progress of the exploration and adjusting instructions via a user interface,

[0810] A means for tracking the current location and operating status of each mobile autonomous device in real time,

[0811] A means of inputting emergency response commands according to the situation by voice,

[0812] A means of dynamically calculating the optimal route based on received environmental data,

[0813] A system that includes this.

[0814] (Claim 2)

[0815] The system according to claim 1, wherein each mobile autonomous device is constructed using a biomimetic structure and designed to enter even confined spaces.

[0816] (Claim 3)

[0817] The system according to claim 1, further comprising means for analyzing spatial data of a specific site and identifying areas that should be explored intensively.

[0818] "Example 2 of combining an emotion engine"

[0819] (Claim 1)

[0820] Using multiple autonomous machines controlled by the generated artificial intelligence,

[0821] A means for generating control signals to search for human lives at a disaster site,

[0822] A means for monitoring the progress of the search and adjusting commands via a user interface,

[0823] A means for tracking the current position and operating status of each autonomous machine in real time,

[0824] A means for recognizing the user's emotional state in real time and dynamically adjusting the content displayed in the user interface,

[0825] A means for receiving map data, analyzing it, and generating a search route,

[0826] A means of adapting autonomous machines with biomimetic structures to confined spaces,

[0827] A system that includes this.

[0828] (Claim 2)

[0829] The system according to claim 1, wherein each autonomous machine is constructed using a biomimetic structure and designed to enter even confined spaces.

[0830] (Claim 3)

[0831] The system according to claim 1, further comprising means for analyzing map data of a disaster site and identifying areas that should be searched intensively.

[0832] "Application example 2 when combining with an emotional engine"

[0833] (Claim 1)

[0834] Using multiple miniature autonomous devices controlled by the generated artificial intelligence,

[0835] A means for generating control signals to search for human lives at a disaster site,

[0836] A means for monitoring the progress of the search and adjusting commands via a user interface,

[0837] Means for tracking the current location and operating status of each autonomous device,

[0838] A means for sensing the user's psychological state and dynamically adjusting the interface between visual information and work support,

[0839] A means of providing operational support and suggesting breaks based on the emotional state of the worker,

[0840] A system that includes this.

[0841] (Claim 2)

[0842] The system according to claim 1, wherein each autonomous device is constructed using a biomimetic structure and designed to enter even confined spaces.

[0843] (Claim 3)

[0844] The system according to claim 1, further comprising means for analyzing environmental data of disaster sites and work environments to identify areas that should be searched intensively. [Explanation of Symbols]

[0845] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Using multiple miniature autonomous devices controlled by the generated artificial intelligence, A means for generating control signals to search for human lives at a disaster site, A means for monitoring the progress of the search and adjusting commands via a user interface, A means for tracking the current location and operating status of each miniature autonomous device in real time, A system that includes this.

2. The system according to claim 1, wherein each miniature autonomous device is constructed using a biomimetic structure and designed to enter even narrow spaces.

3. The system according to claim 1, further comprising means for analyzing map data of a disaster site and identifying areas that should be searched intensively.

Images