system

Abstract

Provide a system. 【Solution means】 Sensor means for acquiring environmental data, Artificial intelligence means for analyzing the acquired environmental data to detect organisms, Suction means for capturing the detected organisms and sealing them in a containment container, Environmental control means for adjusting the environment inside the containment container to make it difficult for organisms to survive, Moving means for monitoring the state of the containment container and automatically moving to the processing station when full, Processing means for incinerating or pulverizing organisms at the processing station, Notification means for monitoring the state of the system and sending notifications as necessary, A system including the above.

Description

Technical Field

[0001] The technology of this disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Recently, the occurrence of pests, especially cockroaches, in houses and business facilities has become a serious hygienic problem. Existing methods for cockroach control often rely on the use of chemical agents, and there are concerns about the risks to health and the environment associated with their use. In addition, in the prior art, the control work requires labor and is accompanied by daily hassle and discomfort, which is a problem. Furthermore, there is a demand for a system that can continuously and effectively control cockroaches and automatically keep the environment healthy without human intervention during the process.

Means for Solving the Problems

[0005] To solve the above problems, the present invention provides a system comprising a sensor means for acquiring environmental data, an artificial intelligence means for analyzing the acquired environmental data to detect organisms, and a suction means for capturing the detected organisms and sealing them in a containment container. This system also includes an environmental control means for adjusting the environment inside the containment container to make it difficult for organisms to survive, and a transport means for monitoring the state of the containment container and automatically moving it to a processing station when it is full. Furthermore, by including a processing means for incinerating or crushing organisms at the processing station and a notification means for monitoring the state of the system and providing notifications as needed, it is possible to effectively exterminate pests while continuously maintaining a hygienic environment.

[0006] "Sensing means" refers to devices or functions used to acquire environmental data, such as temperature, humidity, and motion information.

[0007] "Artificial intelligence tools" refer to software and algorithms that analyze acquired environmental data and detect organisms that meet specified conditions.

[0008] "Suction means" refers to devices or functions used to physically capture detected organisms and seal them in containment containers.

[0009] A "containment container" refers to a space or container used to temporarily store and isolate organisms that have been aspirated.

[0010] "Environmental control means" refers to mechanisms and functions that adjust the environment inside the containment container to make it difficult for organisms to survive.

[0011] "Transportation means" refers to devices or mechanisms that automatically move the container to the processing station when it is full.

[0012] "Processing means" refers to devices or methods used to incinerate or pulverize organisms within a containment container.

[0013] A "notification mechanism" refers to a function or device that monitors the system status and provides information to the user as needed. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]A sequence diagram showing the processing flow of a data processing system in Application Example 2 when combined with an emotion engine.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a fully automated cockroach extermination system incorporating AI technology. Specific embodiments are described below.

[0036] First, the system has a terminal equipped with multiple sensors that acquire indoor environmental data in real time. The terminal monitors environmental data, including temperature, humidity, and acoustic data, using sensor means, and uses artificial intelligence means to analyze this data. Based on the acquired data, the artificial intelligence means detects the presence of cockroaches with high accuracy.

[0037] When a cockroach is detected, the device's suction mechanism automatically activates, capturing the identified cockroach and sucking it into a sealed containment container. After capture, the environment inside the containment container is adjusted by the device's environmental control mechanism to ensure a dry environment and appropriate temperature, making it difficult for the cockroach to survive.

[0038] This system automatically moves a terminal to a processing station when the number of organisms in the containment container reaches a certain level, or periodically. The terminal, having been moved by the transport mechanism, uses processing equipment at the processing station to incinerate or pulverize the cockroaches in the containment container for final disposal.

[0039] Furthermore, the system is managed by a server, which constantly monitors the progress of tasks and the system's operational status. If maintenance is required based on the operational status, the server sends an alert to the user using a notification system. This allows users to receive notifications on devices such as smartphones and check the maintenance status and system status.

[0040] As a concrete example, when a terminal detects a cockroach, the server receives the analysis results from the AI ​​and establishes a processing protocol. The user checks the notified maintenance information and takes appropriate measures according to the situation. This system allows users to maintain a comfortable and hygienic environment without directly seeing the cockroaches.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The device activates multiple sensors, such as infrared sensors and cameras, to acquire indoor environmental data. It collects information such as temperature, humidity, and motion data, and prepares it for analysis.

[0044] Step 2:

[0045] The device passes the acquired environmental data to artificial intelligence (AI) for data analysis. The AI ​​uses image recognition and pattern recognition to accurately detect cockroaches. The detection results are sent to a server.

[0046] Step 3:

[0047] The server receives cockroach detection information sent from the terminal and records it in the database. It also sends instructions to other system components as needed.

[0048] Step 4:

[0049] Once the device detects a cockroach, it activates the suction mechanism. It precisely points the suction nozzle at the cockroach's location, sucks it up, and seals it in a containment container.

[0050] Step 5:

[0051] The device controls the temperature and humidity inside the containment container after capture. This environmental control creates a harsh environment that makes it difficult for cockroaches to survive, encouraging early disposal.

[0052] Step 6:

[0053] The terminal monitors the status of the storage container until it is full, or periodically thereafter. Upon detecting fullness, it automatically activates the means of transporting the container to the processing station.

[0054] Step 7:

[0055] The terminal moves to the processing station, where it processes the cockroaches in the containment container. The contained cockroaches are safely disposed of in the incineration or crushing equipment at the processing station.

[0056] Step 8:

[0057] The server monitors the overall operating status of the system and records processing results. It also notifies users of maintenance needs and system status as required.

[0058] Step 9:

[0059] Users check notifications sent from the server via a smartphone app and perform simple maintenance tasks and system management based on the instructions.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] The occurrence of organisms indoors can lead to a deterioration of sanitary conditions and cause mental stress. Conventional pest control methods often involve manual work and have limited effectiveness. Furthermore, they lack speed because they take time to completely capture or remove the organisms. The objective of this invention is to solve these problems and enable automated and efficient management of organism control.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes detection means for acquiring environmental conditions, reasoning means for analyzing the acquired environmental conditions to identify organisms, and accumulation means for capturing the identified organisms and sealing them in a storage container. This makes it possible to effectively identify, automatically capture, and process organisms in a room.

[0065] "Environmental conditions" refer to physical and chemical properties, including data such as temperature, humidity, and sound waves in a room.

[0066] "Detection means" refers to sensor devices used to acquire environmental conditions, which enable real-time data collection.

[0067] "Inference means" refers to artificial intelligence technology used to analyze acquired environmental conditions and identify the presence and activity of living organisms.

[0068] "Accumulation means" refers to a mechanism for reliably capturing identified organisms and placing them into a sealed storage container.

[0069] "Regulatory means" refers to a device that allows for precise control of the environment inside a container, adjusting temperature, humidity, sound waves, etc., to make it difficult for living organisms to survive.

[0070] "Transfer means" refers to a mechanism that automatically moves the storage container to a designated processing unit.

[0071] "Processing means" refers to a device for removing or decomposing organisms captured in the processing unit.

[0072] "Notification methods" refer to technologies that monitor the system's status and send warnings to users in case of abnormalities or when regular maintenance is required.

[0073] This invention is a system for the automated extermination of organisms indoors and consists of multiple functional components. At the heart of the system are multiple terminals that acquire environmental data, equipped with temperature sensors, humidity sensors, and acoustic sensors. These terminals collect real-time data from the room and can efficiently analyze the data using AI technology.

[0074] The device uses dedicated AI software to analyze data and identify the presence of organisms. The AI ​​model used enables highly accurate organism detection by analyzing changes in sound frequency and temperature / humidity. Once detection occurs, the device activates a suction device as a means of collection, quickly capturing the identified organism and placing it in a sealed container.

[0075] Once captured, the environment within the container is controlled using regulatory mechanisms, adjusting temperature and humidity to make survival difficult. This process is automatically controlled by the system and carried out without any user intervention.

[0076] If the containment container meets certain conditions, the terminal is automatically moved to a processing station by a transport mechanism. At this processing station, the organism is incinerated or pulverized using processing equipment, and the processing is completed safely and hygienically.

[0077] The server manages the overall system status and constantly monitors data sent from terminals. If operational status or anomalies are detected, the server promptly alerts the user using notification methods. This allows users to perform maintenance at the necessary time, ensuring stable system operation.

[0078] As a concrete example, when a terminal detects a cockroach, the server receives the results of the AI ​​analysis and can set an appropriate processing protocol. An example of a prompt message would be, "Please explain in detail how the cockroach extermination system works. Please explain the procedure from data acquisition by sensors to AI analysis, cockroach capture, and final processing." In this way, this invention supports the optimization of the indoor environment and the improvement of user comfort.

[0079] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0080] Step 1:

[0081] The device operates temperature, humidity, and acoustic sensors to acquire environmental conditions. It receives indoor temperature, humidity, and acoustic data as input, collecting it in real time. As output, this data is ready to be sent to an AI system. This process makes it possible to understand the overall picture of the indoor environment.

[0082] Step 2:

[0083] The device passes the acquired environmental data to an AI model for data analysis. Using the various sensor data collected earlier as input, the AI ​​model identifies specific patterns. The output generates a result indicating whether or not abnormal patterns that may indicate cockroach activity have been found. This enables highly accurate detection of the presence of cockroaches.

[0084] Step 3:

[0085] The device activates the suction device if a cockroach is detected based on the AI ​​analysis results. It receives the confidence level of detection as input and automatically activates the suction mechanism. The output is the capture of the target cockroach in the containment container. The specific operation involves this process, including adjusting the suction force and setting the path for capture.

[0086] Step 4:

[0087] The terminal adjusts the environment inside the containment container where the captured cockroaches are stored. It receives current data on the conditions inside the container as input and performs operations such as lowering humidity or changing the temperature. The output is the maintenance of environmental conditions that make it difficult for cockroaches to survive, thereby preventing the organism from re-infesting the area.

[0088] Step 5:

[0089] The terminal automatically transports the containment container to the processing station when it meets certain conditions. Its input is a count of the total number of organisms in the container, and it determines if a certain quantity has been reached. The output is a notification that the transport is complete and the container has reached the processing station.

[0090] Step 6:

[0091] Upon reaching the processing station, the terminal initiates the process of disposing of the cockroaches internally. As input, it receives the current status of the processing station and selects either incineration or crushing as the process. As output, the organisms are safely disposed of. At this stage, the terminal completes the task of properly disposing of the organisms.

[0092] Step 7:

[0093] The server monitors the overall system status and notifies users if an anomaly is detected. It receives operational data from terminals, detection results, and station status information as input. It sends alert notifications to users as output, allowing them to be informed of the need for maintenance or emergency response.

[0094] (Application Example 1)

[0095] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0096] In large-scale facilities such as logistics centers, there is a need to quickly and accurately detect cockroaches, which are invertebrates, and to efficiently exterminate them without human intervention. However, conventional technologies have made it difficult to quickly detect invertebrates and provide information after extermination, which has made it challenging to maintain hygiene. Furthermore, quickly and accurately conveying information to on-site personnel has also been a challenge.

[0097] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0098] In this invention, the server includes a judgment device that analyzes environmental data and detects invertebrates, a collection device and processing device that capture and process invertebrates, and a display means that provides information using a portable display device. This enables rapid detection and extermination of invertebrates, and information is provided to the person in charge in real time, thereby improving hygiene management of the facility.

[0099] A "detection device" is a device that senses the physical characteristics of the environment and acquires that data.

[0100] A "judgment device" is a device that analyzes the presence of invertebrates based on acquired data and identifies their presence.

[0101] A "collection device" is a device used to capture detected invertebrates and seal them in a designated storage location.

[0102] A "control device" is a device that adjusts the environment inside the containment container to make it difficult for invertebrates to survive.

[0103] A "mobilization device" is a device used to automatically move a system to a designated processing location.

[0104] A "processing device" is a device used to physically process transported invertebrates.

[0105] A "communication device" is a device used to transmit system status and information to a user or other device.

[0106] "Display means" refers to a means of providing information to a user visually using a portable display device.

[0107] To carry out this invention, the following system and process are used.

[0108] The system mainly consists of a decision-making device, a detection device, a data collection device, a control device, a mobile device, a processing device, a communication device, and a display means. The server controls data processing and communication between these devices.

[0109] The detection device acquires environmental data such as temperature, humidity, and sound waves in real time. This provides data to confirm the presence of invertebrates (cockroaches).

[0110] The detection system uses an AI model designed with Python's Scikit-learn and TENSORFLOW® to analyze this environmental data and estimate the presence of invertebrates with high accuracy. If detected, the information is aggregated on a server, and appropriate extermination processes are instructed.

[0111] The collection device activates to capture detected invertebrates and seals them in a containment container. During this process, an environmental control system adjusts the temperature and humidity inside the container to make it difficult for the invertebrates to survive.

[0112] When the containment container is full, the transport device automatically moves to the processing station, where the processing unit physically processes the invertebrates.

[0113] Meanwhile, the communication device transmits information such as the system's operating status and pest control progress from the server to the user's portable display device, such as smart glasses. Through this display device, the user on site can immediately receive the information and take the necessary actions.

[0114] As a concrete example, in a logistics center environment, if the system detects a single cockroach, its location information is immediately displayed on the user's smart glasses. The user can see the cockroach's location through the glasses and monitor how the extermination device automatically responds. This enables efficient and streamlined hygiene management.

[0115] An example of a prompt might be: "Provide prompts for developing a real-time AI model for cockroach detection in a logistics center. The model will use temperature, humidity, and acoustic data to detect the presence of cockroaches with high accuracy."

[0116] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0117] Step 1:

[0118] The server receives environmental data from the sensors. This data includes temperature, humidity, and sound wave data. The server temporarily stores this data in storage.

[0119] Step 2:

[0120] The server inputs stored environmental data into an AI model to analyze the presence of invertebrates. The AI ​​model analyzes the data using a generative AI model and outputs the probability of invertebrate presence. This provides a data-based confidence level for detection.

[0121] Step 3:

[0122] When the server detects that the confidence level exceeds a predetermined threshold, it sends a capture command to the terminal. The terminal then activates its collection device and captures the invertebrate at the detected location. This process is performed automatically based on the command generated by the server.

[0123] Step 4:

[0124] The terminal seals the captured invertebrate in a containment container. The environment inside the containment container is maintained in a state that suppresses the survival of the invertebrate by adjusting the temperature and humidity using a control system.

[0125] Step 5:

[0126] The server monitors the fullness of the containment containers and issues a move command if fullness is confirmed. Based on this command, the terminal automatically moves to the processing site and incinerates or grinds the invertebrates in the processing unit.

[0127] Step 6:

[0128] The server monitors the system's operational status in real time and transmits information to the user's portable display device (e.g., smart glasses). The user then uses the displayed information to confirm the situation on-site and take appropriate action.

[0129] Step 7:

[0130] The server generates prompt messages to provide information to the user and, if necessary, suggests further actions. These prompt messages may include information such as, "Please provide prompts for developing a real-time AI model for cockroach detection in a logistics center," to support subsequent work.

[0131] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0132] The present invention is a cockroach extermination system that incorporates an emotion engine that recognizes the user's emotions and dynamically adjusts the operation of the device based on those emotions. In this embodiment, the emotion engine recognizes the user's emotions from voice and facial expression data and optimizes the operation of the system accordingly.

[0133] The device is equipped with multiple sensors that it uses to capture the user's voice and facial expressions. The emotion engine analyzes the acquired data to determine the user's emotional state. For example, if the user is showing discomfort, the alert content of the notification system can be adjusted to avoid excessive warning sounds and change to a more relaxing message.

[0134] Furthermore, this system simultaneously acquires and analyzes environmental data to detect the presence of cockroaches. Once detected, the device can prioritize or quietly activate suction methods based on the results of the emotion engine. This allows for effective extermination while minimizing the user's sense of urgency.

[0135] As an example of this system, consider a scenario where a cockroach is detected while a user is relaxing late at night. In this case, the device will activate the suction device in silent mode and adjust the alert notification to be less frequent. The server will then record the subsequent processing status and set up a notification to the user the following morning. In this way, by considering the user's emotional state, the system provides a more comfortable and adaptive living environment.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] The device uses a microphone and camera to scan the surrounding environment in order to acquire the user's voice and facial expression data. This data is then sent to the emotion engine.

[0139] Step 2:

[0140] The device's emotion engine analyzes acquired voice and facial expression data to identify the user's emotional state. Based on this analysis, it determines whether the user is "relaxed" or "stressed."

[0141] Step 3:

[0142] The device uses sensors to acquire indoor environmental data. It measures temperature, humidity, and motion data, and uses artificial intelligence to detect cockroaches.

[0143] Step 4:

[0144] When a cockroach is detected, the server considers the results of the emotion engine's analysis to decide how to proceed with the extermination. For example, if the user is relaxed, the device will be instructed to activate the suction device in silent mode.

[0145] Step 5:

[0146] The device activates its suction mechanism to capture detected cockroaches. Throughout this process, it uses information from its emotion engine to perform extermination actions that minimize user stress.

[0147] Step 6:

[0148] The terminal monitors the situation inside the containment container. It continues monitoring until the container is full or the extermination process is complete.

[0149] Step 7:

[0150] The server will send a notification to the user once the removal process is complete. However, if the user is resting, the notification can be changed to a less prominent alert or postponed until the following morning. The user can check the notification on their smartphone or other device and take appropriate action.

[0151] (Example 2)

[0152] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0153] In systems that detect the presence of living organisms, there is a need for methods that consider the user's emotional state and efficiently eliminate organisms without causing stress to the user. Conventional pest control systems do not adjust their operation to take user emotions into consideration, and the sounds and notifications during extermination may cause discomfort to the user. Therefore, the challenge is to achieve flexible system control that responds to the user's emotional state.

[0154] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0155] In this invention, the server includes sensor means for acquiring environmental data, data collection means for acquiring user voice and facial expression data, and emotion recognition means for analyzing the acquired voice and facial expression data and determining the emotional state. This makes it possible to optimize system operation and adjust notification content according to the user's emotional state.

[0156] "Environmental data" refers to data that includes information such as ambient temperature and movement, used to detect the presence of living organisms.

[0157] "Sensor means" refers to various sensors used to acquire environmental data and user data, and consists of temperature sensors, motion detection sensors, and devices necessary for acquiring audio and video.

[0158] "Information processing means" refers to computer systems and algorithms used to analyze acquired data and identify the presence of living organisms.

[0159] "Data collection means" refers to a system consisting of devices and programs for acquiring data such as the user's voice and facial expressions.

[0160] An "emotion recognition system" is a system that includes machine learning models and algorithms for analyzing collected data and identifying the user's emotional state.

[0161] "Control means" refers to a mechanism for adjusting the system's operating mode, sound, and notification content based on the user's emotional state and biological detection information.

[0162] A "suction device" is a device used to physically capture an organism and seal it inside a specific container.

[0163] A "notification method" is a means of communicating the system status and detection information to the user through sound or video.

[0164] This invention is a biological pest control system that can flexibly adjust its operation based on the user's emotional state. The system has the ability to recognize the user's voice and facial expressions and change its operation accordingly. The specific implementation method is described below.

[0165] This system incorporates multiple sensor devices on the terminal to collect user voice and facial expression data. Common APIs and libraries are used for voice recognition, while image processing libraries are utilized for facial expression recognition. Specifically, the microphone and camera function as data input devices. Using these devices, the terminal captures raw user data and inputs it into an emotion recognition algorithm.

[0166] The emotion recognition algorithm employs a generative AI model. The server uses this model to process data in real time and analyze the user's emotional state. For example, if the system detects that the user is relaxed, it is controlled to operate in silent mode.

[0167] The device also collects ambient environmental data in parallel and uses information processing to detect the presence of living organisms. When a target organism is detected, the system immediately executes the suction mechanism, but the sound and notification content at that time are adjusted to reflect the results of emotion recognition.

[0168] For example, if the system detects a living organism while the user is relaxing late at night, the device will quietly perform the suction process, and the user will receive a discreet notification. The server will then record the processing log and provide a notification at a time of the user's choosing.

[0169] Examples of prompts to input into a generative AI model:

[0170] "Based on user sentiment data, how can we optimize the device's behavior? Please propose specific adjustments."

[0171] Thus, the present invention realizes biological pest control in an environment that takes user experience into maximum consideration.

[0172] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0173] Step 1:

[0174] The device uses its built-in microphone and camera to acquire user voice and facial expression data. The input is the user's real-time voice and video stream, and the output is generated as audio and image data. This data is then sent directly to the next processing step.

[0175] Step 2:

[0176] The device converts voice data into text data via a speech recognition API and analyzes facial expression data using an image processing library. The input is the raw data obtained in step 1, and the output is information converted into emotion labels (e.g., "relaxed," "stressed," etc.). This allows for an initial determination of the user's emotional state.

[0177] Step 3:

[0178] The server uses a generative AI model to analyze the text data and emotion label data acquired in step 2 to determine the user's detailed emotional state. The input is the emotion label data from step 2, and the output is the accuracy of the emotion and the optimal response strategy. Based on this, the system's operation is adjusted.

[0179] Step 4:

[0180] The terminal utilizes temperature and motion detection sensors to acquire environmental data. Sensor data from the surrounding environment is used as input, and biological detection information is generated as output. This information is immediately analyzed within the system, and pest control measures are taken as needed.

[0181] Step 5:

[0182] The device activates the suction mechanism in silent mode based on the user's emotional state and biological detection information. The input is the emotional analysis result from step 3 and the biological detection result from step 4, and the output is a silent and effective suction operation. This captures organisms without causing excessive stress to the user.

[0183] Step 6:

[0184] The server monitors all processes and is configured to send notifications to the user as needed. Log data from all steps is used as input, and the output consists of processing history and notification information provided to the user. For example, processing results from late-night operations are communicated to the user the following morning. This entire process enables comfortable and efficient pest control while considering the user's feelings.

[0185] (Application Example 2)

[0186] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0187] Conventional pest control systems had the problem of causing unnecessary stress to users through extermination activities without considering their emotions. Furthermore, recognizing the organisms and properly handling them after capture was difficult, making effective extermination challenging. In addition, adjusting services to customer emotions in physical stores was difficult, creating room for improvement in customer satisfaction. Therefore, there was a need for a system that could recognize user emotions and dynamically adjust device operation.

[0188] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0189] In this invention, the server includes detection means for acquiring environmental information, machine learning means for analyzing the acquired environmental information to recognize organisms, a suction device for capturing the recognized organisms and sealing them in a containment device, acquisition means for acquiring emotional data, an analysis device for analyzing the acquired emotional data to determine the user's emotional state, and adjustment means for dynamically adjusting notification content based on the emotional state. This enables effective and less stressful pest control activities and service improvements while taking into account the user's emotions and the state of customers in the store.

[0190] "Environmental information" refers to information about the surroundings of the subject, and includes data such as temperature, humidity, and sound waves.

[0191] "Detection means" refers to devices used to acquire environmental information, and includes various sensors.

[0192] A "machine learning device" is a device that uses a learning algorithm to analyze acquired environmental information and recognize the presence of living organisms.

[0193] A "suction device" is a device used to capture recognized organisms and seal them inside a containment device.

[0194] A "containment device" is a device used to temporarily store captured organisms and has a structure that allows for environmental control.

[0195] "Emotional data" refers to information obtained from the user's voice and facial expressions, and serves as the basis for inferring their emotional state.

[0196] "Means of acquisition" refers to devices used to collect emotional data, including cameras and microphones.

[0197] An "analysis device" is a device that analyzes acquired emotional data to determine the user's emotional state.

[0198] A "modification device" is a device that dynamically changes notification content and extermination activities based on emotional state.

[0199] To implement this invention, a system is utilized in which a server collects environmental information and emotional data and analyzes them. This system is equipped with a combination of sensors and analytical algorithms, and specifically includes smart glasses and dedicated devices.

[0200] The server uses various sensors to acquire environmental information (temperature, humidity, sound waves, etc.) in real time. These include infrared sensors and ultrasonic sensors. Furthermore, a device integrating a microphone and camera is used to acquire emotional data from the user's voice and facial expressions. This device utilizes computer vision and speech recognition technologies to accurately determine the user's emotional state.

[0201] The acquired data is analyzed by a machine learning model on the server (e.g., a deep learning model using Python and TensorFlow). This model achieves highly accurate emotion recognition by utilizing services such as Microsoft® Azure®'s Emotion API and Face API.

[0202] Based on the analysis results, the server dynamically adjusts notification content and extermination strategies. For example, if the user is relaxed, the notification sound is suppressed and the suction device operates in silent mode. This allows for efficient extermination without causing unnecessary stress to the user.

[0203] As a concrete example, considering its application in a retail setting, this system can recognize the emotional state of customers and automatically optimize service. By using this system, users can increase customer satisfaction and provide a less stressful environment.

[0204] An example of a prompt message could be, "If a customer is showing signs of stress in the store, please suggest how to flexibly adjust your service style." This auxiliary function provides store employees and staff with guidance to provide appropriate responses.

[0205] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0206] Step 1:

[0207] The server acquires ambient environmental information (temperature, humidity, sound waves, etc.) using sensors embedded in smart glasses or dedicated devices. The input is raw data from the sensors, and the output is environmental data that has been organized and converted into a format suitable for analysis. This conversion includes saving to a database and converting to a specific format.

[0208] Step 2:

[0209] The device uses a camera and microphone to record and capture the user's voice and facial expressions. Input consists of audio and image data, while output is emotion data extracted through speech recognition and image analysis. This process utilizes Microsoft Azure's Emotion API and Face API to quantify and determine the emotional state.

[0210] Step 3:

[0211] The server inputs the acquired environmental and emotional data into a machine learning model for analysis. The input is the data obtained in the previous step, and the output is a behavioral indicator based on the user's emotional state and surrounding environmental conditions. The data is processed by a TensorFlow model, and an appropriate response is determined for each situation.

[0212] Step 4:

[0213] Based on the analysis results, the server determines the notification content and the device's operating mode. The input is the analysis results from step 3, and the output is the specific notification message and the settings information for the pest control system. For example, the pest control system may be set to silent mode, or the notification sound may be suppressed.

[0214] Step 5:

[0215] The user verifies the optimized device operation based on instructions from the server and makes manual adjustments as needed. Inputs are notifications and suggestions from the server, while outputs are the final system settings based on the user's choices. This process is facilitated through a user interface, providing a relaxed environment.

[0216] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0217] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0218] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0219] [Second Embodiment]

[0220] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0221] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0222] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0223] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0224] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0225] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0226] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0227] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0228] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0229] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0230] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0231] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0232] This invention is a fully automated cockroach extermination system incorporating AI technology. Specific embodiments are described below.

[0233] First, the system has a terminal equipped with multiple sensors that acquire indoor environmental data in real time. The terminal monitors environmental data, including temperature, humidity, and acoustic data, using sensor means, and uses artificial intelligence means to analyze this data. Based on the acquired data, the artificial intelligence means detects the presence of cockroaches with high accuracy.

[0234] When a cockroach is detected, the device's suction mechanism automatically activates, capturing the identified cockroach and sucking it into a sealed containment container. After capture, the environment inside the containment container is adjusted by the device's environmental control mechanism to ensure a dry environment and appropriate temperature, making it difficult for the cockroach to survive.

[0235] This system automatically moves a terminal to a processing station when the number of organisms in the containment container reaches a certain level, or periodically. The terminal, having been moved by the transport mechanism, uses processing equipment at the processing station to incinerate or pulverize the cockroaches in the containment container for final disposal.

[0236] Furthermore, the system is managed by a server, which constantly monitors the progress of tasks and the system's operational status. If maintenance is required based on the operational status, the server sends an alert to the user using a notification system. This allows users to receive notifications on devices such as smartphones and check the maintenance status and system status.

[0237] As a concrete example, when a terminal detects a cockroach, the server receives the analysis results from the AI ​​and establishes a processing protocol. The user checks the notified maintenance information and takes appropriate measures according to the situation. This system allows users to maintain a comfortable and hygienic environment without directly seeing the cockroaches.

[0238] The following describes the processing flow.

[0239] Step 1:

[0240] The device activates multiple sensors, such as infrared sensors and cameras, to acquire indoor environmental data. It collects information such as temperature, humidity, and motion data, and prepares it for analysis.

[0241] Step 2:

[0242] The device passes the acquired environmental data to artificial intelligence (AI) for data analysis. The AI ​​uses image recognition and pattern recognition to accurately detect cockroaches. The detection results are sent to a server.

[0243] Step 3:

[0244] The server receives cockroach detection information sent from the terminal and records it in the database. It also sends instructions to other system components as needed.

[0245] Step 4:

[0246] Once the device detects a cockroach, it activates the suction mechanism. It precisely points the suction nozzle at the cockroach's location, sucks it up, and seals it in a containment container.

[0247] Step 5:

[0248] The device controls the temperature and humidity inside the containment container after capture. This environmental control creates a harsh environment that makes it difficult for cockroaches to survive, encouraging early disposal.

[0249] Step 6:

[0250] The terminal monitors the status of the storage container until it is full, or periodically thereafter. Upon detecting fullness, it automatically activates the means of transporting the container to the processing station.

[0251] Step 7:

[0252] The terminal moves to the processing station, where it processes the cockroaches in the containment container. The contained cockroaches are safely disposed of in the incineration or crushing equipment at the processing station.

[0253] Step 8:

[0254] The server monitors the overall operating status of the system and records processing results. It also notifies users of maintenance needs and system status as required.

[0255] Step 9:

[0256] Users check notifications sent from the server via a smartphone app and perform simple maintenance tasks and system management based on the instructions.

[0257] (Example 1)

[0258] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0259] The occurrence of organisms indoors can lead to a deterioration of sanitary conditions and cause mental stress. Conventional pest control methods often involve manual work and have limited effectiveness. Furthermore, they lack speed because they take time to completely capture or remove the organisms. The objective of this invention is to solve these problems and enable automated and efficient management of organism control.

[0260] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0261] In this invention, the server includes detection means for acquiring environmental conditions, reasoning means for analyzing the acquired environmental conditions to identify organisms, and accumulation means for capturing the identified organisms and sealing them in a storage container. This makes it possible to effectively identify, automatically capture, and process organisms in a room.

[0262] "Environmental conditions" refer to physical and chemical properties, including data such as temperature, humidity, and sound waves in a room.

[0263] "Detection means" refers to sensor devices used to acquire environmental conditions, which enable real-time data collection.

[0264] "Inference means" refers to artificial intelligence technology used to analyze acquired environmental conditions and identify the presence and activity of living organisms.

[0265] "Accumulation means" refers to a mechanism for reliably capturing identified organisms and placing them into a sealed storage container.

[0266] "Regulatory means" refers to a device that allows for precise control of the environment inside a container, adjusting temperature, humidity, sound waves, etc., to make it difficult for living organisms to survive.

[0267] "Transfer means" refers to a mechanism that automatically moves the storage container to a designated processing unit.

[0268] "Processing means" refers to a device for removing or decomposing organisms captured in the processing unit.

[0269] "Notification methods" refer to technologies that monitor the system's status and send warnings to users in case of abnormalities or when regular maintenance is required.

[0270] This invention is a system for the automated extermination of organisms indoors and consists of multiple functional components. At the heart of the system are multiple terminals that acquire environmental data, equipped with temperature sensors, humidity sensors, and acoustic sensors. These terminals collect real-time data from the room and can efficiently analyze the data using AI technology.

[0271] The device uses dedicated AI software to analyze data and identify the presence of organisms. The AI ​​model used enables highly accurate organism detection by analyzing changes in sound frequency and temperature / humidity. Once detection occurs, the device activates a suction device as a means of collection, quickly capturing the identified organism and placing it in a sealed container.

[0272] Once captured, the environment within the container is controlled using regulatory mechanisms, adjusting temperature and humidity to make survival difficult. This process is automatically controlled by the system and carried out without any user intervention.

[0273] If the containment container meets certain conditions, the terminal is automatically moved to a processing station by a transport mechanism. At this processing station, the organism is incinerated or pulverized using processing equipment, and the processing is completed safely and hygienically.

[0274] The server manages the overall system status and constantly monitors data sent from terminals. If operational status or anomalies are detected, the server promptly alerts the user using notification methods. This allows users to perform maintenance at the necessary time, ensuring stable system operation.

[0275] As a concrete example, when a terminal detects a cockroach, the server receives the results of the AI ​​analysis and can set an appropriate processing protocol. An example of a prompt message would be, "Please explain in detail how the cockroach extermination system works. Please explain the procedure from data acquisition by sensors to AI analysis, cockroach capture, and final processing." In this way, this invention supports the optimization of the indoor environment and the improvement of user comfort.

[0276] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0277] Step 1:

[0278] The terminal operates temperature sensors, humidity sensors, and acoustic sensors to obtain environmental conditions. It receives indoor temperature, humidity, and acoustic data as input and collects them in real time. As output, it is ready to send these data to an AI system. Through this process, it is possible to grasp the overall picture of the indoor environment.

[0279] Step 2:

[0280] The terminal passes the acquired environmental data to an AI model and performs data analysis. Using the various sensor data collected earlier as input, the AI model identifies specific patterns. As output, a result is generated indicating whether an abnormal pattern that may indicate cockroach activity is found. This enables highly accurate detection of the presence of cockroaches.

[0281] Step 3:

[0282] When the terminal detects a cockroach based on the AI analysis result, it activates the suction device. It receives the confidence level of the detection as input and automatically operates the suction means. As output, the target cockroach is captured in the storage container. The specific operation is this process including adjustment of the suction force and setting of the path for capture.

[0283] Step 4:

[0284] The terminal adjusts the environment inside the storage container where the captured cockroach is stored. It receives the current situation data inside the container as input and performs operations such as lowering the humidity or changing the temperature. As output, an environmental state that makes it difficult for cockroaches to survive is maintained. This prevents the recurrence of organisms.

[0285] Step 5:

[0286] When the storage container meets specific conditions, the terminal executes automatic transfer to the processing station. As input, it counts the cumulative number of organisms in the container and determines whether a certain quantity has been reached. The output is a notification that the movement has been completed and the processing station has been reached.

[0287] Step 6:

[0288] After reaching the processing station, the terminal starts the procedure for processing cockroaches internally. As input, it receives the status of the current processing station and selects either the incineration or pulverization process. As output, the organisms are safely processed. At this stage, the terminal completes the task of properly disposing of the organisms.

[0289] Step 7:

[0290] The server monitors the overall system situation and reports to the user if an abnormality is detected. As input, it receives operation data, detection results, and status information of the station from the terminal. As output, an alert notification is sent to the user. Thus, the user can know the need for maintenance or emergency response.

[0291] (Application Example 1)

[0292] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0293] In large facilities such as logistics centers, there is a need to quickly and accurately detect cockroaches, which are invertebrates, and efficiently exterminate them without human intervention. However, in the conventional technology, it was difficult to quickly detect invertebrates and provide information after extermination, so there was a problem that it was difficult to maintain hygiene. Furthermore, quickly and accurately transmitting information to the on-site staff was also one of the problems.

[0294] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0295] In this invention, the server includes a judgment device that analyzes environmental data and detects invertebrates, a collection device and processing device that capture and process invertebrates, and a display means that provides information using a portable display device. This enables rapid detection and extermination of invertebrates, and information is provided to the person in charge in real time, thereby improving hygiene management of the facility.

[0296] A "detection device" is a device that senses the physical characteristics of the environment and acquires that data.

[0297] A "judgment device" is a device that analyzes the presence of invertebrates based on acquired data and identifies their presence.

[0298] A "collection device" is a device used to capture detected invertebrates and seal them in a designated storage location.

[0299] A "control device" is a device that adjusts the environment inside the containment container to make it difficult for invertebrates to survive.

[0300] A "mobilization device" is a device used to automatically move a system to a designated processing location.

[0301] A "processing device" is a device used to physically process transported invertebrates.

[0302] A "communication device" is a device used to transmit system status and information to a user or other device.

[0303] "Display means" refers to a means of providing information to a user visually using a portable display device.

[0304] To carry out this invention, the following system and process are used.

[0305] The system is mainly composed of a judgment device, a detection device, a collection device, a control device, a moving device, a processing device, a communication device, and display means. The server controls data processing and communication among these devices.

[0306] The detection device acquires environmental data such as temperature, humidity, and sound waves in real time. This provides data for confirming the presence of invertebrates (cockroaches).

[0307] In the judgment device, an AI model designed using Python's Scikit-learn, TensorFlow, etc. analyzes this environmental data and accurately estimates the presence of invertebrates. If detected, the information is aggregated by the server and an appropriate extermination process is instructed.

[0308] The collection device operates to capture the detected invertebrates and seals them in a containment container. At this time, the environmental control device adjusts the temperature and humidity inside the container, making it difficult for the invertebrates to survive.

[0309] When the containment container is full, the moving device automatically moves to the processing site, and the processing device physically processes the invertebrates there.

[0310] On the other hand, the communication device transmits information such as the operation status and extermination status of the system from the server to the user's portable display device, such as smart glasses. Through the display means, on-site users can immediately receive information and take necessary actions.

[0311] As a specific example, in the environment of a logistics center, when the system detects a single cockroach, its location information is immediately displayed on the user's smart glasses. The user can grasp the location of the cockroach through the glasses and monitor how the extermination device automatically responds. This enables efficient and waste-free hygiene management.

[0312] An example of a prompt might be: "Provide prompts for developing a real-time AI model for cockroach detection in a logistics center. The model will use temperature, humidity, and acoustic data to detect the presence of cockroaches with high accuracy."

[0313] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0314] Step 1:

[0315] The server receives environmental data from the sensors. This data includes temperature, humidity, and sound wave data. The server temporarily stores this data in storage.

[0316] Step 2:

[0317] The server inputs stored environmental data into an AI model to analyze the presence of invertebrates. The AI ​​model analyzes the data using a generative AI model and outputs the probability of invertebrate presence. This provides a data-based confidence level for detection.

[0318] Step 3:

[0319] When the server detects that the confidence level exceeds a predetermined threshold, it sends a capture command to the terminal. The terminal then activates its collection device and captures the invertebrate at the detected location. This process is performed automatically based on the command generated by the server.

[0320] Step 4:

[0321] The terminal seals the captured invertebrate in a containment container. The environment inside the containment container is maintained in a state that suppresses the survival of the invertebrate by adjusting the temperature and humidity using a control system.

[0322] Step 5:

[0323] The server monitors the fullness of the containment containers and issues a move command if fullness is confirmed. Based on this command, the terminal automatically moves to the processing site and incinerates or grinds the invertebrates in the processing unit.

[0324] Step 6:

[0325] The server monitors the system's operational status in real time and transmits information to the user's portable display device (e.g., smart glasses). The user then uses the displayed information to confirm the situation on-site and take appropriate action.

[0326] Step 7:

[0327] The server generates prompt messages to provide information to the user and, if necessary, suggests further actions. These prompt messages may include information such as, "Please provide prompts for developing a real-time AI model for cockroach detection in a logistics center," to support subsequent work.

[0328] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0329] The present invention is a cockroach extermination system that incorporates an emotion engine that recognizes the user's emotions and dynamically adjusts the operation of the device based on those emotions. In this embodiment, the emotion engine recognizes the user's emotions from voice and facial expression data and optimizes the operation of the system accordingly.

[0330] The device is equipped with multiple sensors that it uses to capture the user's voice and facial expressions. The emotion engine analyzes the acquired data to determine the user's emotional state. For example, if the user is showing discomfort, the alert content of the notification system can be adjusted to avoid excessive warning sounds and change to a more relaxing message.

[0331] Furthermore, this system simultaneously acquires and analyzes environmental data to detect the presence of cockroaches. Once detected, the device can prioritize or quietly activate suction methods based on the results of the emotion engine. This allows for effective extermination while minimizing the user's sense of urgency.

[0332] As an example of this system, consider a scenario where a cockroach is detected while a user is relaxing late at night. In this case, the device will activate the suction device in silent mode and adjust the alert notification to be less frequent. The server will then record the subsequent processing status and set up a notification to the user the following morning. In this way, by considering the user's emotional state, the system provides a more comfortable and adaptive living environment.

[0333] The following describes the processing flow.

[0334] Step 1:

[0335] The device uses a microphone and camera to scan the surrounding environment in order to acquire the user's voice and facial expression data. This data is then sent to the emotion engine.

[0336] Step 2:

[0337] The device's emotion engine analyzes acquired voice and facial expression data to identify the user's emotional state. Based on this analysis, it determines whether the user is "relaxed" or "stressed."

[0338] Step 3:

[0339] The device uses sensors to acquire indoor environmental data. It measures temperature, humidity, and motion data, and uses artificial intelligence to detect cockroaches.

[0340] Step 4:

[0341] When a cockroach is detected, the server considers the results of the emotion engine's analysis to decide how to proceed with the extermination. For example, if the user is relaxed, the device will be instructed to activate the suction device in silent mode.

[0342] Step 5:

[0343] The device activates its suction mechanism to capture detected cockroaches. Throughout this process, it uses information from its emotion engine to perform extermination actions that minimize user stress.

[0344] Step 6:

[0345] The terminal monitors the situation inside the containment container. It continues monitoring until the container is full or the extermination process is complete.

[0346] Step 7:

[0347] The server will send a notification to the user once the removal process is complete. However, if the user is resting, the notification can be changed to a less prominent alert or postponed until the following morning. The user can check the notification on their smartphone or other device and take appropriate action.

[0348] (Example 2)

[0349] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0350] In systems that detect the presence of living organisms, there is a need for methods that consider the user's emotional state and efficiently eliminate organisms without causing stress to the user. Conventional pest control systems do not adjust their operation to take user emotions into consideration, and the sounds and notifications during extermination may cause discomfort to the user. Therefore, the challenge is to achieve flexible system control that responds to the user's emotional state.

[0351] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0352] In this invention, the server includes sensor means for acquiring environmental data, data collection means for acquiring user voice and facial expression data, and emotion recognition means for analyzing the acquired voice and facial expression data and determining the emotional state. This makes it possible to optimize system operation and adjust notification content according to the user's emotional state.

[0353] "Environmental data" refers to data that includes information such as ambient temperature and movement, used to detect the presence of living organisms.

[0354] "Sensor means" refers to various sensors used to acquire environmental data and user data, and consists of temperature sensors, motion detection sensors, and devices necessary for acquiring audio and video.

[0355] "Information processing means" refers to computer systems and algorithms used to analyze acquired data and identify the presence of living organisms.

[0356] "Data collection means" refers to a system consisting of devices and programs for acquiring data such as the user's voice and facial expressions.

[0357] An "emotion recognition system" is a system that includes machine learning models and algorithms for analyzing collected data and identifying the user's emotional state.

[0358] "Control means" refers to a mechanism for adjusting the system's operating mode, sound, and notification content based on the user's emotional state and biological detection information.

[0359] A "suction device" is a device used to physically capture an organism and seal it inside a specific container.

[0360] A "notification method" is a means of communicating the system status and detection information to the user through sound or video.

[0361] This invention is a biological pest control system that can flexibly adjust its operation based on the user's emotional state. The system has the ability to recognize the user's voice and facial expressions and change its operation accordingly. The specific implementation method is described below.

[0362] This system incorporates multiple sensor devices on the terminal to collect user voice and facial expression data. Common APIs and libraries are used for voice recognition, while image processing libraries are utilized for facial expression recognition. Specifically, the microphone and camera function as data input devices. Using these devices, the terminal captures raw user data and inputs it into an emotion recognition algorithm.

[0363] The emotion recognition algorithm employs a generative AI model. The server uses this model to process data in real time and analyze the user's emotional state. For example, if the system detects that the user is relaxed, it is controlled to operate in silent mode.

[0364] The device also collects ambient environmental data in parallel and uses information processing to detect the presence of living organisms. When a target organism is detected, the system immediately executes the suction mechanism, but the sound and notification content at that time are adjusted to reflect the results of emotion recognition.

[0365] For example, if the system detects a living organism while the user is relaxing late at night, the device will quietly perform the suction process, and the user will receive a discreet notification. The server will then record the processing log and provide a notification at a time of the user's choosing.

[0366] Examples of prompts to input into a generative AI model:

[0367] "Based on user sentiment data, how can we optimize the device's behavior? Please propose specific adjustments."

[0368] Thus, the present invention realizes biological pest control in an environment that takes user experience into maximum consideration.

[0369] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0370] Step 1:

[0371] The device uses its built-in microphone and camera to acquire user voice and facial expression data. The input is the user's real-time voice and video stream, and the output is generated as audio and image data. This data is then sent directly to the next processing step.

[0372] Step 2:

[0373] The device converts voice data into text data via a speech recognition API and analyzes facial expression data using an image processing library. The input is the raw data obtained in step 1, and the output is information converted into emotion labels (e.g., "relaxed," "stressed," etc.). This allows for an initial determination of the user's emotional state.

[0374] Step 3:

[0375] The server uses a generative AI model to analyze the text data and emotion label data acquired in step 2 to determine the user's detailed emotional state. The input is the emotion label data from step 2, and the output is the accuracy of the emotion and the optimal response strategy. Based on this, the system's operation is adjusted.

[0376] Step 4:

[0377] The terminal utilizes temperature and motion detection sensors to acquire environmental data. Sensor data from the surrounding environment is used as input, and biological detection information is generated as output. This information is immediately analyzed within the system, and pest control measures are taken as needed.

[0378] Step 5:

[0379] The device activates the suction mechanism in silent mode based on the user's emotional state and biological detection information. The input is the emotional analysis result from step 3 and the biological detection result from step 4, and the output is a silent and effective suction operation. This captures organisms without causing excessive stress to the user.

[0380] Step 6:

[0381] The server monitors all processes and is configured to send notifications to the user as needed. Log data from all steps is used as input, and the output consists of processing history and notification information provided to the user. For example, processing results from late-night operations are communicated to the user the following morning. This entire process enables comfortable and efficient pest control while considering the user's feelings.

[0382] (Application Example 2)

[0383] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0384] Conventional pest control systems had the problem of causing unnecessary stress to users through extermination activities without considering their emotions. Furthermore, recognizing the organisms and properly handling them after capture was difficult, making effective extermination challenging. In addition, adjusting services to customer emotions in physical stores was difficult, creating room for improvement in customer satisfaction. Therefore, there was a need for a system that could recognize user emotions and dynamically adjust device operation.

[0385] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0386] In this invention, the server includes detection means for acquiring environmental information, machine learning means for analyzing the acquired environmental information to recognize organisms, a suction device for capturing the recognized organisms and sealing them in a containment device, acquisition means for acquiring emotional data, an analysis device for analyzing the acquired emotional data to determine the user's emotional state, and adjustment means for dynamically adjusting notification content based on the emotional state. This enables effective and less stressful pest control activities and service improvements while taking into account the user's emotions and the state of customers in the store.

[0387] "Environmental information" refers to information about the surroundings of the subject, and includes data such as temperature, humidity, and sound waves.

[0388] "Detection means" refers to devices used to acquire environmental information, and includes various sensors.

[0389] A "machine learning device" is a device that uses a learning algorithm to analyze acquired environmental information and recognize the presence of living organisms.

[0390] A "suction device" is a device used to capture recognized organisms and seal them inside a containment device.

[0391] A "containment device" is a device used to temporarily store captured organisms and has a structure that allows for environmental control.

[0392] "Emotional data" refers to information obtained from the user's voice and facial expressions, and serves as the basis for inferring their emotional state.

[0393] "Means of acquisition" refers to devices used to collect emotional data, including cameras and microphones.

[0394] An "analysis device" is a device that analyzes acquired emotional data to determine the user's emotional state.

[0395] A "modification device" is a device that dynamically changes notification content and extermination activities based on emotional state.

[0396] To implement this invention, a system is utilized in which a server collects environmental information and emotional data and analyzes them. This system is equipped with a combination of sensors and analytical algorithms, and specifically includes smart glasses and dedicated devices.

[0397] The server uses various sensors to acquire environmental information (temperature, humidity, sound waves, etc.) in real time. These include infrared sensors and ultrasonic sensors. Furthermore, a device integrating a microphone and camera is used to acquire emotional data from the user's voice and facial expressions. This device utilizes computer vision and speech recognition technologies to accurately determine the user's emotional state.

[0398] The acquired data is analyzed by a machine learning model on the server (e.g., a deep learning model using Python and TensorFlow). This model achieves highly accurate emotion recognition by utilizing services such as Microsoft Azure's Emotion API and Face API.

[0399] Based on the analysis results, the server dynamically adjusts notification content and extermination strategies. For example, if the user is relaxed, the notification sound is suppressed and the suction device operates in silent mode. This allows for efficient extermination without causing unnecessary stress to the user.

[0400] As a concrete example, considering its application in a retail setting, this system can recognize the emotional state of customers and automatically optimize service. By using this system, users can increase customer satisfaction and provide a less stressful environment.

[0401] An example of a prompt message could be, "If a customer is showing signs of stress in the store, please suggest how to flexibly adjust your service style." This auxiliary function provides store employees and staff with guidance to provide appropriate responses.

[0402] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0403] Step 1:

[0404] The server acquires ambient environmental information (temperature, humidity, sound waves, etc.) using sensors embedded in smart glasses or dedicated devices. The input is raw data from the sensors, and the output is environmental data that has been organized and converted into a format suitable for analysis. This conversion includes saving to a database and converting to a specific format.

[0405] Step 2:

[0406] The device uses a camera and microphone to record and capture the user's voice and facial expressions. Input consists of audio and image data, while output is emotion data extracted through speech recognition and image analysis. This process utilizes Microsoft Azure's Emotion API and Face API to quantify and determine the emotional state.

[0407] Step 3:

[0408] The server inputs the acquired environmental and emotional data into a machine learning model for analysis. The input is the data obtained in the previous step, and the output is a behavioral indicator based on the user's emotional state and surrounding environmental conditions. The data is processed by a TensorFlow model, and an appropriate response is determined for each situation.

[0409] Step 4:

[0410] Based on the analysis results, the server determines the notification content and the device's operating mode. The input is the analysis results from step 3, and the output is the specific notification message and the settings information for the pest control system. For example, the pest control system may be set to silent mode, or the notification sound may be suppressed.

[0411] Step 5:

[0412] The user verifies the optimized device operation based on instructions from the server and makes manual adjustments as needed. Inputs are notifications and suggestions from the server, while outputs are the final system settings based on the user's choices. This process is facilitated through a user interface, providing a relaxed environment.

[0413] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0414] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0415] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0416] [Third Embodiment]

[0417] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0418] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0419] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0420] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0421] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0422] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0423] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0424] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0425] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0426] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0427] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0428] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0429] This invention is a fully automated cockroach extermination system incorporating AI technology. Specific embodiments are described below.

[0430] First, the system has a terminal equipped with multiple sensors that acquire indoor environmental data in real time. The terminal monitors environmental data, including temperature, humidity, and acoustic data, using sensor means, and uses artificial intelligence means to analyze this data. Based on the acquired data, the artificial intelligence means detects the presence of cockroaches with high accuracy.

[0431] When a cockroach is detected, the device's suction mechanism automatically activates, capturing the identified cockroach and sucking it into a sealed containment container. After capture, the environment inside the containment container is adjusted by the device's environmental control mechanism to ensure a dry environment and appropriate temperature, making it difficult for the cockroach to survive.

[0432] This system automatically moves a terminal to a processing station when the number of organisms in the containment container reaches a certain level, or periodically. The terminal, having been moved by the transport mechanism, uses processing equipment at the processing station to incinerate or pulverize the cockroaches in the containment container for final disposal.

[0433] Furthermore, the system is managed by a server, which constantly monitors the progress of tasks and the system's operational status. If maintenance is required based on the operational status, the server sends an alert to the user using a notification system. This allows users to receive notifications on devices such as smartphones and check the maintenance status and system status.

[0434] As a concrete example, when a terminal detects a cockroach, the server receives the analysis results from the AI ​​and establishes a processing protocol. The user checks the notified maintenance information and takes appropriate measures according to the situation. This system allows users to maintain a comfortable and hygienic environment without directly seeing the cockroaches.

[0435] The following describes the processing flow.

[0436] Step 1:

[0437] The device activates multiple sensors, such as infrared sensors and cameras, to acquire indoor environmental data. It collects information such as temperature, humidity, and motion data, and prepares it for analysis.

[0438] Step 2:

[0439] The device passes the acquired environmental data to artificial intelligence (AI) for data analysis. The AI ​​uses image recognition and pattern recognition to accurately detect cockroaches. The detection results are sent to a server.

[0440] Step 3:

[0441] The server receives cockroach detection information sent from the terminal and records it in the database. It also sends instructions to other system components as needed.

[0442] Step 4:

[0443] Once the device detects a cockroach, it activates the suction mechanism. It precisely points the suction nozzle at the cockroach's location, sucks it up, and seals it in a containment container.

[0444] Step 5:

[0445] The device controls the temperature and humidity inside the containment container after capture. This environmental control creates a harsh environment that makes it difficult for cockroaches to survive, encouraging early disposal.

[0446] Step 6:

[0447] The terminal monitors the status of the storage container until it is full, or periodically thereafter. Upon detecting fullness, it automatically activates the means of transporting the container to the processing station.

[0448] Step 7:

[0449] The terminal moves to the processing station, where it processes the cockroaches in the containment container. The contained cockroaches are safely disposed of in the incineration or crushing equipment at the processing station.

[0450] Step 8:

[0451] The server monitors the overall operating status of the system and records processing results. It also notifies users of maintenance needs and system status as required.

[0452] Step 9:

[0453] Users check notifications sent from the server via a smartphone app and perform simple maintenance tasks and system management based on the instructions.

[0454] (Example 1)

[0455] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0456] The occurrence of organisms indoors can lead to a deterioration of sanitary conditions and cause mental stress. Conventional pest control methods often involve manual work and have limited effectiveness. Furthermore, they lack speed because they take time to completely capture or remove the organisms. The objective of this invention is to solve these problems and enable automated and efficient management of organism control.

[0457] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0458] In this invention, the server includes detection means for acquiring environmental conditions, reasoning means for analyzing the acquired environmental conditions to identify organisms, and accumulation means for capturing the identified organisms and sealing them in a storage container. This makes it possible to effectively identify, automatically capture, and process organisms in a room.

[0459] "Environmental conditions" refer to physical and chemical properties, including data such as temperature, humidity, and sound waves in a room.

[0460] "Detection means" refers to sensor devices used to acquire environmental conditions, which enable real-time data collection.

[0461] "Inference means" refers to artificial intelligence technology used to analyze acquired environmental conditions and identify the presence and activity of living organisms.

[0462] "Accumulation means" refers to a mechanism for reliably capturing identified organisms and placing them into a sealed storage container.

[0463] "Regulatory means" refers to a device that allows for precise control of the environment inside a container, adjusting temperature, humidity, sound waves, etc., to make it difficult for living organisms to survive.

[0464] "Transfer means" refers to a mechanism that automatically moves the storage container to a designated processing unit.

[0465] "Processing means" refers to a device for removing or decomposing organisms captured in the processing unit.

[0466] "Notification methods" refer to technologies that monitor the system's status and send warnings to users in case of abnormalities or when regular maintenance is required.

[0467] This invention is a system for the automated extermination of organisms indoors and consists of multiple functional components. At the heart of the system are multiple terminals that acquire environmental data, equipped with temperature sensors, humidity sensors, and acoustic sensors. These terminals collect real-time data from the room and can efficiently analyze the data using AI technology.

[0468] The device uses dedicated AI software to analyze data and identify the presence of organisms. The AI ​​model used enables highly accurate organism detection by analyzing changes in sound frequency and temperature / humidity. Once detection occurs, the device activates a suction device as a means of collection, quickly capturing the identified organism and placing it in a sealed container.

[0469] Once captured, the environment within the container is controlled using regulatory mechanisms, adjusting temperature and humidity to make survival difficult. This process is automatically controlled by the system and carried out without any user intervention.

[0470] If the containment container meets certain conditions, the terminal is automatically moved to a processing station by a transport mechanism. At this processing station, the organism is incinerated or pulverized using processing equipment, and the processing is completed safely and hygienically.

[0471] The server manages the overall system status and constantly monitors data sent from terminals. If operational status or anomalies are detected, the server promptly alerts the user using notification methods. This allows users to perform maintenance at the necessary time, ensuring stable system operation.

[0472] As a concrete example, when a terminal detects a cockroach, the server receives the results of the AI ​​analysis and can set an appropriate processing protocol. An example of a prompt message would be, "Please explain in detail how the cockroach extermination system works. Please explain the procedure from data acquisition by sensors to AI analysis, cockroach capture, and final processing." In this way, this invention supports the optimization of the indoor environment and the improvement of user comfort.

[0473] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0474] Step 1:

[0475] The device operates temperature, humidity, and acoustic sensors to acquire environmental conditions. It receives indoor temperature, humidity, and acoustic data as input, collecting it in real time. As output, this data is ready to be sent to an AI system. This process makes it possible to understand the overall picture of the indoor environment.

[0476] Step 2:

[0477] The device passes the acquired environmental data to an AI model for data analysis. Using the various sensor data collected earlier as input, the AI ​​model identifies specific patterns. The output generates a result indicating whether or not abnormal patterns that may indicate cockroach activity have been found. This enables highly accurate detection of the presence of cockroaches.

[0478] Step 3:

[0479] The device activates the suction device if a cockroach is detected based on the AI ​​analysis results. It receives the confidence level of detection as input and automatically activates the suction mechanism. The output is the capture of the target cockroach in the containment container. The specific operation involves this process, including adjusting the suction force and setting the path for capture.

[0480] Step 4:

[0481] The terminal adjusts the environment inside the containment container where the captured cockroaches are stored. It receives current data on the conditions inside the container as input and performs operations such as lowering humidity or changing the temperature. The output is the maintenance of environmental conditions that make it difficult for cockroaches to survive, thereby preventing the organism from re-infesting the area.

[0482] Step 5:

[0483] The terminal automatically transports the containment container to the processing station when it meets certain conditions. Its input is a count of the total number of organisms in the container, and it determines if a certain quantity has been reached. The output is a notification that the transport is complete and the container has reached the processing station.

[0484] Step 6:

[0485] Upon reaching the processing station, the terminal initiates the process of disposing of the cockroaches internally. As input, it receives the current status of the processing station and selects either incineration or crushing as the process. As output, the organisms are safely disposed of. At this stage, the terminal completes the task of properly disposing of the organisms.

[0486] Step 7:

[0487] The server monitors the overall system status and notifies users if an anomaly is detected. It receives operational data from terminals, detection results, and station status information as input. It sends alert notifications to users as output, allowing them to be informed of the need for maintenance or emergency response.

[0488] (Application Example 1)

[0489] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0490] In large-scale facilities such as logistics centers, there is a need to quickly and accurately detect cockroaches, which are invertebrates, and to efficiently exterminate them without human intervention. However, conventional technologies have made it difficult to quickly detect invertebrates and provide information after extermination, which has made it challenging to maintain hygiene. Furthermore, quickly and accurately conveying information to on-site personnel has also been a challenge.

[0491] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0492] In this invention, the server includes a judgment device that analyzes environmental data and detects invertebrates, a collection device and processing device that capture and process invertebrates, and a display means that provides information using a portable display device. This enables rapid detection and extermination of invertebrates, and information is provided to the person in charge in real time, thereby improving hygiene management of the facility.

[0493] A "detection device" is a device that senses the physical characteristics of the environment and acquires that data.

[0494] A "judgment device" is a device that analyzes the presence of invertebrates based on acquired data and identifies their presence.

[0495] A "collection device" is a device used to capture detected invertebrates and seal them in a designated storage location.

[0496] A "control device" is a device that adjusts the environment inside the containment container to make it difficult for invertebrates to survive.

[0497] A "mobilization device" is a device used to automatically move a system to a designated processing location.

[0498] A "processing device" is a device used to physically process transported invertebrates.

[0499] A "communication device" is a device used to transmit system status and information to a user or other device.

[0500] "Display means" refers to a means of providing information to a user visually using a portable display device.

[0501] To carry out this invention, the following system and process are used.

[0502] The system mainly consists of a decision-making device, a detection device, a data collection device, a control device, a mobile device, a processing device, a communication device, and a display means. The server controls data processing and communication between these devices.

[0503] The detection device acquires environmental data such as temperature, humidity, and sound waves in real time. This provides data to confirm the presence of invertebrates (cockroaches).

[0504] The detection system uses AI models designed with Python's Scikit-learn and TensorFlow to analyze this environmental data and estimate the presence of invertebrates with high accuracy. If detected, the information is aggregated on a server, and appropriate extermination processes are instructed.

[0505] The collection device activates to capture detected invertebrates and seals them in a containment container. During this process, an environmental control system adjusts the temperature and humidity inside the container to make it difficult for the invertebrates to survive.

[0506] When the containment container is full, the transport device automatically moves to the processing station, where the processing unit physically processes the invertebrates.

[0507] Meanwhile, the communication device transmits information such as the system's operating status and pest control progress from the server to the user's portable display device, such as smart glasses. Through this display device, the user on site can immediately receive the information and take the necessary actions.

[0508] As a concrete example, in a logistics center environment, if the system detects a single cockroach, its location information is immediately displayed on the user's smart glasses. The user can see the cockroach's location through the glasses and monitor how the extermination device automatically responds. This enables efficient and streamlined hygiene management.

[0509] An example of a prompt might be: "Provide prompts for developing a real-time AI model for cockroach detection in a logistics center. The model will use temperature, humidity, and acoustic data to detect the presence of cockroaches with high accuracy."

[0510] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0511] Step 1:

[0512] The server receives environmental data from the sensors. This data includes temperature, humidity, and sound wave data. The server temporarily stores this data in storage.

[0513] Step 2:

[0514] The server inputs stored environmental data into an AI model to analyze the presence of invertebrates. The AI ​​model analyzes the data using a generative AI model and outputs the probability of invertebrate presence. This provides a data-based confidence level for detection.

[0515] Step 3:

[0516] When the server detects that the confidence level exceeds a predetermined threshold, it sends a capture command to the terminal. The terminal then activates its collection device and captures the invertebrate at the detected location. This process is performed automatically based on the command generated by the server.

[0517] Step 4:

[0518] The terminal seals the captured invertebrate in a containment container. The environment inside the containment container is maintained in a state that suppresses the survival of the invertebrate by adjusting the temperature and humidity using a control system.

[0519] Step 5:

[0520] The server monitors the fullness of the containment containers and issues a move command if fullness is confirmed. Based on this command, the terminal automatically moves to the processing site and incinerates or grinds the invertebrates in the processing unit.

[0521] Step 6:

[0522] The server monitors the system's operational status in real time and transmits information to the user's portable display device (e.g., smart glasses). The user then uses the displayed information to confirm the situation on-site and take appropriate action.

[0523] Step 7:

[0524] The server generates prompt messages to provide information to the user and, if necessary, suggests further actions. These prompt messages may include information such as, "Please provide prompts for developing a real-time AI model for cockroach detection in a logistics center," to support subsequent work.

[0525] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0526] The present invention is a cockroach extermination system that incorporates an emotion engine that recognizes the user's emotions and dynamically adjusts the operation of the device based on those emotions. In this embodiment, the emotion engine recognizes the user's emotions from voice and facial expression data and optimizes the operation of the system accordingly.

[0527] The device is equipped with multiple sensors that it uses to capture the user's voice and facial expressions. The emotion engine analyzes the acquired data to determine the user's emotional state. For example, if the user is showing discomfort, the alert content of the notification system can be adjusted to avoid excessive warning sounds and change to a more relaxing message.

[0528] Furthermore, this system simultaneously acquires and analyzes environmental data to detect the presence of cockroaches. Once detected, the device can prioritize or quietly activate suction methods based on the results of the emotion engine. This allows for effective extermination while minimizing the user's sense of urgency.

[0529] As an example of this system, consider a scenario where a cockroach is detected while a user is relaxing late at night. In this case, the device will activate the suction device in silent mode and adjust the alert notification to be less frequent. The server will then record the subsequent processing status and set up a notification to the user the following morning. In this way, by considering the user's emotional state, the system provides a more comfortable and adaptive living environment.

[0530] The following describes the processing flow.

[0531] Step 1:

[0532] The device uses a microphone and camera to scan the surrounding environment in order to acquire the user's voice and facial expression data. This data is then sent to the emotion engine.

[0533] Step 2:

[0534] The device's emotion engine analyzes acquired voice and facial expression data to identify the user's emotional state. Based on this analysis, it determines whether the user is "relaxed" or "stressed."

[0535] Step 3:

[0536] The device uses sensors to acquire indoor environmental data. It measures temperature, humidity, and motion data, and uses artificial intelligence to detect cockroaches.

[0537] Step 4:

[0538] When a cockroach is detected, the server considers the results of the emotion engine's analysis to decide how to proceed with the extermination. For example, if the user is relaxed, the device will be instructed to activate the suction device in silent mode.

[0539] Step 5:

[0540] The device activates its suction mechanism to capture detected cockroaches. Throughout this process, it uses information from its emotion engine to perform extermination actions that minimize user stress.

[0541] Step 6:

[0542] The terminal monitors the situation inside the containment container. It continues monitoring until the container is full or the extermination process is complete.

[0543] Step 7:

[0544] The server will send a notification to the user once the removal process is complete. However, if the user is resting, the notification can be changed to a less prominent alert or postponed until the following morning. The user can check the notification on their smartphone or other device and take appropriate action.

[0545] (Example 2)

[0546] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0547] In systems that detect the presence of living organisms, there is a need for methods that consider the user's emotional state and efficiently eliminate organisms without causing stress to the user. Conventional pest control systems do not adjust their operation to take user emotions into consideration, and the sounds and notifications during extermination may cause discomfort to the user. Therefore, the challenge is to achieve flexible system control that responds to the user's emotional state.

[0548] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0549] In this invention, the server includes sensor means for acquiring environmental data, data collection means for acquiring user voice and facial expression data, and emotion recognition means for analyzing the acquired voice and facial expression data and determining the emotional state. This makes it possible to optimize system operation and adjust notification content according to the user's emotional state.

[0550] "Environmental data" refers to data that includes information such as ambient temperature and movement, used to detect the presence of living organisms.

[0551] "Sensor means" refers to various sensors used to acquire environmental data and user data, and consists of temperature sensors, motion detection sensors, and devices necessary for acquiring audio and video.

[0552] "Information processing means" refers to computer systems and algorithms used to analyze acquired data and identify the presence of living organisms.

[0553] "Data collection means" refers to a system consisting of devices and programs for acquiring data such as the user's voice and facial expressions.

[0554] An "emotion recognition system" is a system that includes machine learning models and algorithms for analyzing collected data and identifying the user's emotional state.

[0555] "Control means" refers to a mechanism for adjusting the system's operating mode, sound, and notification content based on the user's emotional state and biological detection information.

[0556] A "suction device" is a device used to physically capture an organism and seal it inside a specific container.

[0557] A "notification method" is a means of communicating the system status and detection information to the user through sound or video.

[0558] This invention is a biological pest control system that can flexibly adjust its operation based on the user's emotional state. The system has the ability to recognize the user's voice and facial expressions and change its operation accordingly. The specific implementation method is described below.

[0559] This system incorporates multiple sensor devices on the terminal to collect user voice and facial expression data. Common APIs and libraries are used for voice recognition, while image processing libraries are utilized for facial expression recognition. Specifically, the microphone and camera function as data input devices. Using these devices, the terminal captures raw user data and inputs it into an emotion recognition algorithm.

[0560] The emotion recognition algorithm employs a generative AI model. The server uses this model to process data in real time and analyze the user's emotional state. For example, if the system detects that the user is relaxed, it is controlled to operate in silent mode.

[0561] The device also collects ambient environmental data in parallel and uses information processing to detect the presence of living organisms. When a target organism is detected, the system immediately executes the suction mechanism, but the sound and notification content at that time are adjusted to reflect the results of emotion recognition.

[0562] For example, if the system detects a living organism while the user is relaxing late at night, the device will quietly perform the suction process, and the user will receive a discreet notification. The server will then record the processing log and provide a notification at a time of the user's choosing.

[0563] Examples of prompts to input into a generative AI model:

[0564] "Based on user sentiment data, how can we optimize the device's behavior? Please propose specific adjustments."

[0565] Thus, the present invention realizes biological pest control in an environment that takes user experience into maximum consideration.

[0566] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0567] Step 1:

[0568] The device uses its built-in microphone and camera to acquire user voice and facial expression data. The input is the user's real-time voice and video stream, and the output is generated as audio and image data. This data is then sent directly to the next processing step.

[0569] Step 2:

[0570] The device converts voice data into text data via a speech recognition API and analyzes facial expression data using an image processing library. The input is the raw data obtained in step 1, and the output is information converted into emotion labels (e.g., "relaxed," "stressed," etc.). This allows for an initial determination of the user's emotional state.

[0571] Step 3:

[0572] The server uses a generative AI model to analyze the text data and emotion label data acquired in step 2 to determine the user's detailed emotional state. The input is the emotion label data from step 2, and the output is the accuracy of the emotion and the optimal response strategy. Based on this, the system's operation is adjusted.

[0573] Step 4:

[0574] The terminal utilizes temperature and motion detection sensors to acquire environmental data. Sensor data from the surrounding environment is used as input, and biological detection information is generated as output. This information is immediately analyzed within the system, and pest control measures are taken as needed.

[0575] Step 5:

[0576] The device activates the suction mechanism in silent mode based on the user's emotional state and biological detection information. The input is the emotional analysis result from step 3 and the biological detection result from step 4, and the output is a silent and effective suction operation. This captures organisms without causing excessive stress to the user.

[0577] Step 6:

[0578] The server monitors all processes and is configured to send notifications to the user as needed. Log data from all steps is used as input, and the output consists of processing history and notification information provided to the user. For example, processing results from late-night operations are communicated to the user the following morning. This entire process enables comfortable and efficient pest control while considering the user's feelings.

[0579] (Application Example 2)

[0580] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0581] Conventional pest control systems had the problem of causing unnecessary stress to users through extermination activities without considering their emotions. Furthermore, recognizing the organisms and properly handling them after capture was difficult, making effective extermination challenging. In addition, adjusting services to customer emotions in physical stores was difficult, creating room for improvement in customer satisfaction. Therefore, there was a need for a system that could recognize user emotions and dynamically adjust device operation.

[0582] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0583] In this invention, the server includes detection means for acquiring environmental information, machine learning means for analyzing the acquired environmental information to recognize organisms, a suction device for capturing the recognized organisms and sealing them in a containment device, acquisition means for acquiring emotional data, an analysis device for analyzing the acquired emotional data to determine the user's emotional state, and adjustment means for dynamically adjusting notification content based on the emotional state. This enables effective and less stressful pest control activities and service improvements while taking into account the user's emotions and the state of customers in the store.

[0584] "Environmental information" refers to information about the surroundings of the subject, and includes data such as temperature, humidity, and sound waves.

[0585] "Detection means" refers to devices used to acquire environmental information, and includes various sensors.

[0586] A "machine learning device" is a device that uses a learning algorithm to analyze acquired environmental information and recognize the presence of living organisms.

[0587] A "suction device" is a device used to capture recognized organisms and seal them inside a containment device.

[0588] A "containment device" is a device used to temporarily store captured organisms and has a structure that allows for environmental control.

[0589] "Emotional data" refers to information obtained from the user's voice and facial expressions, and serves as the basis for inferring their emotional state.

[0590] "Means of acquisition" refers to devices used to collect emotional data, including cameras and microphones.

[0591] An "analysis device" is a device that analyzes acquired emotional data to determine the user's emotional state.

[0592] A "modification device" is a device that dynamically changes notification content and extermination activities based on emotional state.

[0593] To implement this invention, a system is utilized in which a server collects environmental information and emotional data and analyzes them. This system is equipped with a combination of sensors and analytical algorithms, and specifically includes smart glasses and dedicated devices.

[0594] The server uses various sensors to acquire environmental information (temperature, humidity, sound waves, etc.) in real time. These include infrared sensors and ultrasonic sensors. Furthermore, a device integrating a microphone and camera is used to acquire emotional data from the user's voice and facial expressions. This device utilizes computer vision and speech recognition technologies to accurately determine the user's emotional state.

[0595] The acquired data is analyzed by a machine learning model on the server (e.g., a deep learning model using Python and TensorFlow). This model achieves highly accurate emotion recognition by utilizing services such as Microsoft Azure's Emotion API and Face API.

[0596] Based on the analysis results, the server dynamically adjusts notification content and extermination strategies. For example, if the user is relaxed, the notification sound is suppressed and the suction device operates in silent mode. This allows for efficient extermination without causing unnecessary stress to the user.

[0597] As a concrete example, considering its application in a retail setting, this system can recognize the emotional state of customers and automatically optimize service. By using this system, users can increase customer satisfaction and provide a less stressful environment.

[0598] An example of a prompt message could be, "If a customer is showing signs of stress in the store, please suggest how to flexibly adjust your service style." This auxiliary function provides store employees and staff with guidance to provide appropriate responses.

[0599] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0600] Step 1:

[0601] The server acquires ambient environmental information (temperature, humidity, sound waves, etc.) using sensors embedded in smart glasses or dedicated devices. The input is raw data from the sensors, and the output is environmental data that has been organized and converted into a format suitable for analysis. This conversion includes saving to a database and converting to a specific format.

[0602] Step 2:

[0603] The device uses a camera and microphone to record and capture the user's voice and facial expressions. Input consists of audio and image data, while output is emotion data extracted through speech recognition and image analysis. This process utilizes Microsoft Azure's Emotion API and Face API to quantify and determine the emotional state.

[0604] Step 3:

[0605] The server inputs the acquired environmental and emotional data into a machine learning model for analysis. The input is the data obtained in the previous step, and the output is a behavioral indicator based on the user's emotional state and surrounding environmental conditions. The data is processed by a TensorFlow model, and an appropriate response is determined for each situation.

[0606] Step 4:

[0607] Based on the analysis results, the server determines the notification content and the device's operating mode. The input is the analysis results from step 3, and the output is the specific notification message and the settings information for the pest control system. For example, the pest control system may be set to silent mode, or the notification sound may be suppressed.

[0608] Step 5:

[0609] The user verifies the optimized device operation based on instructions from the server and makes manual adjustments as needed. Inputs are notifications and suggestions from the server, while outputs are the final system settings based on the user's choices. This process is facilitated through a user interface, providing a relaxed environment.

[0610] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0611] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0612] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0613] [Fourth Embodiment]

[0614] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0615] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0616] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0617] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0618] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0619] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0620] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0621] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0622] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0623] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0624] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0625] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0626] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0627] This invention is a fully automated cockroach extermination system incorporating AI technology. Specific embodiments are described below.

[0628] First, the system has a terminal equipped with multiple sensors that acquire indoor environmental data in real time. The terminal monitors environmental data, including temperature, humidity, and acoustic data, using sensor means, and uses artificial intelligence means to analyze this data. Based on the acquired data, the artificial intelligence means detects the presence of cockroaches with high accuracy.

[0629] When a cockroach is detected, the device's suction mechanism automatically activates, capturing the identified cockroach and sucking it into a sealed containment container. After capture, the environment inside the containment container is adjusted by the device's environmental control mechanism to ensure a dry environment and appropriate temperature, making it difficult for the cockroach to survive.

[0630] This system automatically moves a terminal to a processing station when the number of organisms in the containment container reaches a certain level, or periodically. The terminal, having been moved by the transport mechanism, uses processing equipment at the processing station to incinerate or pulverize the cockroaches in the containment container for final disposal.

[0631] Furthermore, the system is managed by a server, which constantly monitors the progress of tasks and the system's operational status. If maintenance is required based on the operational status, the server sends an alert to the user using a notification system. This allows users to receive notifications on devices such as smartphones and check the maintenance status and system status.

[0632] As a concrete example, when a terminal detects a cockroach, the server receives the analysis results from the AI ​​and establishes a processing protocol. The user checks the notified maintenance information and takes appropriate measures according to the situation. This system allows users to maintain a comfortable and hygienic environment without directly seeing the cockroaches.

[0633] The following describes the processing flow.

[0634] Step 1:

[0635] The device activates multiple sensors, such as infrared sensors and cameras, to acquire indoor environmental data. It collects information such as temperature, humidity, and motion data, and prepares it for analysis.

[0636] Step 2:

[0637] The device passes the acquired environmental data to artificial intelligence (AI) for data analysis. The AI ​​uses image recognition and pattern recognition to accurately detect cockroaches. The detection results are sent to a server.

[0638] Step 3:

[0639] The server receives cockroach detection information sent from the terminal and records it in the database. It also sends instructions to other system components as needed.

[0640] Step 4:

[0641] Once the device detects a cockroach, it activates the suction mechanism. It precisely points the suction nozzle at the cockroach's location, sucks it up, and seals it in a containment container.

[0642] Step 5:

[0643] The device controls the temperature and humidity inside the containment container after capture. This environmental control creates a harsh environment that makes it difficult for cockroaches to survive, encouraging early disposal.

[0644] Step 6:

[0645] The terminal monitors the status of the storage container until it is full, or periodically thereafter. Upon detecting fullness, it automatically activates the means of transporting the container to the processing station.

[0646] Step 7:

[0647] The terminal moves to the processing station, where it processes the cockroaches in the containment container. The contained cockroaches are safely disposed of in the incineration or crushing equipment at the processing station.

[0648] Step 8:

[0649] The server monitors the overall operating status of the system and records processing results. It also notifies users of maintenance needs and system status as required.

[0650] Step 9:

[0651] Users check notifications sent from the server via a smartphone app and perform simple maintenance tasks and system management based on the instructions.

[0652] (Example 1)

[0653] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0654] The occurrence of organisms indoors can lead to a deterioration of sanitary conditions and cause mental stress. Conventional pest control methods often involve manual work and have limited effectiveness. Furthermore, they lack speed because they take time to completely capture or remove the organisms. The objective of this invention is to solve these problems and enable automated and efficient management of organism control.

[0655] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0656] In this invention, the server includes detection means for acquiring environmental conditions, reasoning means for analyzing the acquired environmental conditions to identify organisms, and accumulation means for capturing the identified organisms and sealing them in a storage container. This makes it possible to effectively identify, automatically capture, and process organisms in a room.

[0657] "Environmental conditions" refer to physical and chemical properties, including data such as temperature, humidity, and sound waves in a room.

[0658] "Detection means" refers to sensor devices used to acquire environmental conditions, which enable real-time data collection.

[0659] "Inference means" refers to artificial intelligence technology used to analyze acquired environmental conditions and identify the presence and activity of living organisms.

[0660] "Accumulation means" refers to a mechanism for reliably capturing identified organisms and placing them into a sealed storage container.

[0661] "Regulatory means" refers to a device that allows for precise control of the environment inside a container, adjusting temperature, humidity, sound waves, etc., to make it difficult for living organisms to survive.

[0662] "Transfer means" refers to a mechanism that automatically moves the storage container to a designated processing unit.

[0663] "Processing means" refers to a device for removing or decomposing organisms captured in the processing unit.

[0664] "Notification methods" refer to technologies that monitor the system's status and send warnings to users in case of abnormalities or when regular maintenance is required.

[0665] This invention is a system for the automated extermination of organisms indoors and consists of multiple functional components. At the heart of the system are multiple terminals that acquire environmental data, equipped with temperature sensors, humidity sensors, and acoustic sensors. These terminals collect real-time data from the room and can efficiently analyze the data using AI technology.

[0666] The device uses dedicated AI software to analyze data and identify the presence of organisms. The AI ​​model used enables highly accurate organism detection by analyzing changes in sound frequency and temperature / humidity. Once detection occurs, the device activates a suction device as a means of collection, quickly capturing the identified organism and placing it in a sealed container.

[0667] Once captured, the environment within the container is controlled using regulatory mechanisms, adjusting temperature and humidity to make survival difficult. This process is automatically controlled by the system and carried out without any user intervention.

[0668] If the containment container meets certain conditions, the terminal is automatically moved to a processing station by a transport mechanism. At this processing station, the organism is incinerated or pulverized using processing equipment, and the processing is completed safely and hygienically.

[0669] The server manages the overall system status and constantly monitors data sent from terminals. If operational status or anomalies are detected, the server promptly alerts the user using notification methods. This allows users to perform maintenance at the necessary time, ensuring stable system operation.

[0670] As a concrete example, when a terminal detects a cockroach, the server receives the results of the AI ​​analysis and can set an appropriate processing protocol. An example of a prompt message would be, "Please explain in detail how the cockroach extermination system works. Please explain the procedure from data acquisition by sensors to AI analysis, cockroach capture, and final processing." In this way, this invention supports the optimization of the indoor environment and the improvement of user comfort.

[0671] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0672] Step 1:

[0673] The device operates temperature, humidity, and acoustic sensors to acquire environmental conditions. It receives indoor temperature, humidity, and acoustic data as input, collecting it in real time. As output, this data is ready to be sent to an AI system. This process makes it possible to understand the overall picture of the indoor environment.

[0674] Step 2:

[0675] The device passes the acquired environmental data to an AI model for data analysis. Using the various sensor data collected earlier as input, the AI ​​model identifies specific patterns. The output generates a result indicating whether or not abnormal patterns that may indicate cockroach activity have been found. This enables highly accurate detection of the presence of cockroaches.

[0676] Step 3:

[0677] The device activates the suction device if a cockroach is detected based on the AI ​​analysis results. It receives the confidence level of detection as input and automatically activates the suction mechanism. The output is the capture of the target cockroach in the containment container. The specific operation involves this process, including adjusting the suction force and setting the path for capture.

[0678] Step 4:

[0679] The terminal adjusts the environment inside the containment container where the captured cockroaches are stored. It receives current data on the conditions inside the container as input and performs operations such as lowering humidity or changing the temperature. The output is the maintenance of environmental conditions that make it difficult for cockroaches to survive, thereby preventing the organism from re-infesting the area.

[0680] Step 5:

[0681] The terminal automatically transports the containment container to the processing station when it meets certain conditions. Its input is a count of the total number of organisms in the container, and it determines if a certain quantity has been reached. The output is a notification that the transport is complete and the container has reached the processing station.

[0682] Step 6:

[0683] Upon reaching the processing station, the terminal initiates the process of disposing of the cockroaches internally. As input, it receives the current status of the processing station and selects either incineration or crushing as the process. As output, the organisms are safely disposed of. At this stage, the terminal completes the task of properly disposing of the organisms.

[0684] Step 7:

[0685] The server monitors the overall system status and notifies users if an anomaly is detected. It receives operational data from terminals, detection results, and station status information as input. It sends alert notifications to users as output, allowing them to be informed of the need for maintenance or emergency response.

[0686] (Application Example 1)

[0687] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0688] In large-scale facilities such as logistics centers, there is a need to quickly and accurately detect cockroaches, which are invertebrates, and to efficiently exterminate them without human intervention. However, conventional technologies have made it difficult to quickly detect invertebrates and provide information after extermination, which has made it challenging to maintain hygiene. Furthermore, quickly and accurately conveying information to on-site personnel has also been a challenge.

[0689] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0690] In this invention, the server includes a judgment device that analyzes environmental data and detects invertebrates, a collection device and processing device that capture and process invertebrates, and a display means that provides information using a portable display device. This enables rapid detection and extermination of invertebrates, and information is provided to the person in charge in real time, thereby improving hygiene management of the facility.

[0691] A "detection device" is a device that senses the physical characteristics of the environment and acquires that data.

[0692] A "judgment device" is a device that analyzes the presence of invertebrates based on acquired data and identifies their presence.

[0693] A "collection device" is a device used to capture detected invertebrates and seal them in a designated storage location.

[0694] A "control device" is a device that adjusts the environment inside the containment container to make it difficult for invertebrates to survive.

[0695] A "mobilization device" is a device used to automatically move a system to a designated processing location.

[0696] A "processing device" is a device used to physically process transported invertebrates.

[0697] A "communication device" is a device used to transmit system status and information to a user or other device.

[0698] "Display means" refers to a means of providing information to a user visually using a portable display device.

[0699] To carry out this invention, the following system and process are used.

[0700] The system mainly consists of a decision-making device, a detection device, a data collection device, a control device, a mobile device, a processing device, a communication device, and a display means. The server controls data processing and communication between these devices.

[0701] The detection device acquires environmental data such as temperature, humidity, and sound waves in real time. This provides data to confirm the presence of invertebrates (cockroaches).

[0702] The detection system uses AI models designed with Python's Scikit-learn and TensorFlow to analyze this environmental data and estimate the presence of invertebrates with high accuracy. If detected, the information is aggregated on a server, and appropriate extermination processes are instructed.

[0703] The collection device activates to capture detected invertebrates and seals them in a containment container. During this process, an environmental control system adjusts the temperature and humidity inside the container to make it difficult for the invertebrates to survive.

[0704] When the containment container is full, the transport device automatically moves to the processing station, where the processing unit physically processes the invertebrates.

[0705] Meanwhile, the communication device transmits information such as the system's operating status and pest control progress from the server to the user's portable display device, such as smart glasses. Through this display device, the user on site can immediately receive the information and take the necessary actions.

[0706] As a concrete example, in a logistics center environment, if the system detects a single cockroach, its location information is immediately displayed on the user's smart glasses. The user can see the cockroach's location through the glasses and monitor how the extermination device automatically responds. This enables efficient and streamlined hygiene management.

[0707] An example of a prompt might be: "Provide prompts for developing a real-time AI model for cockroach detection in a logistics center. The model will use temperature, humidity, and acoustic data to detect the presence of cockroaches with high accuracy."

[0708] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0709] Step 1:

[0710] The server receives environmental data from the sensors. This data includes temperature, humidity, and sound wave data. The server temporarily stores this data in storage.

[0711] Step 2:

[0712] The server inputs stored environmental data into an AI model to analyze the presence of invertebrates. The AI ​​model analyzes the data using a generative AI model and outputs the probability of invertebrate presence. This provides a data-based confidence level for detection.

[0713] Step 3:

[0714] When the server detects that the confidence level exceeds a predetermined threshold, it sends a capture command to the terminal. The terminal then activates its collection device and captures the invertebrate at the detected location. This process is performed automatically based on the command generated by the server.

[0715] Step 4:

[0716] The terminal seals the captured invertebrate in a containment container. The environment inside the containment container is maintained in a state that suppresses the survival of the invertebrate by adjusting the temperature and humidity using a control system.

[0717] Step 5:

[0718] The server monitors the fullness of the containment containers and issues a move command if fullness is confirmed. Based on this command, the terminal automatically moves to the processing site and incinerates or grinds the invertebrates in the processing unit.

[0719] Step 6:

[0720] The server monitors the system's operational status in real time and transmits information to the user's portable display device (e.g., smart glasses). The user then uses the displayed information to confirm the situation on-site and take appropriate action.

[0721] Step 7:

[0722] The server generates prompt messages to provide information to the user and, if necessary, suggests further actions. These prompt messages may include information such as, "Please provide prompts for developing a real-time AI model for cockroach detection in a logistics center," to support subsequent work.

[0723] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0724] The present invention is a cockroach extermination system that incorporates an emotion engine that recognizes the user's emotions and dynamically adjusts the operation of the device based on those emotions. In this embodiment, the emotion engine recognizes the user's emotions from voice and facial expression data and optimizes the operation of the system accordingly.

[0725] The device is equipped with multiple sensors that it uses to capture the user's voice and facial expressions. The emotion engine analyzes the acquired data to determine the user's emotional state. For example, if the user is showing discomfort, the alert content of the notification system can be adjusted to avoid excessive warning sounds and change to a more relaxing message.

[0726] Furthermore, this system simultaneously acquires and analyzes environmental data to detect the presence of cockroaches. Once detected, the device can prioritize or quietly activate suction methods based on the results of the emotion engine. This allows for effective extermination while minimizing the user's sense of urgency.

[0727] As an example of this system, consider a scenario where a cockroach is detected while a user is relaxing late at night. In this case, the device will activate the suction device in silent mode and adjust the alert notification to be less frequent. The server will then record the subsequent processing status and set up a notification to the user the following morning. In this way, by considering the user's emotional state, the system provides a more comfortable and adaptive living environment.

[0728] The following describes the processing flow.

[0729] Step 1:

[0730] The device uses a microphone and camera to scan the surrounding environment in order to acquire the user's voice and facial expression data. This data is then sent to the emotion engine.

[0731] Step 2:

[0732] The device's emotion engine analyzes acquired voice and facial expression data to identify the user's emotional state. Based on this analysis, it determines whether the user is "relaxed" or "stressed."

[0733] Step 3:

[0734] The device uses sensors to acquire indoor environmental data. It measures temperature, humidity, and motion data, and uses artificial intelligence to detect cockroaches.

[0735] Step 4:

[0736] When a cockroach is detected, the server considers the results of the emotion engine's analysis to decide how to proceed with the extermination. For example, if the user is relaxed, the device will be instructed to activate the suction device in silent mode.

[0737] Step 5:

[0738] The device activates its suction mechanism to capture detected cockroaches. Throughout this process, it uses information from its emotion engine to perform extermination actions that minimize user stress.

[0739] Step 6:

[0740] The terminal monitors the situation inside the containment container. It continues monitoring until the container is full or the extermination process is complete.

[0741] Step 7:

[0742] The server will send a notification to the user once the removal process is complete. However, if the user is resting, the notification can be changed to a less prominent alert or postponed until the following morning. The user can check the notification on their smartphone or other device and take appropriate action.

[0743] (Example 2)

[0744] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0745] In systems that detect the presence of living organisms, there is a need for methods that consider the user's emotional state and efficiently eliminate organisms without causing stress to the user. Conventional pest control systems do not adjust their operation to take user emotions into consideration, and the sounds and notifications during extermination may cause discomfort to the user. Therefore, the challenge is to achieve flexible system control that responds to the user's emotional state.

[0746] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0747] In this invention, the server includes sensor means for acquiring environmental data, data collection means for acquiring user voice and facial expression data, and emotion recognition means for analyzing the acquired voice and facial expression data and determining the emotional state. This makes it possible to optimize system operation and adjust notification content according to the user's emotional state.

[0748] "Environmental data" refers to data that includes information such as ambient temperature and movement, used to detect the presence of living organisms.

[0749] "Sensor means" refers to various sensors used to acquire environmental data and user data, and consists of temperature sensors, motion detection sensors, and devices necessary for acquiring audio and video.

[0750] "Information processing means" refers to computer systems and algorithms used to analyze acquired data and identify the presence of living organisms.

[0751] "Data collection means" refers to a system consisting of devices and programs for acquiring data such as the user's voice and facial expressions.

[0752] An "emotion recognition system" is a system that includes machine learning models and algorithms for analyzing collected data and identifying the user's emotional state.

[0753] "Control means" refers to a mechanism for adjusting the system's operating mode, sound, and notification content based on the user's emotional state and biological detection information.

[0754] A "suction device" is a device used to physically capture an organism and seal it inside a specific container.

[0755] A "notification method" is a means of communicating the system status and detection information to the user through sound or video.

[0756] This invention is a biological pest control system that can flexibly adjust its operation based on the user's emotional state. The system has the ability to recognize the user's voice and facial expressions and change its operation accordingly. The specific implementation method is described below.

[0757] This system incorporates multiple sensor devices on the terminal to collect user voice and facial expression data. Common APIs and libraries are used for voice recognition, while image processing libraries are utilized for facial expression recognition. Specifically, the microphone and camera function as data input devices. Using these devices, the terminal captures raw user data and inputs it into an emotion recognition algorithm.

[0758] The emotion recognition algorithm employs a generative AI model. The server uses this model to process data in real time and analyze the user's emotional state. For example, if the system detects that the user is relaxed, it is controlled to operate in silent mode.

[0759] The device also collects ambient environmental data in parallel and uses information processing to detect the presence of living organisms. When a target organism is detected, the system immediately executes the suction mechanism, but the sound and notification content at that time are adjusted to reflect the results of emotion recognition.

[0760] For example, if the system detects a living organism while the user is relaxing late at night, the device will quietly perform the suction process, and the user will receive a discreet notification. The server will then record the processing log and provide a notification at a time of the user's choosing.

[0761] Examples of prompts to input into a generative AI model:

[0762] "Based on user sentiment data, how can we optimize the device's behavior? Please propose specific adjustments."

[0763] Thus, the present invention realizes biological pest control in an environment that takes user experience into maximum consideration.

[0764] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0765] Step 1:

[0766] The device uses its built-in microphone and camera to acquire user voice and facial expression data. The input is the user's real-time voice and video stream, and the output is generated as audio and image data. This data is then sent directly to the next processing step.

[0767] Step 2:

[0768] The device converts voice data into text data via a speech recognition API and analyzes facial expression data using an image processing library. The input is the raw data obtained in step 1, and the output is information converted into emotion labels (e.g., "relaxed," "stressed," etc.). This allows for an initial determination of the user's emotional state.

[0769] Step 3:

[0770] The server uses a generative AI model to analyze the text data and emotion label data acquired in step 2 to determine the user's detailed emotional state. The input is the emotion label data from step 2, and the output is the accuracy of the emotion and the optimal response strategy. Based on this, the system's operation is adjusted.

[0771] Step 4:

[0772] The terminal utilizes temperature and motion detection sensors to acquire environmental data. Sensor data from the surrounding environment is used as input, and biological detection information is generated as output. This information is immediately analyzed within the system, and pest control measures are taken as needed.

[0773] Step 5:

[0774] The device activates the suction mechanism in silent mode based on the user's emotional state and biological detection information. The input is the emotional analysis result from step 3 and the biological detection result from step 4, and the output is a silent and effective suction operation. This captures organisms without causing excessive stress to the user.

[0775] Step 6:

[0776] The server monitors all processes and is configured to send notifications to the user as needed. Log data from all steps is used as input, and the output consists of processing history and notification information provided to the user. For example, processing results from late-night operations are communicated to the user the following morning. This entire process enables comfortable and efficient pest control while considering the user's feelings.

[0777] (Application Example 2)

[0778] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0779] Conventional pest control systems had the problem of causing unnecessary stress to users through extermination activities without considering their emotions. Furthermore, recognizing the organisms and properly handling them after capture was difficult, making effective extermination challenging. In addition, adjusting services to customer emotions in physical stores was difficult, creating room for improvement in customer satisfaction. Therefore, there was a need for a system that could recognize user emotions and dynamically adjust device operation.

[0780] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0781] In this invention, the server includes detection means for acquiring environmental information, machine learning means for analyzing the acquired environmental information to recognize organisms, a suction device for capturing the recognized organisms and sealing them in a containment device, acquisition means for acquiring emotional data, an analysis device for analyzing the acquired emotional data to determine the user's emotional state, and adjustment means for dynamically adjusting notification content based on the emotional state. This enables effective and less stressful pest control activities and service improvements while taking into account the user's emotions and the state of customers in the store.

[0782] "Environmental information" refers to information about the surroundings of the subject, and includes data such as temperature, humidity, and sound waves.

[0783] "Detection means" refers to devices used to acquire environmental information, and includes various sensors.

[0784] A "machine learning device" is a device that uses a learning algorithm to analyze acquired environmental information and recognize the presence of living organisms.

[0785] A "suction device" is a device used to capture recognized organisms and seal them inside a containment device.

[0786] A "containment device" is a device used to temporarily store captured organisms and has a structure that allows for environmental control.

[0787] "Emotional data" refers to information obtained from the user's voice and facial expressions, and serves as the basis for inferring their emotional state.

[0788] "Means of acquisition" refers to devices used to collect emotional data, including cameras and microphones.

[0789] An "analysis device" is a device that analyzes acquired emotional data to determine the user's emotional state.

[0790] A "modification device" is a device that dynamically changes notification content and extermination activities based on emotional state.

[0791] To implement this invention, a system is utilized in which a server collects environmental information and emotional data and analyzes them. This system is equipped with a combination of sensors and analytical algorithms, and specifically includes smart glasses and dedicated devices.

[0792] The server uses various sensors to acquire environmental information (temperature, humidity, sound waves, etc.) in real time. These include infrared sensors and ultrasonic sensors. Furthermore, a device integrating a microphone and camera is used to acquire emotional data from the user's voice and facial expressions. This device utilizes computer vision and speech recognition technologies to accurately determine the user's emotional state.

[0793] The acquired data is analyzed by a machine learning model on the server (e.g., a deep learning model using Python and TensorFlow). This model achieves highly accurate emotion recognition by utilizing services such as Microsoft Azure's Emotion API and Face API.

[0794] Based on the analysis results, the server dynamically adjusts notification content and extermination strategies. For example, if the user is relaxed, the notification sound is suppressed and the suction device operates in silent mode. This allows for efficient extermination without causing unnecessary stress to the user.

[0795] As a concrete example, considering its application in a retail setting, this system can recognize the emotional state of customers and automatically optimize service. By using this system, users can increase customer satisfaction and provide a less stressful environment.

[0796] An example of a prompt message could be, "If a customer is showing signs of stress in the store, please suggest how to flexibly adjust your service style." This auxiliary function provides store employees and staff with guidance to provide appropriate responses.

[0797] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0798] Step 1:

[0799] The server acquires ambient environmental information (temperature, humidity, sound waves, etc.) using sensors embedded in smart glasses or dedicated devices. The input is raw data from the sensors, and the output is environmental data that has been organized and converted into a format suitable for analysis. This conversion includes saving to a database and converting to a specific format.

[0800] Step 2:

[0801] The device uses a camera and microphone to record and capture the user's voice and facial expressions. Input consists of audio and image data, while output is emotion data extracted through speech recognition and image analysis. This process utilizes Microsoft Azure's Emotion API and Face API to quantify and determine the emotional state.

[0802] Step 3:

[0803] The server inputs the acquired environmental and emotional data into a machine learning model for analysis. The input is the data obtained in the previous step, and the output is a behavioral indicator based on the user's emotional state and surrounding environmental conditions. The data is processed by a TensorFlow model, and an appropriate response is determined for each situation.

[0804] Step 4:

[0805] Based on the analysis results, the server determines the notification content and the device's operating mode. The input is the analysis results from step 3, and the output is the specific notification message and the settings information for the pest control system. For example, the pest control system may be set to silent mode, or the notification sound may be suppressed.

[0806] Step 5:

[0807] The user verifies the optimized device operation based on instructions from the server and makes manual adjustments as needed. Inputs are notifications and suggestions from the server, while outputs are the final system settings based on the user's choices. This process is facilitated through a user interface, providing a relaxed environment.

[0808] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0809] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0810] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0811] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0812] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0813] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0814] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0815] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0816] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0817] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0818] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0819] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0820] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0821] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0822] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0823] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0824] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0825] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0826] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0827] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0828] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0829] The following is further disclosed regarding the embodiments described above.

[0830] (Claim 1)

[0831] Sensor means for acquiring environmental data,

[0832] An artificial intelligence method that analyzes acquired environmental data to detect living organisms,

[0833] A suction device for capturing detected organisms and sealing them in containment containers,

[0834] Environmental control means that adjust the environment inside the containment container to make it difficult for organisms to survive,

[0835] A means of transport that monitors the status of the storage containers and automatically moves them to a processing station when they are full,

[0836] A processing means for incinerating or pulverizing living organisms at a processing station,

[0837] A notification mechanism that monitors the system status and provides notifications as needed,

[0838] A system that includes this.

[0839] (Claim 2)

[0840] The system according to claim 1, characterized in that the artificial intelligence means calculates the confidence level of detecting an organism based on the analysis of environmental data.

[0841] (Claim 3)

[0842] The system according to claim 1, characterized by including environmental control means for controlling the environment inside a containment container by temperature, humidity, and sound waves.

[0843] "Example 1"

[0844] (Claim 1)

[0845] A detection means for acquiring environmental conditions,

[0846] An inference method for identifying organisms by analyzing acquired environmental conditions,

[0847] A means for capturing identified organisms and sealing them in storage containers,

[0848] A regulatory means that adjusts the environment inside the storage container to make it difficult for organisms to survive,

[0849] A transfer means that monitors the state of the storage container and automatically moves it to the processing unit when it reaches a specified amount,

[0850] Processing means for removing or decomposing organisms in the processing unit,

[0851] A notification system that monitors the system status and issues warnings as needed,

[0852] A system that includes this.

[0853] (Claim 2)

[0854] The system according to claim 1, characterized in that the inference means calculates the degree of confidence in identifying an organism based on an analysis of environmental conditions.

[0855] (Claim 3)

[0856] The system according to claim 1, characterized by including a control means for controlling the environment inside the storage container by temperature, humidity, and sound waves.

[0857] "Application Example 1"

[0858] (Claim 1)

[0859] A detection device and means for acquiring environmental data,

[0860] A device and means for detecting invertebrates by analyzing acquired environmental data.

[0861] A collection device and means for capturing detected invertebrates and sealing them in a storage container,

[0862] A control device and means for adjusting the environment inside a storage container to make it difficult for invertebrates to survive,

[0863] A mobile device and means for monitoring the status of a storage container and automatically moving it to a processing station when it is full.

[0864] A processing device and means for incinerating or crushing invertebrates at a processing site,

[0865] A communication device and means for monitoring the system status and transmitting information as needed.

[0866] A display means for conveying correction information to a portable display device,

[0867] A system that includes this.

[0868] (Claim 2)

[0869] The system according to claim 1, characterized in that the judgment device calculates the confidence level of detecting invertebrates based on the analysis of environmental data and provides information to a portable display device.

[0870] (Claim 3)

[0871] The system according to claim 1, characterized in that the environment inside the storage container is controlled by temperature, humidity, and sound waves, and can be monitored by a portable display device.

[0872] "Example 2 of combining an emotion engine"

[0873] (Claim 1)

[0874] Sensor means for acquiring environmental data,

[0875] An information processing means for detecting organisms by analyzing acquired environmental data,

[0876] A data collection means for acquiring user voice and facial expression data,

[0877] An emotion recognition means that analyzes acquired voice and facial expression data to determine the emotional state,

[0878] Control means that adjust the operation of the system based on the detected emotional state of the organism and the user,

[0879] A suction device for capturing detected organisms and sealing them in containment containers,

[0880] A notification system that monitors the system status and provides notifications based on the user's emotions,

[0881] A system that includes this.

[0882] (Claim 2)

[0883] The system according to claim 1, characterized in that the emotion recognition means analyzes the user's emotional state using voice data and facial expression data.

[0884] (Claim 3)

[0885] The system according to claim 1, characterized in that the control means adjusts the operating sound of the system based on a specific emotional state indicated by the user.

[0886] "Application example 2 when combining with an emotional engine"

[0887] (Claim 1)

[0888] A detection means for acquiring environmental information,

[0889] A machine learning method that analyzes acquired environmental information to recognize organisms,

[0890] A suction device and means for capturing recognized organisms and sealing them in a containment device,

[0891] An environmental control device and means for adjusting the environment within a containment device to make it difficult for organisms to survive.

[0892] A mobile device and means for monitoring the status of the storage device and automatically moving it to a processing base when it is full.

[0893] A processing device and means for incinerating or pulverizing living organisms at a processing site.

[0894] A notification device and means for monitoring the system status and transmitting information as needed.

[0895] Means of acquiring emotional data,

[0896] An analytical device and means for analyzing acquired emotional data to determine the user's emotional state.

[0897] A means of dynamically adjusting notification content based on emotional state,

[0898] A system that includes this.

[0899] (Claim 2)

[0900] The system according to claim 1, characterized in that the machine learning means calculates the degree of confidence in recognizing an organism based on the analysis of environmental information.

[0901] (Claim 3)

[0902] The system according to claim 1, characterized by including an environmental control device that controls the environment inside the containment device by temperature, humidity, and sound waves. [Explanation of Symbols]

[0903] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Sensor means for acquiring environmental data, An artificial intelligence method that analyzes acquired environmental data to detect living organisms, A suction device for capturing detected organisms and sealing them in containment containers, Environmental control means that adjust the environment inside the containment container to make it difficult for organisms to survive, A means of transport that monitors the status of the storage containers and automatically moves them to a processing station when they are full, A processing means for incinerating or pulverizing living organisms at a processing station, A notification mechanism that monitors the system status and provides notifications as needed, A system that includes this.

2. The system according to claim 1, characterized in that the artificial intelligence means calculates the confidence level of detecting an organism based on the analysis of environmental data.

3. The system according to claim 1, characterized by including environmental control means for controlling the environment inside the containment container by temperature, humidity, and sound waves.

Images