System

Abstract

A system is provided.SOLUTION: A system comprising: means for taking a picture of pests by a user using a device and transmitting the picture to a server; means for analyzing the picture of pests received by the server using a generated AI model and identifying a type of the pests; and means for generating an extermination method using the generated AI model based on environmental information associated with the identified type of the pests and transmitting the extermination method to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Once pest infestations occur, they are difficult to control, and delays in early detection and appropriate countermeasures can lead to greater damage in homes, commercial facilities, and agricultural fields. However, currently, individual users often do not know how to properly identify and exterminate pests, making it difficult to take effective countermeasures. Furthermore, public places such as buildings and restaurants lack systems that can detect pest infestations early and implement countermeasures in real time. To address these issues, a system that provides more efficient and accurate pest identification and extermination methods is needed. [Means for solving the problem]

[0005] This invention provides a means for users to take photos of pests using a terminal and send them to a server. The server then analyzes the received photos of pests using a generative AI model to identify the type of pest. The server then uses the generative AI model to generate an extermination method based on the identified type of pest and related environmental information, and sends the method to the user. This system allows users to identify appropriate extermination methods in real time.

[0006] Furthermore, a means of detecting pest infestations in real time will be added by acquiring video data from fixed cameras installed in buildings and restaurants. The server will then notify the administrator of immediate extermination measures based on the detected pest infestation. Furthermore, the server will acquire previously saved environmental data and combine it with current environmental information to provide a means of optimizing pest control methods, enabling even more effective measures. It is expected that this invention will enable comprehensive and efficient pest control in homes, commercial facilities, and agricultural fields.

[0007] "Pests" are insects or animals that cause harm to residential, commercial, agricultural, and other environments.

[0008] "User" refers to any individual or organization that uses the system to take photos of pests and send the information to the server.

[0009] "Device" refers to the device, specifically a smartphone or tablet, that a user uses to take photos of pests and transmit the photos and related information to the server.

[0010] "Server" refers to a central system that receives photos of pests sent by users, analyzes them, and provides the most appropriate extermination method.

[0011] A "generative AI model" is an artificial intelligence system that uses machine learning techniques such as deep learning to analyze photos of pests and identify their type and appropriate extermination methods.

[0012] A "fixed camera" refers to a camera device that is installed in a specific location, continuously captures video data, and transmits that data to a server.

[0013] "Environmental information" refers to data that indicates the surrounding conditions and circumstances related to the occurrence and extermination of pests, such as temperature, humidity, and cleaning status.

[0014] "Video data" refers to visual information, including pest movements, captured by a fixed camera.

[0015] "Manager" refers to an individual or organization responsible for overseeing pest infestations and countermeasures in facilities such as buildings and restaurants.

[0016] "Control methods" refer to techniques or products that effectively eliminate specific pests, and specifically include the use of pesticides and physical removal methods. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention relates to a system that identifies pest species and proposes optimal extermination methods based on the current environment. Specifically, a user takes a photo of the pest using a device and sends it to a server. The server then analyzes the received photo using a generative AI model to identify the type of pest. The server then generates an optimal extermination method based on the identification results and the user's environmental information, enabling it to send the method to the user.

[0039] System configuration

[0040] User device (smartphone, tablet, etc.)

[0041] Server (central analysis system)

[0042] Generative AI model (analysis model based on deep learning)

[0043] Fixed camera (optional surveillance camera)

[0044] IoT devices (for environmental monitoring in buildings, restaurants, etc.)

[0045] Program processing explanation

[0046] 1. User-submitted photos of pests

[0047] First, the user launches the dedicated application on their device and takes a photo of the pest they find. Next, the user presses the send button in the application to send the photo to the server. If necessary, the user can also enter environmental information such as humidity and temperature at home or on site.

[0048] 2. Receiving and analyzing images on the server

[0049] The server receives photos of pests sent by users, records a log, assigns a unique identifier, and sends the received photos to a generative AI model, which analyzes the received images and identifies the type of pest. The identification results are stored in a database.

[0050] 3. Obtaining environmental information and generating optimal removal methods

[0051] The server receives environmental information provided by the user and previously saved data, and uses a generative AI model to generate the optimal extermination method based on the type of pest and environmental information.The server then sends the selected extermination method and product information to the user.

[0052] 4. Notice to Users

[0053] The server then sends the generated recommendations to the user's device, where the user can view details of the suggested removal methods and products and act accordingly through the application.

[0054] Specific examples

[0055] User Case

[0056] When a user finds a cockroach in their kitchen, they take a photo of it using a dedicated app and send it to the server. The server identifies the type of cockroach, for example, "Cockroach Species A." Based on environmental information, such as the tendency for humidity in the home, the app then suggests effective ways to use cockroach extermination spray and how to manage grains and food. The user can then click a link within the app to purchase the extermination spray.

[0057] Building maintenance case

[0058] The building manager uses fixed cameras installed in the backyard of the building to monitor the environment in real time. The server analyzes the footage from the fixed cameras and detects rats. The server immediately sends an alert to the manager and suggests appropriate trap locations and cleaning methods. The manager implements countermeasures according to the instructions and then monitors again.

[0059] This system allows for the rapid detection of pests and the implementation of effective countermeasures, enabling users and administrators to take appropriate measures.

[0060] The processing flow will be explained below.

[0061] Step 1:

[0062] The user uses the device to launch a dedicated application and take a photo of the pest.

[0063] Step 2:

[0064] The user presses the send button in the application to send the photograph of the pest they have taken to the server. If necessary, the user can also input environmental information (humidity, temperature, etc.) and send it at the same time.

[0065] Step 3:

[0066] The server receives the pest photos sent by the user, logs them, and assigns them a unique identifier.

[0067] Step 4:

[0068] The server sends the received photo of the pest to the generative AI model, which then analyzes the image and identifies the type of pest.

[0069] Step 5:

[0070] The server stores the analysis results in a database, which includes the type of pest, the date and time the photo was taken, location information (if necessary), and environmental information.

[0071] Step 6:

[0072] The server acquires environmental information provided by the user and previously saved data. The acquired environmental information includes the temperature, humidity, cleaning status, etc. at a specific location.

[0073] Step 7:

[0074] The server uses a generative AI model to generate the optimal extermination method based on the identified pest species and the acquired environmental information. This generative AI model determines effective extermination methods based on the pest species and environmental conditions.

[0075] Step 8:

[0076] The server then formats the generated removal instructions for the user, generating content that includes removal instructions details, recommended product information, and links to where to purchase them.

[0077] Step 9:

[0078] The server sends the formatted proposal to the user's terminal.

[0079] Step 10:

[0080] The user opens the application and reviews the received suggestions. The user implements the suggested action (e.g., purchases the recommended product and follows its instructions).

[0081] Step 11:

[0082] Terminals (IoT device option) are installed in buildings and restaurants, and video data is acquired using fixed cameras. This video data is continuously sent to a server.

[0083] Step 12:

[0084] The server analyzes the received video data in real time and monitors for pest infestations. Based on the analysis results, an alert is generated immediately if a pest infestation is detected.

[0085] Step 13:

[0086] The server notifies the administrator of detected pest information and immediately suggests effective extermination measures, including where to set traps and how to clean the area.

[0087] Step 14:

[0088] Based on the notification received from the server, the administrator will implement the specified disinfection measures and cleaning, then monitor again and feed the results back to the server.

[0089] These steps result in a system where the process from pest detection to providing effective treatment is carried out quickly and efficiently.

[0090] Example 1

[0091] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0092] The present invention relates to a system that quickly and accurately identifies pest species and proposes optimal extermination methods to users or administrators. Conventional pest control methods rely heavily on human labor, which takes time to identify and select appropriate extermination methods, and can lead to incorrect responses. In particular, implementing optimal measures that take environmental information and past data into account requires specialized knowledge, which is difficult for average users to understand. This has led to problems such as delayed pest control and insufficient measures.

[0093] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0094] In this invention, the server includes: [means for a user to take a photo of a pest using a terminal and send it to the server; [means for the server to preprocess the received photo of the pest and analyze it with a generative AI model to identify the type of pest; and [means for the server to generate an optimal extermination method using the generative AI model based on current environmental information and past data related to the identified type of pest and send it to the user.] This makes it possible to quickly and accurately identify the type of pest and suggest the optimal extermination method.

[0095] A "terminal" is an electronic device operated by a user, and primarily refers to a portable information terminal such as a smartphone or tablet.

[0096] "Server" refers to the central processing unit that receives, processes, and stores data sent by users and analyzes it using the generative AI model.

[0097] A "generative AI model" is an analytical algorithm based on deep learning technology that identifies the type of pest based on input data and generates the optimal extermination method.

[0098] "Environmental information" refers to data provided by users, including information such as temperature, humidity, and geographic location, that is used to determine pest occurrence and optimize extermination methods.

[0099] A "fixed camera" refers to a surveillance camera installed in a building or facility that provides video data in real time.

[0100] "Control method" refers to the specific means or product application that effectively eliminates the targeted pest.

[0101] A "log" refers to a record of a series of operations and events that occur within a system, including received image data and its processing status.

[0102] "Preprocessing" refers to the processing performed to make the received image data easier to analyze by the generative AI model, and refers to operations such as resizing and noise removal.

[0103] "Identification Results" refers to information about the type of pest identified by the generative AI model.

[0104] A "database" refers to a collection of data that systematically stores identification results, environmental information, past data, etc., and makes them searchable and accessible as needed.

[0105] This invention is a system that allows users to take photos of pests and send them to a server, and then uses a generative AI model to identify the type of pest and suggest the optimal extermination method. Specifically, the system uses the following hardware and software:

[0106] Hardware and software used

[0107] User terminal: Refers to a device such as a smartphone or tablet used to take photographs and operate applications.

[0108] Server: The central analysis system that receives, processes, and analyzes data.

[0109] Generative AI model: An analytical model based on deep learning that identifies the type of pest and generates the optimal extermination method.

[0110] Fixed camera (optional): A surveillance camera installed in a building or facility that transmits video data to a server in real time.

[0111] IoT devices (optional): Refers to sensors used for environmental monitoring in buildings, restaurants, etc.

[0112] System operation explanation

[0113] User-submitted photos of pests

[0114] The user launches a dedicated application on their device and takes a photo of the pest they find. For example, they can use the camera function of their smartphone to capture a high-resolution image. They then press the send button in the application to send the captured photo data to the server. At this time, the user can also enter environmental information (temperature, humidity, etc.) if necessary.

[0115] Image reception and preprocessing on the server

[0116] The server receives photo data sent by users, records logs, assigns a unique identifier, checks the quality of the received images, and performs preprocessing (size adjustment, noise removal, etc.). The preprocessed image data is then sent to the generative AI model.

[0117] Pest species identification using generative AI models

[0118] The generative AI model receives preprocessed images sent from the server and uses deep learning techniques to identify the type of pest. Specifically, it extracts features from the image and identifies the type of pest through comparison with existing labeled data. The identification results are sent back to the server and stored in a database.

[0119] Acquiring and analyzing environmental information on the server

[0120] The server analyzes the environmental information provided by the user and similar environmental data retrieved from a past database. This evaluates the pest occurrence trends and the effectiveness of extermination methods. Based on the analysis results, a generative AI model is used to generate the optimal extermination method.

[0121] Generate and send optimal removal instructions

[0122] The server then notifies the user of the optimal removal method obtained from the generative AI model. Specifically, it creates a detailed proposal including the selected removal method and product information and sends it to the user's device. The user can then review the proposal through the application and implement countermeasures according to the instructions.

[0123] Specific examples

[0124] User Case

[0125] When a user finds a cockroach in their kitchen, they take a photo of it using a dedicated app and send it to the server. The server identifies the type of cockroach, for example, recognizing it as "Cockroach Species A." Based on the user's environmental information, such as the fact that humidity in the home tends to be high, the app then suggests effective ways to use cockroach extermination spray and how to manage grains and food. The user can then click on a link within the app to purchase the extermination spray.

[0126] Building maintenance case

[0127] The building manager monitors the environment in real time using fixed cameras installed in the backyard of the building. The server analyzes the footage from the fixed cameras and detects rats. The server immediately sends an alert to the manager and suggests appropriate locations for traps and cleaning methods. The manager implements countermeasures according to the instructions and then monitors again. This system allows for rapid detection of pests and the implementation of effective countermeasures, enabling users and managers to take appropriate measures.

[0128] Prompt Sentence Examples

[0129] 1. "I want to take photos of pests with my smartphone, send them to a server, and receive analysis results and recommendations for the best extermination method."

[0130] 2. "A system that analyzes footage from surveillance cameras installed in office buildings, immediately notifies the operator when pests are discovered, and suggests the most appropriate countermeasures."

[0131] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0132] Step 1:

[0133] A user takes a photo of the pest and submits it

[0134] Specific operation: The user launches the dedicated application using a device such as a smartphone or tablet. They take a photo of the pest they find using the device's camera. After taking the photo, they press the send button in the application to send the photo data and, if necessary, environmental information (temperature, humidity, etc.) to the server.

[0135] Input: User-taken photo data of the pest, and optionally environmental information.

[0136] Output: Photo data and environment information sent to the server.

[0137] Step 2:

[0138] The server receives the photo data and performs preprocessing.

[0139] Specific operation: The server receives photo data sent by users, logs the received data, assigns a unique identifier, checks the quality of the received image, and performs image preprocessing (e.g., resizing, noise reduction) if necessary.

[0140] Input: Photo data submitted by the user.

[0141] Output: Preprocessed image data.

[0142] Step 3:

[0143] A generative AI model analyzes image data to identify the type of pest.

[0144] How it works: The server sends the preprocessed image data to the generative AI model, which uses deep learning techniques to extract features from the image and identify the type of pest by matching it with existing labeled data. The identification results are stored in a database.

[0145] Input: Preprocessed image data.

[0146] Output: Identification results specifying the pest species.

[0147] Step 4:

[0148] The server acquires and analyzes current environmental information and past data.

[0149] Specific operation: The server obtains current environmental information provided by the user and past environmental data stored in the database. Based on this information, it evaluates and analyzes trends in pest occurrence and the effectiveness of extermination methods.

[0150] Input: Current environmental information and historical environmental data.

[0151] Output: Analysis results on pest occurrence trends and effectiveness of control methods.

[0152] Step 5:

[0153] A generative AI model generates the optimal removal method and notifies the user.

[0154] Specific operation: The server uses a generative AI model to generate the optimal extermination method based on the identified pest species and analyzed environmental information. It then creates a detailed proposal including the generated extermination method and product information and sends it to the user's device. The user can then review the proposal through the application and implement countermeasures according to the instructions.

[0155] Input: Identified pest species and analyzed environmental information.

[0156] Output: Notification of best removal method and product information, recommendations.

[0157] This allows the user to take prompt and appropriate measures to exterminate pests.

[0158] (Application example 1)

[0159] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0160] When pests occur in a physical store, they must be exterminated quickly and effectively. Conventional methods lack sufficient means for identifying pests and quickly finding the optimal extermination method, which is time-consuming and labor-intensive. After discovering pests, purchasing the appropriate extermination products and learning how to use them also takes a lot of time. Furthermore, in stores that do not use fixed-point cameras for monitoring, it is difficult to grasp the occurrence of pests in real time. This can have a negative impact on the cleanliness of the physical store and business activities.

[0161] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0162] In this invention, the server includes: [means for a user to take a photo of a pest using a terminal and send it to the information device; [means for the information device to analyze the received photo of the pest using a generative AI model and identify the type of pest; and [means for the information device to generate an extermination method using the generative AI model based on the identified type of pest and related environmental information and send it to the user.] This makes it possible to quickly identify the type of pest and generate an optimal extermination method. Furthermore, users can instantly purchase extermination supplies through their terminal, enabling effective measures to maintain the cleanliness of physical stores. In addition, fixed cameras can be used to monitor pest occurrence in real time and implement extermination measures immediately.

[0163] A "terminal" is an electronic device used by a user to take photos of pests and send them to a server.

[0164] The "information device" is a central analysis system that receives photo data sent by users, analyzes the images using a generative AI model, and generates a removal method.

[0165] A "generative AI model" is a computational model that uses deep learning technology to identify pest species and suggest appropriate extermination methods.

[0166] A "fixed camera" is a surveillance camera device that is installed in a specific location and provides images in real time.

[0167] "Environmental information" refers to data provided by users, such as building or site temperature, humidity, and food types.

[0168] A "link" is information displayed on the device about a destination on the Internet for purchasing the appropriate pest control products.

[0169] The "extermination method" is the optimal pest extermination method suggested by the generative AI model based on the type of pest and environmental information.

[0170] A "User" is a person who uses the System to detect and eradicate pests in a physical store or building.

[0171] A "manager" is a person in charge of the operation and management of a building or store.

[0172] "Real-time" refers to the system's ability to immediately detect pest infestations and quickly implement appropriate extermination measures.

[0173] This invention describes a system that can quickly identify pest species in brick-and-mortar stores and propose optimal extermination methods. The system's main components include a terminal, an information device (server), a generative AI model, a fixed camera, and environmental sensors. It also includes a dedicated application for users.

[0174] Hardware and software used

[0175] 1. Device (smartphone or tablet):

[0176] This is a device that allows users to take photos of pests and send them to the information device.

[0177] 2. Information device (server):

[0178] It analyzes incoming photos of pests and uses a generative AI model to identify the type of pest.

[0179] Furthermore, the system generates an optimal removal method based on the identification results and related environmental information and sends it to the user.

[0180] 3. Generative AI Model:

[0181] Deep learning technology is used to identify the type of pest and generate the optimal extermination method.

[0182] Specifically, TENSORFLOW (registered trademark) and PyTorch can be used.

[0183] 4. Fixed Camera:

[0184] These are surveillance cameras installed in specific locations within the store and provide real-time footage.

[0185] For example, a Raspberry Pi camera can be used.

[0186] 5. Environmental Sensors:

[0187] It is an IoT device for monitoring the temperature and humidity inside a store.

[0188] For example, using a DHT22 sensor.

[0189] Program processing explanation

[0190] 1. Take a photo of the pest and send it to us

[0191] The user takes a photo of the pest using a device and sends it to the information device via a dedicated application. The application is designed to send environmental information such as temperature and humidity along with the photo.

[0192] 2. Image analysis and pest identification

[0193] The information device analyzes the received photos and identifies the type of pest using a generative AI model. The generative AI model uses deep learning technology and is capable of highly accurate identification based on past learning data. The analysis results of the generative AI model are stored in a database.

[0194] 3. Generating and notifying removal instructions

[0195] The information device uses a generative AI model to generate the optimal extermination method based on the identified pest species and environmental information provided by the user. The generated extermination method and recommended extermination products are then sent to the user's device, which also provides a function to instantly purchase extermination products based on this information.

[0196] 4. Real-time monitoring using fixed cameras

[0197] Fixed cameras monitor specific areas within a store in real time and transmit the images to an information device. The information device analyzes the video data and immediately notifies the manager if a pest infestation is detected. This notification also includes specific extermination methods.

[0198] Specific examples

[0199] For example, if a store staff member finds a cockroach in the kitchen, they can take a photo with their device and send it to the information device via a dedicated application. The information device then uses a generative AI model to identify the type of cockroach and, taking into account humidity and temperature information, suggests the optimal extermination spray and management method. The suggested method and product are instantly notified to the device, along with a link to purchase the extermination products.

[0200] In addition, if fixed cameras are monitoring the backyard, the information device will analyze the footage in real time and send out an alert if it detects any pests. For example, if a mouse is detected, it will also suggest appropriate locations to set traps and cleaning methods.

[0201] Prompt Sentence Examples

[0202] Input prompt: "Identify the type of pest in this photo."

[0203] Output: "This pest is cockroach species A."

[0204] Input prompt: "Please suggest the best method to control cockroach species A."

[0205] Output: "Use an insecticide spray appropriate for your current environment and take good care of your food."

[0206] This allows for fast and effective pest management in brick-and-mortar stores.

[0207] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0208] Step 1:

[0209] Users take photos of pests using their devices and send them to the server via a dedicated application, where the photos become input data.

[0210] Input: Pest photo, (optional) environmental information (temperature, humidity)

[0211] Output: Photos and environment information sent to the server

[0212] How it works: The user launches the application, uses the camera to take a photo of a pest, and then presses the send button to send the data to the server.

[0213] Step 2:

[0214] The photo data of pests received by the server is input into a generative AI model for analysis, and the type of pest is identified.

[0215] Input: User-submitted photo of a pest

[0216] Output: Pest species identification result

[0217] How it works: After the server receives the photo data, it sends it to the generative AI model, which analyzes the image, identifies the type of pest, and stores the results in a database.

[0218] Step 3:

[0219] The server obtains environmental information provided by the user and previously saved environmental data, and based on this, a generative AI model generates the optimal eradication method.

[0220] Input: Pest species, user's environmental information, historical environmental data

[0221] Output: Recommendation of the best removal method

[0222] How it works: The server retrieves historical environmental data from the database, combines it with current environmental information, and feeds it into a generative AI model. The model calculates the optimal removal method and generates a recommendation.

[0223] Step 4:

[0224] The server generates a proposed removal method and sends the associated removal product to the user's device.

[0225] Input: Best extermination method, relevant extermination product information

[0226] Output: Notification sent to the user's device

[0227] How it works: The server sends details of the proposed removal method and product to the user's device and sends a notification. The user can then review the information in the application and access a link to purchase the product.

[0228] Step 5:

[0229] Real-time video footage is sent from the fixed camera location to a server, which then analyzes the footage to detect the occurrence of pests.

[0230] Input: Real-time video from a fixed camera

[0231] Output: Pest infestation detection results, alarm

[0232] How it works: Fixed cameras continuously transmit video footage to a server, which then uses a generative AI model to analyze the video data in real time and send an alert to the administrator if a pest is detected.

[0233] Step 6:

[0234] The server notifies the administrator of immediate removal measures and suggests specific removal methods.

[0235] Input: Pest infestation detection results

[0236] Output: Notification to administrator, specific removal instructions

[0237] How it works: If the server detects a pest outbreak, it immediately notifies the administrator and provides details on the appropriate extermination method, allowing for a prompt response.

[0238] This series of steps enables quick identification of pests in physical stores and the suggestion of effective extermination methods.

[0239] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0240] This invention improves the user experience by incorporating an emotion engine that recognizes the user's emotions in addition to a system that identifies the type of pest and suggests the optimal extermination method based on the current environment. Specifically, the user takes a photo of the pest using their device and sends it to a server. The server then analyzes the received photo using a generative AI model to identify the type of pest. The server then generates the optimal extermination method based on the identification results and the user's environmental information and sends it to the user. The emotion engine also responds based on the user's emotional state.

[0241] System configuration

[0242] User device (smartphone, tablet, etc.)

[0243] Server (central analysis system)

[0244] Generative AI model (analysis model based on deep learning)

[0245] Fixed camera (optional surveillance camera)

[0246] IoT devices (for environmental monitoring in buildings, restaurants, etc.)

[0247] Emotion engine (a system that recognizes and analyzes user emotions)

[0248] Program processing explanation

[0249] 1. User-submitted photos of pests

[0250] First, the user launches the dedicated application on their device and takes a photo of the pest they find. Next, the user presses the send button in the application to send the photo to the server. If necessary, the user can also enter environmental information such as humidity and temperature at home or on site.

[0251] 2. Receiving and analyzing images on the server

[0252] The server receives photos of pests sent by users, records a log, assigns a unique identifier, and sends the received photos to a generative AI model, which analyzes the received images and identifies the type of pest. The identification results are stored in a database.

[0253] 3. Obtaining environmental information and generating optimal removal methods

[0254] The server receives environmental information provided by the user and previously saved data, and uses a generative AI model to generate the optimal extermination method based on the type of pest and environmental information.The server then sends the selected extermination method and product information to the user.

[0255] 4. User Emotion Recognition

[0256] The server uses an emotion engine to analyze the emotional state of the user's voice and text as they submit photos, determining their stress and anxiety levels and adjusting the content and format of their responses accordingly.

[0257] 5. Emotional response and notification to users

[0258] The server adapts the type of information it provides depending on the user's emotional state: for example, it provides more detailed explanations and step-by-step guides if the user is feeling highly stressed, or it offers additional support and options for consulting a specialist if the user is feeling particularly anxious.

[0259] Specific examples

[0260] User Case

[0261] When a user finds a cockroach in their kitchen, they take a photo of it using a dedicated app and send it to the server. The server identifies the type of cockroach, for example, recognizing it as "Cockroach Species A." Based on environmental information indicating that the home tends to be humid, the app then suggests effective ways to use cockroach extermination spray and food management. If the user is experiencing strong discomfort or stress, the emotion engine recognizes the situation and provides detailed instructions on extermination measures and additional support options.

[0262] Building maintenance case

[0263] Building managers use fixed cameras installed in the backyard of the building to monitor the environment in real time. The server analyzes the footage from the fixed cameras and detects rats. The server immediately sends an alert to the manager and suggests appropriate locations for traps and cleaning methods. If the manager is feeling stressed, the emotion engine recognizes this and flexibly adjusts the response, for example by providing detailed countermeasure procedures or additional support information.

[0264] In this way, the user experience can be improved by taking into account the user's emotions and responding to the process from detecting pests to providing effective countermeasures.

[0265] The processing flow will be explained below.

[0266] Step 1:

[0267] The user launches a dedicated application on their device and takes a photo of any pests they find.

[0268] Step 2:

[0269] The user presses the send button in the application to send the photograph of the pest and environmental information (e.g., humidity, temperature, etc.) to the server.

[0270] Step 3:

[0271] The server receives user-submitted photos of pests, logs them, assigns them unique identifiers, and sends the photos to a generative AI model.

[0272] Step 4:

[0273] The server uses the generative AI model to analyze the received pest photos and identify the pest species, and the analysis results are stored in a database.

[0274] Step 5:

[0275] The server retrieves environmental information provided by the user and previously saved data, including temperature, humidity, cleaning status, etc.

[0276] Step 6:

[0277] The server uses an AI model to generate the optimal extermination method based on the identified pest type and environmental information.

[0278] Step 7:

[0279] Formats the removal instructions and product recommendations generated by the server and prepares them for delivery to the user.

[0280] Step 8:

[0281] The server uses an emotion engine to analyze the voice and text inputs made by the user when they submit a photo and determine the user's emotional state.

[0282] Step 9:

[0283] The server tailors the notification content based on the user's emotional state, for example adding detailed step-by-step guides and support options if the user is feeling highly stressed.

[0284] Step 10:

[0285] The server sends the adjusted notification content to the user's terminal.

[0286] Step 11:

[0287] The user opens the application, reviews the received suggestions, and implements the suggested removal method or product.

[0288] Step 12:

[0289] When the terminal (an optional IoT device) is installed in a building or restaurant, it uses fixed cameras to continuously monitor the environment and transmits video data to a server.

[0290] Step 13:

[0291] The server analyzes the received video data in real time to detect pest infestations, and stores the analysis results in a database as needed.

[0292] Step 14:

[0293] When the server detects the occurrence of pests, it immediately sends an alert to the administrator and suggests the most appropriate extermination measures (for example, where to set traps and how to clean).

[0294] Step 15:

[0295] The administrator implements eradication measures according to instructions from the server and monitors the results again.

[0296] This allows for a fast and efficient process from pest detection to the implementation of effective countermeasures, and also enables responses that take into account the emotional state of users and administrators.

[0297] Example 2

[0298] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0299] Conventional pest control systems have difficulty accurately identifying pest species and providing appropriate extermination methods. In particular, providing information without considering the user's emotions or stress level can sometimes impair the user experience. Furthermore, building and store managers lack the means to detect pest infestations in real time and take prompt action.

[0300] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to take a photo of a pest using an information terminal and send it to the counting device, a means for the counting device to analyze the received photo of the pest using a generative AI model and identify the type of pest, a means for the counting device to generate an extermination method using the generative AI model based on environmental information related to the identified type of pest and send it to the user, and a means for the counting device to analyze the user's emotions from voice or text and adjust the response content. This makes it possible not only to accurately identify the type of pest and provide an appropriate extermination method, but also to respond according to the user's emotional state.

[0301] An "information terminal" is an electronic device that a user uses to take photos and send them to a server, and primarily includes devices such as smartphones and tablets.

[0302] "Aggregator" means the central system that receives, processes, and analyzes data sent by users.

[0303] A "generative AI model" is an artificial intelligence technology used to analyze received data, identify the type of pest, and generate optimal extermination methods.

[0304] "Environmental information" refers to data that indicates the local conditions, such as temperature, humidity, and light conditions, at the location where the pest has occurred.

[0305] A "fixed camera" is a surveillance camera that is installed in a specific location and continuously collects video data.

[0306] "Emotion analysis" is a technology that reads the user's emotional state from the voice and text in the data they send and determines their stress level and anxiety.

[0307] "Control methods" are the means and procedures for effectively eliminating pests, identified based on the type of pest and environmental information.

[0308] This invention is a system that allows a user to take a photo of a pest and send it to a server, which then identifies the type of pest and provides the optimal extermination method. It also has the ability to analyze the user's emotions and provide an appropriate response. This system is implemented primarily using the following hardware and software:

[0309] Hardware and Software Configuration

[0310] User device: An electronic device capable of taking photos and transmitting data, such as a smartphone or tablet.

[0311] Aggregation device (server): A central analysis system that receives, analyzes, and stores data, and runs generative AI models.

[0312] Generative AI model: An analytical model based on deep learning that identifies pest species and generates extermination methods.

[0313] Fixed camera: A surveillance camera installed in a facility or store that provides continuous video data.

[0314] Sentiment analysis engine: A system that reads emotions from a user's voice or text and tailors appropriate responses.

[0315] Specific example of system operation

[0316] 1. User-submitted photos of pests

[0317] Users launch the dedicated application on their smartphone or tablet, take a photo of any pests they find, and then press the send button in the application to send the photo to the server. If necessary, users can also enter environmental information such as humidity and temperature at home or on site.

[0318] Example: A user finds a cockroach in the kitchen, takes a photo with their smartphone, and submits it through an application.

[0319] 2. Receiving and analyzing images on the server

[0320] The server receives photos of pests sent by users, records a log, and assigns a unique identifier to each photo. The server then sends the photos to a generative AI model to identify the type of pest. The identification results are stored in a database.

[0321] Example: The server assigns an identifier such as "User ID_Photo ID_20231001" to a photo and sends the photo data to the generative AI model, which then stores the identification result, "This is cockroach species A," in the database.

[0322] 3. Obtaining environmental information and generating optimal removal methods

[0323] The server receives environmental information provided by the user and previously saved data, and generates the optimal removal method using a generative AI model. It then notifies the user of the selected removal method and product information.

[0324] Example: The server retrieves information from the database, such as "Kitchen humidity: 70%, temperature: 25°C," passes it to a generative AI model, and generates an extermination method, such as "Use an extermination spray suitable for cockroach species A and increase ventilation to reduce humidity," and notifies the user of this information.

[0325] 4. Recognizing and responding to user emotions

[0326] The server uses an emotion analysis engine to analyze the user's emotional state from the voice and text when they send a photo, determining their stress and anxiety levels and tailoring the response accordingly.

[0327] Example: If a user is feeling uncomfortable or stressed, the sentiment analysis engine will recognize the situation and provide detailed guidance such as "How to get rid of cockroach species A: 1) Get the spray. 2) Spray it directly on the cockroaches. 3) Clean the kitchen." or additional support information such as "If you would like to consult with an expert, please contact us here."

[0328] Prompt Sentence Examples

[0329] "I sent a photo of a cockroach I found in my kitchen. It's a cockroach species A. How can I get rid of it in a home with high humidity? How can you support users when they're feeling anxious?"

[0330]

[0331] "Fixed cameras have detected rats. Please provide advice on the appropriate trap locations and cleaning methods. What additional support do managers need if they are stressed?"

[0332] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0333] Step 1:

[0334] Users launch a dedicated application on their smartphone or tablet and take a photo of the pest. Then, they press the send button in the application to send the photo to the server. If necessary, they can enter environmental information such as humidity and temperature at home or on the job site.

[0335] (Input) Pest photos, environmental information

[0336] (Output) Photo data and environmental information sent to the server

[0337] (Specific operation) A user finds a cockroach in the kitchen, takes a photo of the cockroach with their smartphone, and taps the send button in the app to send the data to the server.

[0338] Step 2:

[0339] The server receives photos of pests submitted by users, assigns them unique identifiers, records logs, and stores the photo data and environmental information in a database.The server then sends the photos to a generative AI model to identify the type of pest.

[0340] (Input) Photo data and environmental information sent by the user

[0341] (Output) Data stored in the database, unique identifiers assigned, and pest species identification results

[0342] (Specific operation) The server assigns an identifier such as "User ID_Photo ID_20231001" to the photo and sends the photo data to the generative AI model for analysis.

[0343] Step 3:

[0344] The server uses a generative AI model to analyze the received photo data and identify the type of pest, storing the identification results in a database and using them to generate the next necessary extermination method.

[0345] (Input) Photo data, generative AI model

[0346] (Output) Pest identification results

[0347] (Specific operation) The generative AI model analyzes the photo, generates an identification result that says, "This is cockroach species A," and stores this in the database.

[0348] Step 4:

[0349] The server acquires environmental information provided by the user and previously saved environmental data, generates an extermination method suitable for the type of pest, and sends the generated extermination method and product information to the user.

[0350] (Input) Pest identification results, environmental information, past environmental data

[0351] (Output) The best method of eradication and its information

[0352] (Specific operation) The server generates notifications to the user based on data such as "The humidity in the home is high" and "In the case of cockroach species A, use an extermination spray."

[0353] Step 5:

[0354] The server uses the voice and text input from the user when sending a photo to analyze the user's emotional state using an emotion analysis engine, and adjusts the format and content of the information provided based on this.

[0355] (Input) User voice and text messages

[0356] (Output) Adjusting response content

[0357] (Specific operation) If a user sends a comment such as "It was really scary," the emotion analysis engine will determine that the emotional state is at a high stress level.

[0358] Step 6:

[0359] The server generates a response tailored to the user's emotional state and notifies the user, providing reassuring information such as detailed countermeasure procedures and options for consulting with an expert.

[0360] (Input) Emotion analysis results

[0361] (Output) A tailored response that is communicated to the user

[0362] (Specific actions) The server sends a detailed guide such as "How to get rid of cockroach species A: 1) Get the spray. 2) Spray it directly on the cockroaches. 3) Clean the kitchen." It also provides additional support information such as "If you would like to consult with an expert, please contact us here."

[0363] (Application example 2)

[0364] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0365] With conventional pest control systems, users often experienced a great deal of stress between discovering pests and finding the appropriate extermination method. Furthermore, the information provided about pest species and extermination methods was inconsistent, leaving room for improvement in the user experience. Furthermore, the system was unable to respond in a way that took the user's emotional state into account, resulting in low user satisfaction. To solve these problems, a system that could analyze the user's emotional state and respond flexibly based on that was needed.

[0366] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0367] In this invention, the server includes: [means for a user to take a photo of a pest using a terminal and send it to the server; [means for the server to analyze the received photo of the pest using a generative AI model and identify the type of pest; and [means for the server to generate an extermination method using the generative AI model based on the identified type of pest and related environmental information and send it to the user.] This allows the user to quickly and accurately identify the type of pest and provide the optimal extermination method, enabling effective pest extermination.

[0368] In addition, the server uses an emotion engine to analyze the user's emotional state from voice and text, and adjusts the extermination method provided, thereby reducing stress and anxiety when the user discovers a pest, improving the user experience.

[0369] A "terminal" is a device used by a user, such as a mobile information terminal, a smartphone, or a tablet.

[0370] A "server" is a central device that receives, analyzes, stores, and provides results on a computer network for image data and environmental information.

[0371] A "generative AI model" is an algorithm that uses artificial intelligence technologies such as deep learning to analyze image data and generate information on the type of pest and appropriate extermination methods.

[0372] An "emotion engine" is a system that analyzes a user's emotional state from their voice and text and provides appropriate responses based on that.

[0373] A "fixed camera" is a device such as a surveillance camera that is installed in a specific location such as a building or restaurant and provides video footage in real time.

[0374] "Environmental information" refers to information such as the temperature, humidity, lighting conditions, and food management status of the area where pests live.

[0375] "Extermination method" is a general term for procedures, chemicals, traps, etc. used to efficiently exterminate specific pests.

[0376] "Emotional state" refers to a user's psychological state, such as stress, anxiety, anger, or joy.

[0377] "User experience" refers to the overall satisfaction and emotional response a user feels when using a system or service.

[0378] This invention is a system for identifying pests found by users and supporting appropriate extermination methods. To improve the user experience, the system features an emotion engine that provides responses tailored to the user's emotional state. The system is composed of a combination of hardware and software, including a user terminal, a server, a generative AI model, an emotion engine, and a fixed camera.

[0379] 1. Send photos of pests

[0380] First, the user launches the dedicated application on their device (such as a smartphone or tablet) and takes a photo of the pest they find. After taking the photo, the user presses the send button in the application to send the photo to the server. If necessary, the user can also enter environmental information such as the temperature and humidity of their home or work site.

[0381] 2. Image Analysis

[0382] The server receives photos of pests sent by users. It then uses a generative AI model to analyze the received images and identify the type of pest. This generative AI model is built on deep learning technology and can identify the characteristics of pests with high accuracy. The analyzed information is stored in a database.

[0383] 3. Generating removal methods

[0384] The server generates the optimal extermination method based on the identified pest species and environmental information provided by the user. Using this generative AI model, an efficient and effective extermination method is automatically selected. The selected extermination method and related product information are then notified to the user.

[0385] 4. Emotion recognition

[0386] The emotion engine analyzes the voice and text information when users send photos to determine their emotional state. Developed based on an empirical psychological model, the engine accurately recognizes emotions such as stress, anxiety, and anger.

[0387] 5. Response and Notification

[0388] The application adapts the type of information it provides depending on the user's emotional state: for example, if the user is feeling highly stressed, it will provide more detailed removal instructions and step-by-step guides, while if the user is feeling anxious, it will offer additional support and options to consult with a specialist.

[0389] Specific examples

[0390] If a store staff member spots a rat in the backroom, they can take a photo of it using a dedicated smartphone app and send it to the server. The server will immediately identify the type of rat and suggest effective extermination methods and where to set traps. If a staff member is feeling anxious or stressed, the emotion engine will detect this and provide more detailed instructions or the option to consult with a specialist.

[0391] Prompt Sentence Examples

[0392] "Images taken with the app are sent to a server, and a generative AI model is used to identify the type of pest. The app then generates the optimal extermination method based on environmental information, and uses an emotion engine to provide a response that reflects the user's emotional state."

[0393] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0394] Step 1:

[0395] The user uses the device to take a photo of the pest.

[0396] Input: A user launches a smartphone app and takes a photo of a pest.

[0397] How it works: Users can take photos of pests in a dedicated application and manually enter the necessary environmental information (temperature, humidity, etc.).

[0398] Output: The captured photos of the pests and environmental information are saved on the device.

[0399] Step 2:

[0400] The device sends photos and environmental information to the server.

[0401] Input: Pest photos and environmental information stored on your device.

[0402] How it works: The user presses the send button in the application, sending the photo and environmental information to the server.

[0403] Output: The server receives the pest photos and environmental information.

[0404] Step 3:

[0405] The image received by the server is analyzed using a generative AI model to identify the type of pest.

[0406] Input: A photo of a pest received by the server.

[0407] How it works: The server inputs image data into a generative AI model, which uses deep learning to analyze and identify the type of pest. The identified data is then stored in a database.

[0408] Output: Pest species information.

[0409] Step 4:

[0410] The server generates the optimal extermination method based on the identified pest type and environmental information.

[0411] Input: Identified pest species information and environmental information.

[0412] How it works: The server uses a generative AI model to combine pest species and environmental information to generate the optimal extermination method.

[0413] Output: Information on the best removal methods and related products.

[0414] Step 5:

[0415] The server analyzes the user's emotional state from voice and text using an emotion engine.

[0416] Input: Voice and text data entered by the user when sending a photo.

[0417] How it works: The emotion engine allows the server to analyze the user's emotional state and identify emotions such as stress, anxiety, and anger.

[0418] Output: User's emotional state information.

[0419] Step 6:

[0420] The server adjusts its offerings of extermination methods based on the emotional state.

[0421] Input: optimal removal method and user emotional state information.

[0422] How it works: The server adapts the type and level of detail of the information it provides based on the user's emotional state, for example by adding detailed instructions or the option to consult with an expert.

[0423] Output: Adjusted removal instructions and supporting information.

[0424] Step 7:

[0425] The server sends coordinated removal instructions and support information to the user.

[0426] Input: Coordinated removal instructions and support information.

[0427] Operation: The server sends the generated removal instructions and support information to the user's device.

[0428] Output: Removal instructions and support information are made available to the user on their device.

[0429] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0430] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0431] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0432] [Second embodiment]

[0433] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0434] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0435] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0436] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0437] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0438] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0439] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0440] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0441] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0442] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0443] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0444] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0445] This invention relates to a system that identifies pest species and proposes optimal extermination methods based on the current environment. Specifically, a user takes a photo of the pest using a device and sends it to a server. The server then analyzes the received photo using a generative AI model to identify the type of pest. The server then generates an optimal extermination method based on the identification results and the user's environmental information, enabling it to send the method to the user.

[0446] System configuration

[0447] User device (smartphone, tablet, etc.)

[0448] Server (central analysis system)

[0449] Generative AI model (analysis model based on deep learning)

[0450] Fixed camera (optional surveillance camera)

[0451] IoT devices (for environmental monitoring in buildings, restaurants, etc.)

[0452] Program processing explanation

[0453] 1. User-submitted photos of pests

[0454] First, the user launches the dedicated application on their device and takes a photo of the pest they find. Next, the user presses the send button in the application to send the photo to the server. If necessary, the user can also enter environmental information such as humidity and temperature at home or on site.

[0455] 2. Receiving and analyzing images on the server

[0456] The server receives photos of pests sent by users, records a log, assigns a unique identifier, and sends the received photos to a generative AI model, which analyzes the received images and identifies the type of pest. The identification results are stored in a database.

[0457] 3. Obtaining environmental information and generating optimal removal methods

[0458] The server receives environmental information provided by the user and previously saved data, and uses a generative AI model to generate the optimal extermination method based on the type of pest and environmental information.The server then sends the selected extermination method and product information to the user.

[0459] 4. Notice to Users

[0460] The server then sends the generated recommendations to the user's device, where the user can view details of the suggested removal methods and products and act accordingly through the application.

[0461] Specific examples

[0462] User Case

[0463] When a user finds a cockroach in their kitchen, they take a photo of it using a dedicated app and send it to the server. The server identifies the type of cockroach, for example, "Cockroach Species A." Based on environmental information, such as the tendency for humidity in the home, the app then suggests effective ways to use cockroach extermination spray and how to manage grains and food. The user can then click a link within the app to purchase the extermination spray.

[0464] Building maintenance case

[0465] The building manager uses fixed cameras installed in the backyard of the building to monitor the environment in real time. The server analyzes the footage from the fixed cameras and detects rats. The server immediately sends an alert to the manager and suggests appropriate trap locations and cleaning methods. The manager implements countermeasures according to the instructions and then monitors again.

[0466] This system allows for the rapid detection of pests and the implementation of effective countermeasures, enabling users and administrators to take appropriate measures.

[0467] The processing flow will be explained below.

[0468] Step 1:

[0469] The user uses the device to launch a dedicated application and take a photo of the pest.

[0470] Step 2:

[0471] The user presses the send button in the application to send the photograph of the pest they have taken to the server. If necessary, the user can also input environmental information (humidity, temperature, etc.) and send it at the same time.

[0472] Step 3:

[0473] The server receives the pest photos sent by the user, logs them, and assigns them a unique identifier.

[0474] Step 4:

[0475] The server sends the received photo of the pest to the generative AI model, which then analyzes the image and identifies the type of pest.

[0476] Step 5:

[0477] The server stores the analysis results in a database, which includes the type of pest, the date and time the photo was taken, location information (if necessary), and environmental information.

[0478] Step 6:

[0479] The server acquires environmental information provided by the user and previously saved data. The acquired environmental information includes the temperature, humidity, cleaning status, etc. at a specific location.

[0480] Step 7:

[0481] The server uses a generative AI model to generate the optimal extermination method based on the identified pest species and the acquired environmental information. This generative AI model determines effective extermination methods based on the pest species and environmental conditions.

[0482] Step 8:

[0483] The server then formats the generated removal instructions for the user, generating content that includes removal instructions details, recommended product information, and links to where to purchase them.

[0484] Step 9:

[0485] The server sends the formatted proposal to the user's terminal.

[0486] Step 10:

[0487] The user opens the application and reviews the received suggestions. The user implements the suggested action (e.g., purchases the recommended product and follows its instructions).

[0488] Step 11:

[0489] Terminals (IoT device option) are installed in buildings and restaurants, and video data is acquired using fixed cameras. This video data is continuously sent to a server.

[0490] Step 12:

[0491] The server analyzes the received video data in real time and monitors for pest infestations. Based on the analysis results, an alert is generated immediately if a pest infestation is detected.

[0492] Step 13:

[0493] The server notifies the administrator of detected pest information and immediately suggests effective extermination measures, including where to set traps and how to clean the area.

[0494] Step 14:

[0495] Based on the notification received from the server, the administrator will implement the specified disinfection measures and cleaning, then monitor again and feed the results back to the server.

[0496] These steps result in a system where the process from pest detection to providing effective treatment is carried out quickly and efficiently.

[0497] Example 1

[0498] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0499] The present invention relates to a system that quickly and accurately identifies pest species and proposes optimal extermination methods to users or administrators. Conventional pest control methods rely heavily on human labor, which takes time to identify and select appropriate extermination methods, and can lead to incorrect responses. In particular, implementing optimal measures that take environmental information and past data into account requires specialized knowledge, which is difficult for average users to understand. This has led to problems such as delayed pest control and insufficient measures.

[0500] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0501] In this invention, the server includes: [means for a user to take a photo of a pest using a terminal and send it to the server; [means for the server to preprocess the received photo of the pest and analyze it with a generative AI model to identify the type of pest; and [means for the server to generate an optimal extermination method using the generative AI model based on current environmental information and past data related to the identified type of pest and send it to the user.] This makes it possible to quickly and accurately identify the type of pest and suggest the optimal extermination method.

[0502] A "terminal" is an electronic device operated by a user, and primarily refers to a portable information terminal such as a smartphone or tablet.

[0503] "Server" refers to the central processing unit that receives, processes, and stores data sent by users and analyzes it using the generative AI model.

[0504] A "generative AI model" is an analytical algorithm based on deep learning technology that identifies the type of pest based on input data and generates the optimal extermination method.

[0505] "Environmental information" refers to data provided by users, including information such as temperature, humidity, and geographic location, that is used to determine pest occurrence and optimize extermination methods.

[0506] A "fixed camera" refers to a surveillance camera installed in a building or facility that provides video data in real time.

[0507] "Control method" refers to the specific means or product application that effectively eliminates the targeted pest.

[0508] A "log" refers to a record of a series of operations and events that occur within a system, including received image data and its processing status.

[0509] "Preprocessing" refers to the processing performed to make the received image data easier to analyze by the generative AI model, and refers to operations such as resizing and noise removal.

[0510] "Identification Results" refers to information about the type of pest identified by the generative AI model.

[0511] A "database" refers to a collection of data that systematically stores identification results, environmental information, past data, etc., and makes them searchable and accessible as needed.

[0512] This invention is a system that allows users to take photos of pests and send them to a server, and then uses a generative AI model to identify the type of pest and suggest the optimal extermination method. Specifically, the system uses the following hardware and software:

[0513] Hardware and software used

[0514] User terminal: Refers to a device such as a smartphone or tablet used to take photographs and operate applications.

[0515] Server: The central analysis system that receives, processes, and analyzes data.

[0516] Generative AI model: An analytical model based on deep learning that identifies the type of pest and generates the optimal extermination method.

[0517] Fixed camera (optional): A surveillance camera installed in a building or facility that transmits video data to a server in real time.

[0518] IoT devices (optional): Refers to sensors used for environmental monitoring in buildings, restaurants, etc.

[0519] System operation explanation

[0520] User-submitted photos of pests

[0521] The user launches a dedicated application on their device and takes a photo of the pest they find. For example, they can use the camera function of their smartphone to capture a high-resolution image. They then press the send button in the application to send the captured photo data to the server. At this time, the user can also enter environmental information (temperature, humidity, etc.) if necessary.

[0522] Image reception and preprocessing on the server

[0523] The server receives photo data sent by users, records logs, assigns a unique identifier, checks the quality of the received images, and performs preprocessing (size adjustment, noise removal, etc.). The preprocessed image data is then sent to the generative AI model.

[0524] Pest species identification using generative AI models

[0525] The generative AI model receives preprocessed images sent from the server and uses deep learning techniques to identify the type of pest. Specifically, it extracts features from the image and identifies the type of pest through comparison with existing labeled data. The identification results are sent back to the server and stored in a database.

[0526] Acquiring and analyzing environmental information on the server

[0527] The server analyzes the environmental information provided by the user and similar environmental data retrieved from a past database. This evaluates the pest occurrence trends and the effectiveness of extermination methods. Based on the analysis results, a generative AI model is used to generate the optimal extermination method.

[0528] Generate and send optimal removal instructions

[0529] The server then notifies the user of the optimal removal method obtained from the generative AI model. Specifically, it creates a detailed proposal including the selected removal method and product information and sends it to the user's device. The user can then review the proposal through the application and implement countermeasures according to the instructions.

[0530] Specific examples

[0531] User Case

[0532] When a user finds a cockroach in their kitchen, they take a photo of it using a dedicated app and send it to the server. The server identifies the type of cockroach, for example, recognizing it as "Cockroach Species A." Based on the user's environmental information, such as the fact that humidity in the home tends to be high, the app then suggests effective ways to use cockroach extermination spray and how to manage grains and food. The user can then click on a link within the app to purchase the extermination spray.

[0533] Building maintenance case

[0534] The building manager monitors the environment in real time using fixed cameras installed in the backyard of the building. The server analyzes the footage from the fixed cameras and detects rats. The server immediately sends an alert to the manager and suggests appropriate locations for traps and cleaning methods. The manager implements countermeasures according to the instructions and then monitors again. This system allows for rapid detection of pests and the implementation of effective countermeasures, enabling users and managers to take appropriate measures.

[0535] Prompt Sentence Examples

[0536] 1. "I want to take photos of pests with my smartphone, send them to a server, and receive analysis results and recommendations for the best extermination method."

[0537] 2. "A system that analyzes footage from surveillance cameras installed in office buildings, immediately notifies the operator when pests are discovered, and suggests the most appropriate countermeasures."

[0538] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0539] Step 1:

[0540] A user takes a photo of the pest and submits it

[0541] Specific operation: The user launches the dedicated application using a device such as a smartphone or tablet. They take a photo of the pest they find using the device's camera. After taking the photo, they press the send button in the application to send the photo data and, if necessary, environmental information (temperature, humidity, etc.) to the server.

[0542] Input: User-taken photo data of the pest, and optionally environmental information.

[0543] Output: Photo data and environment information sent to the server.

[0544] Step 2:

[0545] The server receives the photo data and performs preprocessing.

[0546] Specific operation: The server receives photo data sent by users, logs the received data, assigns a unique identifier, checks the quality of the received image, and performs image preprocessing (e.g., resizing, noise reduction) if necessary.

[0547] Input: Photo data submitted by the user.

[0548] Output: Preprocessed image data.

[0549] Step 3:

[0550] A generative AI model analyzes image data to identify the type of pest.

[0551] How it works: The server sends the preprocessed image data to the generative AI model, which uses deep learning techniques to extract features from the image and identify the type of pest by matching it with existing labeled data. The identification results are stored in a database.

[0552] Input: Preprocessed image data.

[0553] Output: Identification results specifying the pest species.

[0554] Step 4:

[0555] The server acquires and analyzes current environmental information and past data.

[0556] Specific operation: The server obtains current environmental information provided by the user and past environmental data stored in the database. Based on this information, it evaluates and analyzes trends in pest occurrence and the effectiveness of extermination methods.

[0557] Input: Current environmental information and historical environmental data.

[0558] Output: Analysis results on pest occurrence trends and effectiveness of control methods.

[0559] Step 5:

[0560] A generative AI model generates the optimal removal method and notifies the user.

[0561] Specific operation: The server uses a generative AI model to generate the optimal extermination method based on the identified pest species and analyzed environmental information. It then creates a detailed proposal including the generated extermination method and product information and sends it to the user's device. The user can then review the proposal through the application and implement countermeasures according to the instructions.

[0562] Input: Identified pest species and analyzed environmental information.

[0563] Output: Notification of best removal method and product information, recommendations.

[0564] This allows the user to take prompt and appropriate measures to exterminate pests.

[0565] (Application example 1)

[0566] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0567] When pests occur in a physical store, they must be exterminated quickly and effectively. Conventional methods lack sufficient means for identifying pests and quickly finding the optimal extermination method, which is time-consuming and labor-intensive. After discovering pests, purchasing the appropriate extermination products and learning how to use them also takes a lot of time. Furthermore, in stores that do not use fixed-point cameras for monitoring, it is difficult to grasp the occurrence of pests in real time. This can have a negative impact on the cleanliness of the physical store and business activities.

[0568] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0569] In this invention, the server includes: [means for a user to take a photo of a pest using a terminal and send it to the information device; [means for the information device to analyze the received photo of the pest using a generative AI model and identify the type of pest; and [means for the information device to generate an extermination method using the generative AI model based on the identified type of pest and related environmental information and send it to the user.] This makes it possible to quickly identify the type of pest and generate an optimal extermination method. Furthermore, users can instantly purchase extermination supplies through their terminal, enabling effective measures to maintain the cleanliness of physical stores. In addition, fixed cameras can be used to monitor pest occurrence in real time and implement extermination measures immediately.

[0570] A "terminal" is an electronic device used by a user to take photos of pests and send them to a server.

[0571] The "information device" is a central analysis system that receives photo data sent by users, analyzes the images using a generative AI model, and generates a removal method.

[0572] A "generative AI model" is a computational model that uses deep learning technology to identify pest species and suggest appropriate extermination methods.

[0573] A "fixed camera" is a surveillance camera device that is installed in a specific location and provides images in real time.

[0574] "Environmental information" refers to data provided by users, such as building or site temperature, humidity, and food types.

[0575] A "link" is information displayed on the device about a destination on the Internet for purchasing the appropriate pest control products.

[0576] The "extermination method" is the optimal pest extermination method suggested by the generative AI model based on the type of pest and environmental information.

[0577] A "User" is a person who uses the System to detect and eradicate pests in a physical store or building.

[0578] A "manager" is a person in charge of the operation and management of a building or store.

[0579] "Real-time" refers to the system's ability to immediately detect pest infestations and quickly implement appropriate extermination measures.

[0580] This invention describes a system that can quickly identify pest species in brick-and-mortar stores and propose optimal extermination methods. The system's main components include a terminal, an information device (server), a generative AI model, a fixed camera, and environmental sensors. It also includes a dedicated application for users.

[0581] Hardware and software used

[0582] 1. Device (smartphone or tablet):

[0583] This is a device that allows users to take photos of pests and send them to the information device.

[0584] 2. Information device (server):

[0585] It analyzes incoming photos of pests and uses a generative AI model to identify the type of pest.

[0586] Furthermore, the system generates an optimal removal method based on the identification results and related environmental information and sends it to the user.

[0587] 3. Generative AI Model:

[0588] Deep learning technology is used to identify the type of pest and generate the optimal extermination method.

[0589] Specifically, TensorFlow and PyTorch can be used.

[0590] 4. Fixed Camera:

[0591] These are surveillance cameras installed in specific locations within the store and provide real-time footage.

[0592] For example, a Raspberry Pi camera can be used.

[0593] 5. Environmental Sensors:

[0594] It is an IoT device for monitoring the temperature and humidity inside a store.

[0595] For example, using a DHT22 sensor.

[0596] Program processing explanation

[0597] 1. Take a photo of the pest and send it to us

[0598] The user takes a photo of the pest using a device and sends it to the information device via a dedicated application. The application is designed to send environmental information such as temperature and humidity along with the photo.

[0599] 2. Image analysis and pest identification

[0600] The information device analyzes the received photos and uses a generative AI model to identify the type of pest. The generative AI model uses deep learning technology and is capable of highly accurate identification based on past learning data. The analysis results of the generative AI model are stored in a database.

[0601] 3. Generating and notifying removal instructions

[0602] The information device uses a generative AI model to generate the optimal extermination method based on the identified pest species and environmental information provided by the user. The generated extermination method and recommended extermination products are then sent to the user's device, which also provides a function to instantly purchase extermination products based on this information.

[0603] 4. Real-time monitoring using fixed cameras

[0604] Fixed cameras monitor specific areas within a store in real time and transmit the images to an information device. The information device analyzes the video data and immediately notifies the manager if a pest infestation is detected. This notification also includes specific extermination methods.

[0605] Specific examples

[0606] For example, if a store staff member finds a cockroach in the kitchen, they can take a photo with their device and send it to the information device via a dedicated application. The information device then uses a generative AI model to identify the type of cockroach and, taking into account humidity and temperature information, suggests the optimal extermination spray and management method. The suggested method and product are instantly notified to the device, along with a link to purchase the extermination products.

[0607] In addition, if fixed cameras are monitoring the backyard, the information device will analyze the footage in real time and send out an alert if it detects any pests. For example, if a mouse is detected, it will also suggest appropriate locations to set traps and cleaning methods.

[0608] Prompt Sentence Examples

[0609] Input prompt: "Identify the type of pest in this photo."

[0610] Output: "This pest is cockroach species A."

[0611] Input prompt: "Please suggest the best method to control cockroach species A."

[0612] Output: "Use an insecticide spray appropriate for your current environment and take good care of your food."

[0613] This allows for fast and effective pest management in brick-and-mortar stores.

[0614] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0615] Step 1:

[0616] Users take photos of pests using their devices and send them to the server via a dedicated application, where the photos become input data.

[0617] Input: Pest photo, (optional) environmental information (temperature, humidity)

[0618] Output: Photos and environment information sent to the server

[0619] How it works: The user launches the application, uses the camera to take a photo of a pest, and then presses the send button to send the data to the server.

[0620] Step 2:

[0621] The photo data of pests received by the server is input into a generative AI model for analysis, and the type of pest is identified.

[0622] Input: User-submitted photo of a pest

[0623] Output: Pest species identification result

[0624] How it works: After the server receives the photo data, it sends it to the generative AI model, which analyzes the image, identifies the type of pest, and stores the results in a database.

[0625] Step 3:

[0626] The server obtains environmental information provided by the user and previously saved environmental data, and based on this, a generative AI model generates the optimal eradication method.

[0627] Input: Pest species, user's environmental information, historical environmental data

[0628] Output: Recommendation of the best removal method

[0629] How it works: The server retrieves historical environmental data from the database, combines it with current environmental information, and feeds it into a generative AI model. The model calculates the optimal removal method and generates a recommendation.

[0630] Step 4:

[0631] The server generates a proposed removal method and sends the associated removal product to the user's device.

[0632] Input: Best extermination method, relevant extermination product information

[0633] Output: Notification sent to the user's device

[0634] How it works: The server sends details of the proposed removal method and product to the user's device and sends a notification. The user can then review the information in the application and access a link to purchase the product.

[0635] Step 5:

[0636] Real-time video footage is sent from the fixed camera location to a server, which then analyzes the footage to detect the occurrence of pests.

[0637] Input: Real-time video from a fixed camera

[0638] Output: Pest infestation detection results, alarm

[0639] How it works: Fixed cameras continuously transmit video footage to a server, which then uses a generative AI model to analyze the video data in real time and send an alert to the administrator if a pest is detected.

[0640] Step 6:

[0641] The server notifies the administrator of immediate removal measures and suggests specific removal methods.

[0642] Input: Pest infestation detection results

[0643] Output: Notification to administrator, specific removal instructions

[0644] How it works: If the server detects a pest outbreak, it immediately notifies the administrator and provides details on the appropriate extermination method, allowing for a prompt response.

[0645] This series of steps enables quick identification of pests in physical stores and the suggestion of effective extermination methods.

[0646] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0647] This invention improves the user experience by incorporating an emotion engine that recognizes the user's emotions in addition to a system that identifies the type of pest and suggests the optimal extermination method based on the current environment. Specifically, the user takes a photo of the pest using their device and sends it to a server. The server then analyzes the received photo using a generative AI model to identify the type of pest. The server then generates the optimal extermination method based on the identification results and the user's environmental information and sends it to the user. The emotion engine also responds based on the user's emotional state.

[0648] System configuration

[0649] User device (smartphone, tablet, etc.)

[0650] Server (central analysis system)

[0651] Generative AI model (analysis model based on deep learning)

[0652] Fixed camera (optional surveillance camera)

[0653] IoT devices (for environmental monitoring in buildings, restaurants, etc.)

[0654] Emotion engine (a system that recognizes and analyzes user emotions)

[0655] Program processing explanation

[0656] 1. User-submitted photos of pests

[0657] First, the user launches the dedicated application on their device and takes a photo of the pest they find. Next, the user presses the send button in the application to send the photo to the server. If necessary, the user can also enter environmental information such as humidity and temperature at home or on site.

[0658] 2. Receiving and analyzing images on the server

[0659] The server receives photos of pests sent by users, records a log, assigns a unique identifier, and sends the received photos to a generative AI model, which analyzes the received images and identifies the type of pest. The identification results are stored in a database.

[0660] 3. Obtaining environmental information and generating optimal removal methods

[0661] The server receives environmental information provided by the user and previously saved data, and uses a generative AI model to generate the optimal extermination method based on the type of pest and environmental information.The server then sends the selected extermination method and product information to the user.

[0662] 4. User Emotion Recognition

[0663] The server uses an emotion engine to analyze the emotional state of the user's voice and text as they submit photos, determining their stress and anxiety levels and adjusting the content and format of their responses accordingly.

[0664] 5. Emotional response and notification to users

[0665] The server adapts the type of information it provides depending on the user's emotional state: for example, it provides more detailed explanations and step-by-step guides if the user is feeling highly stressed, or it offers additional support and options for consulting a specialist if the user is feeling particularly anxious.

[0666] Specific examples

[0667] User Case

[0668] When a user finds a cockroach in their kitchen, they take a photo of it using a dedicated app and send it to the server. The server identifies the type of cockroach, for example, recognizing it as "Cockroach Species A." Based on environmental information indicating that the home tends to be humid, the app then suggests effective ways to use cockroach extermination spray and food management. If the user is experiencing strong discomfort or stress, the emotion engine recognizes the situation and provides detailed instructions on extermination measures and additional support options.

[0669] Building maintenance case

[0670] Building managers use fixed cameras installed in the backyard of the building to monitor the environment in real time. The server analyzes the footage from the fixed cameras and detects rats. The server immediately sends an alert to the manager and suggests appropriate locations for traps and cleaning methods. If the manager is feeling stressed, the emotion engine recognizes this and flexibly adjusts the response, for example by providing detailed countermeasure procedures or additional support information.

[0671] In this way, the user experience can be improved by taking into account the user's emotions and responding to the process from detecting pests to providing effective countermeasures.

[0672] The processing flow will be explained below.

[0673] Step 1:

[0674] The user launches a dedicated application on their device and takes a photo of any pests they find.

[0675] Step 2:

[0676] The user presses the send button in the application to send the photograph of the pest and environmental information (e.g., humidity, temperature, etc.) to the server.

[0677] Step 3:

[0678] The server receives user-submitted photos of pests, logs them, assigns them unique identifiers, and sends the photos to a generative AI model.

[0679] Step 4:

[0680] The server uses the generative AI model to analyze the received pest photos and identify the pest species, and the analysis results are stored in a database.

[0681] Step 5:

[0682] The server retrieves environmental information provided by the user and previously saved data, including temperature, humidity, cleaning status, etc.

[0683] Step 6:

[0684] The server uses an AI model to generate the optimal extermination method based on the identified pest type and environmental information.

[0685] Step 7:

[0686] Formats the removal instructions and product recommendations generated by the server and prepares them for delivery to the user.

[0687] Step 8:

[0688] The server uses an emotion engine to analyze the voice and text inputs made by the user when they submit a photo and determine the user's emotional state.

[0689] Step 9:

[0690] The server tailors the notification content based on the user's emotional state, for example adding detailed step-by-step guides and support options if the user is feeling highly stressed.

[0691] Step 10:

[0692] The server sends the adjusted notification content to the user's terminal.

[0693] Step 11:

[0694] The user opens the application, reviews the received suggestions, and implements the suggested removal method or product.

[0695] Step 12:

[0696] When the terminal (an optional IoT device) is installed in a building or restaurant, it uses fixed cameras to continuously monitor the environment and transmits video data to a server.

[0697] Step 13:

[0698] The server analyzes the received video data in real time to detect pest infestations, and stores the analysis results in a database as needed.

[0699] Step 14:

[0700] When the server detects the occurrence of pests, it immediately sends an alert to the administrator and suggests the most appropriate extermination measures (for example, where to set traps and how to clean).

[0701] Step 15:

[0702] The administrator implements eradication measures according to instructions from the server and monitors the results again.

[0703] This allows for a fast and efficient process from pest detection to the implementation of effective countermeasures, and also enables responses that take into account the emotional state of users and administrators.

[0704] Example 2

[0705] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0706] Conventional pest control systems have difficulty accurately identifying pest species and providing appropriate extermination methods. In particular, providing information without considering the user's emotions or stress level can sometimes impair the user experience. Furthermore, building and store managers lack the means to detect pest infestations in real time and take prompt action.

[0707] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to take a photo of a pest using an information terminal and send it to the counting device, a means for the counting device to analyze the received photo of the pest using a generative AI model and identify the type of pest, a means for the counting device to generate an extermination method using the generative AI model based on environmental information related to the identified type of pest and send it to the user, and a means for the counting device to analyze the user's emotions from voice or text and adjust the response content. This makes it possible not only to accurately identify the type of pest and provide an appropriate extermination method, but also to respond according to the user's emotional state.

[0708] An "information terminal" is an electronic device that a user uses to take photos and send them to a server, and primarily includes devices such as smartphones and tablets.

[0709] "Aggregator" means the central system that receives, processes, and analyzes data sent by users.

[0710] A "generative AI model" is an artificial intelligence technology used to analyze received data, identify the type of pest, and generate optimal extermination methods.

[0711] "Environmental information" refers to data that indicates the local conditions, such as temperature, humidity, and light conditions, at the location where the pest has occurred.

[0712] A "fixed camera" is a surveillance camera that is installed in a specific location and continuously collects video data.

[0713] "Emotion analysis" is a technology that reads the user's emotional state from the voice and text in the data they send and determines their stress level and anxiety.

[0714] "Control methods" are the means and procedures for effectively eliminating pests, identified based on the type of pest and environmental information.

[0715] This invention is a system that allows a user to take a photo of a pest and send it to a server, which then identifies the type of pest and provides the optimal extermination method. It also has the ability to analyze the user's emotions and provide an appropriate response. This system is implemented primarily using the following hardware and software:

[0716] Hardware and Software Configuration

[0717] User device: An electronic device capable of taking photos and transmitting data, such as a smartphone or tablet.

[0718] Aggregation device (server): A central analysis system that receives, analyzes, and stores data, and runs generative AI models.

[0719] Generative AI model: An analytical model based on deep learning that identifies pest species and generates extermination methods.

[0720] Fixed camera: A surveillance camera installed in a facility or store that provides continuous video data.

[0721] Sentiment analysis engine: A system that reads emotions from a user's voice or text and tailors appropriate responses.

[0722] Specific example of system operation

[0723] 1. User-submitted photos of pests

[0724] Users launch the dedicated application on their smartphone or tablet, take a photo of any pests they find, and then press the send button in the application to send the photo to the server. If necessary, users can also enter environmental information such as humidity and temperature at home or on site.

[0725] Example: A user finds a cockroach in the kitchen, takes a photo with their smartphone, and submits it through an application.

[0726] 2. Receiving and analyzing images on the server

[0727] The server receives photos of pests sent by users, records a log, and assigns a unique identifier to each photo. The server then sends the photos to a generative AI model to identify the type of pest. The identification results are stored in a database.

[0728] Example: The server assigns an identifier such as "User ID_Photo ID_20231001" to a photo and sends the photo data to the generative AI model, which then stores the identification result, "This is cockroach species A," in the database.

[0729] 3. Obtaining environmental information and generating optimal removal methods

[0730] The server receives environmental information provided by the user and previously saved data, and generates the optimal removal method using a generative AI model. It then notifies the user of the selected removal method and product information.

[0731] Example: The server retrieves information from the database, such as "Kitchen humidity: 70%, temperature: 25°C," passes it to a generative AI model, and generates an extermination method, such as "Use an extermination spray suitable for cockroach species A and increase ventilation to reduce humidity," and notifies the user of this information.

[0732] 4. Recognizing and responding to user emotions

[0733] The server uses an emotion analysis engine to analyze the user's emotional state from the voice and text when they send a photo, determining their stress and anxiety levels and tailoring the response accordingly.

[0734] Example: If a user is feeling uncomfortable or stressed, the sentiment analysis engine will recognize the situation and provide detailed guidance such as "How to get rid of cockroach species A: 1) Get the spray. 2) Spray it directly on the cockroaches. 3) Clean the kitchen." or additional support information such as "If you would like to consult with an expert, please contact us here."

[0735] Prompt Sentence Examples

[0736] "I sent a photo of a cockroach I found in my kitchen. It's a cockroach species A. How can I get rid of it in a home with high humidity? How can you support users when they're feeling anxious?"

[0737]

[0738] "Fixed cameras have detected rats. Please provide advice on the appropriate trap locations and cleaning methods. What additional support do managers need if they are stressed?"

[0739] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0740] Step 1:

[0741] Users launch a dedicated application on their smartphone or tablet and take a photo of the pest. Then, they press the send button in the application to send the photo to the server. If necessary, they can enter environmental information such as humidity and temperature at home or on the job site.

[0742] (Input) Pest photos, environmental information

[0743] (Output) Photo data and environmental information sent to the server

[0744] (Specific operation) A user finds a cockroach in the kitchen, takes a photo of the cockroach with their smartphone, and taps the send button in the app to send the data to the server.

[0745] Step 2:

[0746] The server receives photos of pests submitted by users, assigns them unique identifiers, records logs, and stores the photo data and environmental information in a database.The server then sends the photos to a generative AI model to identify the type of pest.

[0747] (Input) Photo data and environmental information sent by the user

[0748] (Output) Data stored in the database, unique identifiers assigned, and pest species identification results

[0749] (Specific operation) The server assigns an identifier such as "User ID_Photo ID_20231001" to the photo and sends the photo data to the generative AI model for analysis.

[0750] Step 3:

[0751] The server uses a generative AI model to analyze the received photo data and identify the type of pest, storing the identification results in a database and using them to generate the next necessary extermination method.

[0752] (Input) Photo data, generative AI model

[0753] (Output) Pest identification results

[0754] (Specific operation) The generative AI model analyzes the photo, generates an identification result that says, "This is cockroach species A," and stores this in the database.

[0755] Step 4:

[0756] The server acquires environmental information provided by the user and previously saved environmental data, generates an extermination method suitable for the type of pest, and sends the generated extermination method and product information to the user.

[0757] (Input) Pest identification results, environmental information, past environmental data

[0758] (Output) The best method of eradication and its information

[0759] (Specific operation) The server generates notifications to the user based on data such as "The humidity in the home is high" and "In the case of cockroach species A, use an extermination spray."

[0760] Step 5:

[0761] The server uses the voice and text input from the user when sending a photo to analyze the user's emotional state using an emotion analysis engine, and adjusts the format and content of the information provided based on this.

[0762] (Input) User voice and text messages

[0763] (Output) Adjusting response content

[0764] (Specific operation) If a user sends a comment such as "It was really scary," the emotion analysis engine will determine that the emotional state is at a high stress level.

[0765] Step 6:

[0766] The server generates a response tailored to the user's emotional state and notifies the user, providing reassuring information such as detailed countermeasure procedures and options for consulting with an expert.

[0767] (Input) Emotion analysis results

[0768] (Output) A tailored response that is communicated to the user

[0769] (Specific actions) The server sends a detailed guide such as "How to get rid of cockroach species A: 1) Get the spray. 2) Spray it directly on the cockroaches. 3) Clean the kitchen." It also provides additional support information such as "If you would like to consult with an expert, please contact us here."

[0770] (Application example 2)

[0771] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0772] With conventional pest control systems, users often experienced a great deal of stress between discovering pests and finding the appropriate extermination method. Furthermore, the information provided about pest species and extermination methods was inconsistent, leaving room for improvement in the user experience. Furthermore, the system was unable to respond in a way that took the user's emotional state into account, resulting in low user satisfaction. To solve these problems, a system that could analyze the user's emotional state and respond flexibly based on that was needed.

[0773] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0774] In this invention, the server includes: [means for a user to take a photo of a pest using a terminal and send it to the server; [means for the server to analyze the received photo of the pest using a generative AI model and identify the type of pest; and [means for the server to generate an extermination method using the generative AI model based on the identified type of pest and related environmental information and send it to the user.] This allows the user to quickly and accurately identify the type of pest and provide the optimal extermination method, enabling effective pest extermination.

[0775] In addition, the server uses an emotion engine to analyze the user's emotional state from voice and text, and adjusts the extermination method provided, thereby reducing stress and anxiety when the user discovers a pest, improving the user experience.

[0776] A "terminal" is a device used by a user, such as a mobile information terminal, a smartphone, or a tablet.

[0777] A "server" is a central device that receives, analyzes, stores, and provides results on a computer network for image data and environmental information.

[0778] A "generative AI model" is an algorithm that uses artificial intelligence technologies such as deep learning to analyze image data and generate information on the type of pest and appropriate extermination methods.

[0779] An "emotion engine" is a system that analyzes a user's emotional state from their voice and text and provides appropriate responses based on that.

[0780] A "fixed camera" is a device such as a surveillance camera that is installed in a specific location such as a building or restaurant and provides video footage in real time.

[0781] "Environmental information" refers to information such as the temperature, humidity, lighting conditions, and food management status of the area where pests live.

[0782] "Extermination method" is a general term for procedures, chemicals, traps, etc. used to efficiently exterminate specific pests.

[0783] "Emotional state" refers to a user's psychological state, such as stress, anxiety, anger, or joy.

[0784] "User experience" refers to the overall satisfaction and emotional response a user feels when using a system or service.

[0785] This invention is a system for identifying pests found by users and supporting appropriate extermination methods. To improve the user experience, the system features an emotion engine that provides responses tailored to the user's emotional state. The system is composed of a combination of hardware and software, including a user terminal, a server, a generative AI model, an emotion engine, and a fixed camera.

[0786] 1. Send photos of pests

[0787] First, the user launches the dedicated application on their device (such as a smartphone or tablet) and takes a photo of the pest they find. After taking the photo, the user presses the send button in the application to send the photo to the server. If necessary, the user can also enter environmental information such as the temperature and humidity of their home or work site.

[0788] 2. Image Analysis

[0789] The server receives photos of pests sent by users. It then uses a generative AI model to analyze the received images and identify the type of pest. This generative AI model is built on deep learning technology and can identify the characteristics of pests with high accuracy. The analyzed information is stored in a database.

[0790] 3. Generating removal methods

[0791] The server generates the optimal extermination method based on the identified pest species and environmental information provided by the user. Using this generative AI model, an efficient and effective extermination method is automatically selected. The selected extermination method and related product information are then notified to the user.

[0792] 4. Emotion recognition

[0793] The emotion engine analyzes the voice and text information when users send photos to determine their emotional state. Developed based on an empirical psychological model, the engine accurately recognizes emotions such as stress, anxiety, and anger.

[0794] 5. Response and Notification

[0795] The application adapts the type of information it provides depending on the user's emotional state: for example, if the user is feeling highly stressed, it will provide more detailed removal instructions and step-by-step guides, while if the user is feeling anxious, it will offer additional support and options to consult with a specialist.

[0796] Specific examples

[0797] If a store staff member spots a rat in the backroom, they can take a photo of it using a dedicated smartphone app and send it to the server. The server will immediately identify the type of rat and suggest effective extermination methods and where to set traps. If a staff member is feeling anxious or stressed, the emotion engine will detect this and provide more detailed instructions or the option to consult with a specialist.

[0798] Prompt Sentence Examples

[0799] "Images taken with the app are sent to a server, and a generative AI model is used to identify the type of pest. The app then generates the optimal extermination method based on environmental information, and uses an emotion engine to provide a response that reflects the user's emotional state."

[0800] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0801] Step 1:

[0802] The user uses the device to take a photo of the pest.

[0803] Input: A user launches a smartphone app and takes a photo of a pest.

[0804] How it works: Users can take photos of pests in a dedicated application and manually enter the necessary environmental information (temperature, humidity, etc.).

[0805] Output: The captured photos of the pests and environmental information are saved on the device.

[0806] Step 2:

[0807] The device sends photos and environmental information to the server.

[0808] Input: Pest photos and environmental information stored on your device.

[0809] How it works: The user presses the send button in the application, sending the photo and environmental information to the server.

[0810] Output: The server receives the pest photos and environmental information.

[0811] Step 3:

[0812] The image received by the server is analyzed using a generative AI model to identify the type of pest.

[0813] Input: A photo of a pest received by the server.

[0814] How it works: The server inputs image data into a generative AI model, which uses deep learning to analyze and identify the type of pest. The identified data is then stored in a database.

[0815] Output: Pest species information.

[0816] Step 4:

[0817] The server generates the optimal extermination method based on the identified pest type and environmental information.

[0818] Input: Identified pest species information and environmental information.

[0819] How it works: The server uses a generative AI model to combine pest species and environmental information to generate the optimal extermination method.

[0820] Output: Information on the best removal methods and related products.

[0821] Step 5:

[0822] The server analyzes the user's emotional state from voice and text using an emotion engine.

[0823] Input: Voice and text data entered by the user when sending a photo.

[0824] How it works: The emotion engine allows the server to analyze the user's emotional state and identify emotions such as stress, anxiety, and anger.

[0825] Output: User's emotional state information.

[0826] Step 6:

[0827] The server adjusts its offerings of extermination methods based on the emotional state.

[0828] Input: optimal removal method and user emotional state information.

[0829] How it works: The server adapts the type and level of detail of the information it provides based on the user's emotional state, for example by adding detailed instructions or the option to consult with an expert.

[0830] Output: Adjusted removal instructions and supporting information.

[0831] Step 7:

[0832] The server sends coordinated removal instructions and support information to the user.

[0833] Input: Coordinated removal instructions and support information.

[0834] Operation: The server sends the generated removal instructions and support information to the user's device.

[0835] Output: Removal instructions and support information are made available to the user on their device.

[0836] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0837] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0838] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0839] [Third embodiment]

[0840] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0841] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0842] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0843] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0844] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0845] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0846] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0847] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0848] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0849] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0850] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0851] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0852] This invention relates to a system that identifies pest species and proposes optimal extermination methods based on the current environment. Specifically, a user takes a photo of the pest using a device and sends it to a server. The server then analyzes the received photo using a generative AI model to identify the type of pest. The server then generates an optimal extermination method based on the identification results and the user's environmental information, enabling it to send the method to the user.

[0853] System configuration

[0854] User device (smartphone, tablet, etc.)

[0855] Server (central analysis system)

[0856] Generative AI model (analysis model based on deep learning)

[0857] Fixed camera (optional surveillance camera)

[0858] IoT devices (for environmental monitoring in buildings, restaurants, etc.)

[0859] Program processing explanation

[0860] 1. User-submitted photos of pests

[0861] First, the user launches the dedicated application on their device and takes a photo of the pest they find. Next, the user presses the send button in the application to send the photo to the server. If necessary, the user can also enter environmental information such as humidity and temperature at home or on site.

[0862] 2. Receiving and analyzing images on the server

[0863] The server receives photos of pests sent by users, records a log, assigns a unique identifier, and sends the received photos to a generative AI model, which analyzes the received images and identifies the type of pest. The identification results are stored in a database.

[0864] 3. Obtaining environmental information and generating optimal removal methods

[0865] The server receives environmental information provided by the user and previously saved data, and uses a generative AI model to generate the optimal extermination method based on the type of pest and environmental information.The server then sends the selected extermination method and product information to the user.

[0866] 4. Notice to Users

[0867] The server then sends the generated recommendations to the user's device, where the user can view details of the suggested removal methods and products and act accordingly through the application.

[0868] Specific examples

[0869] User Case

[0870] When a user finds a cockroach in their kitchen, they take a photo of it using a dedicated app and send it to the server. The server identifies the type of cockroach, for example, "Cockroach Species A." Based on environmental information, such as the tendency for humidity in the home, the app then suggests effective ways to use cockroach extermination spray and how to manage grains and food. The user can then click a link within the app to purchase the extermination spray.

[0871] Building maintenance case

[0872] The building manager uses fixed cameras installed in the backyard of the building to monitor the environment in real time. The server analyzes the footage from the fixed cameras and detects rats. The server immediately sends an alert to the manager and suggests appropriate trap locations and cleaning methods. The manager implements countermeasures according to the instructions and then monitors again.

[0873] This system allows for the rapid detection of pests and the implementation of effective countermeasures, enabling users and administrators to take appropriate measures.

[0874] The processing flow will be explained below.

[0875] Step 1:

[0876] The user uses the device to launch a dedicated application and take a photo of the pest.

[0877] Step 2:

[0878] The user presses the send button in the application to send the photograph of the pest they have taken to the server. If necessary, the user can also input environmental information (humidity, temperature, etc.) and send it at the same time.

[0879] Step 3:

[0880] The server receives the pest photos sent by the user, logs them, and assigns them a unique identifier.

[0881] Step 4:

[0882] The server sends the received photo of the pest to the generative AI model, which then analyzes the image and identifies the type of pest.

[0883] Step 5:

[0884] The server stores the analysis results in a database, which includes the type of pest, the date and time the photo was taken, location information (if necessary), and environmental information.

[0885] Step 6:

[0886] The server acquires environmental information provided by the user and previously saved data. The acquired environmental information includes the temperature, humidity, cleaning status, etc. at a specific location.

[0887] Step 7:

[0888] The server uses a generative AI model to generate the optimal extermination method based on the identified pest species and the acquired environmental information. This generative AI model determines effective extermination methods based on the pest species and environmental conditions.

[0889] Step 8:

[0890] The server then formats the generated removal instructions for the user, generating content that includes removal instructions details, recommended product information, and links to where to purchase them.

[0891] Step 9:

[0892] The server sends the formatted proposal to the user's terminal.

[0893] Step 10:

[0894] The user opens the application and reviews the received suggestions. The user implements the suggested action (e.g., purchases the recommended product and follows its instructions).

[0895] Step 11:

[0896] Terminals (IoT device option) are installed in buildings and restaurants, and video data is acquired using fixed cameras. This video data is continuously sent to a server.

[0897] Step 12:

[0898] The server analyzes the received video data in real time and monitors for pest infestations. Based on the analysis results, an alert is generated immediately if a pest infestation is detected.

[0899] Step 13:

[0900] The server notifies the administrator of detected pest information and immediately suggests effective extermination measures, including where to set traps and how to clean the area.

[0901] Step 14:

[0902] Based on the notification received from the server, the administrator will implement the specified disinfection measures and cleaning, then monitor again and feed the results back to the server.

[0903] These steps result in a system where the process from pest detection to providing effective treatment is carried out quickly and efficiently.

[0904] Example 1

[0905] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0906] The present invention relates to a system that quickly and accurately identifies pest species and proposes optimal extermination methods to users or administrators. Conventional pest control methods rely heavily on human labor, which takes time to identify and select appropriate extermination methods, and can lead to incorrect responses. In particular, implementing optimal measures that take environmental information and past data into account requires specialized knowledge, which is difficult for average users to understand. This has led to problems such as delayed pest control and insufficient measures.

[0907] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0908] In this invention, the server includes: [means for a user to take a photo of a pest using a terminal and send it to the server; [means for the server to preprocess the received photo of the pest and analyze it with a generative AI model to identify the type of pest; and [means for the server to generate an optimal extermination method using the generative AI model based on current environmental information and past data related to the identified type of pest and send it to the user.] This makes it possible to quickly and accurately identify the type of pest and suggest the optimal extermination method.

[0909] A "terminal" is an electronic device operated by a user, and primarily refers to a portable information terminal such as a smartphone or tablet.

[0910] "Server" refers to the central processing unit that receives, processes, and stores data sent by users and analyzes it using the generative AI model.

[0911] A "generative AI model" is an analytical algorithm based on deep learning technology that identifies the type of pest based on input data and generates the optimal extermination method.

[0912] "Environmental information" refers to data provided by users, including information such as temperature, humidity, and geographic location, that is used to determine pest occurrence and optimize extermination methods.

[0913] A "fixed camera" refers to a surveillance camera installed in a building or facility that provides video data in real time.

[0914] "Control method" refers to the specific means or product application that effectively eliminates the targeted pest.

[0915] A "log" refers to a record of a series of operations and events that occur within a system, including received image data and its processing status.

[0916] "Preprocessing" refers to the processing performed to make the received image data easier to analyze by the generative AI model, and refers to operations such as resizing and noise removal.

[0917] "Identification Results" refers to information about the type of pest identified by the generative AI model.

[0918] A "database" refers to a collection of data that systematically stores identification results, environmental information, past data, etc., and makes them searchable and accessible as needed.

[0919] This invention is a system that allows users to take photos of pests and send them to a server, and then uses a generative AI model to identify the type of pest and suggest the optimal extermination method. Specifically, the system uses the following hardware and software:

[0920] Hardware and software used

[0921] User terminal: Refers to a device such as a smartphone or tablet used to take photographs and operate applications.

[0922] Server: The central analysis system that receives, processes, and analyzes data.

[0923] Generative AI model: An analytical model based on deep learning that identifies the type of pest and generates the optimal extermination method.

[0924] Fixed camera (optional): A surveillance camera installed in a building or facility that transmits video data to a server in real time.

[0925] IoT devices (optional): Refers to sensors used for environmental monitoring in buildings, restaurants, etc.

[0926] System operation explanation

[0927] User-submitted photos of pests

[0928] The user launches a dedicated application on their device and takes a photo of the pest they find. For example, they can use the camera function of their smartphone to capture a high-resolution image. They then press the send button in the application to send the captured photo data to the server. At this time, the user can also enter environmental information (temperature, humidity, etc.) if necessary.

[0929] Image reception and preprocessing on the server

[0930] The server receives photo data sent by users, records logs, assigns a unique identifier, checks the quality of the received images, and performs preprocessing (size adjustment, noise removal, etc.). The preprocessed image data is then sent to the generative AI model.

[0931] Pest species identification using generative AI models

[0932] The generative AI model receives preprocessed images sent from the server and uses deep learning techniques to identify the type of pest. Specifically, it extracts features from the image and identifies the type of pest through comparison with existing labeled data. The identification results are sent back to the server and stored in a database.

[0933] Acquiring and analyzing environmental information on the server

[0934] The server analyzes the environmental information provided by the user and similar environmental data retrieved from a past database. This evaluates the pest occurrence trends and the effectiveness of extermination methods. Based on the analysis results, a generative AI model is used to generate the optimal extermination method.

[0935] Generate and send optimal removal instructions

[0936] The server then notifies the user of the optimal removal method obtained from the generative AI model. Specifically, it creates a detailed proposal including the selected removal method and product information and sends it to the user's device. The user can then review the proposal through the application and implement countermeasures according to the instructions.

[0937] Specific examples

[0938] User Case

[0939] When a user finds a cockroach in their kitchen, they take a photo of it using a dedicated app and send it to the server. The server identifies the type of cockroach, for example, recognizing it as "Cockroach Species A." Based on the user's environmental information, such as the fact that humidity in the home tends to be high, the app then suggests effective ways to use cockroach extermination spray and how to manage grains and food. The user can then click on a link within the app to purchase the extermination spray.

[0940] Building maintenance case

[0941] The building manager monitors the environment in real time using fixed cameras installed in the backyard of the building. The server analyzes the footage from the fixed cameras and detects rats. The server immediately sends an alert to the manager and suggests appropriate locations for traps and cleaning methods. The manager implements countermeasures according to the instructions and then monitors again. This system allows for rapid detection of pests and the implementation of effective countermeasures, enabling users and managers to take appropriate measures.

[0942] Prompt Sentence Examples

[0943] 1. "I want to take photos of pests with my smartphone, send them to a server, and receive analysis results and recommendations for the best extermination method."

[0944] 2. "A system that analyzes footage from surveillance cameras installed in office buildings, immediately notifies the operator when pests are discovered, and suggests the most appropriate countermeasures."

[0945] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0946] Step 1:

[0947] A user takes a photo of the pest and submits it

[0948] Specific operation: The user launches the dedicated application using a device such as a smartphone or tablet. They take a photo of the pest they find using the device's camera. After taking the photo, they press the send button in the application to send the photo data and, if necessary, environmental information (temperature, humidity, etc.) to the server.

[0949] Input: User-taken photo data of the pest, and optionally environmental information.

[0950] Output: Photo data and environment information sent to the server.

[0951] Step 2:

[0952] The server receives the photo data and performs preprocessing.

[0953] Specific operation: The server receives photo data sent by users, logs the received data, assigns a unique identifier, checks the quality of the received image, and performs image preprocessing (e.g., resizing, noise reduction) if necessary.

[0954] Input: Photo data submitted by the user.

[0955] Output: Preprocessed image data.

[0956] Step 3:

[0957] A generative AI model analyzes image data to identify the type of pest.

[0958] How it works: The server sends the preprocessed image data to the generative AI model, which uses deep learning techniques to extract features from the image and identify the type of pest by matching it with existing labeled data. The identification results are stored in a database.

[0959] Input: Preprocessed image data.

[0960] Output: Identification results specifying the pest species.

[0961] Step 4:

[0962] The server acquires and analyzes current environmental information and past data.

[0963] Specific operation: The server obtains current environmental information provided by the user and past environmental data stored in the database. Based on this information, it evaluates and analyzes trends in pest occurrence and the effectiveness of extermination methods.

[0964] Input: Current environmental information and historical environmental data.

[0965] Output: Analysis results on pest occurrence trends and effectiveness of control methods.

[0966] Step 5:

[0967] A generative AI model generates the optimal removal method and notifies the user.

[0968] Specific operation: The server uses a generative AI model to generate the optimal extermination method based on the identified pest species and analyzed environmental information. It then creates a detailed proposal including the generated extermination method and product information and sends it to the user's device. The user can then review the proposal through the application and implement countermeasures according to the instructions.

[0969] Input: Identified pest species and analyzed environmental information.

[0970] Output: Notification of best removal method and product information, recommendations.

[0971] This allows the user to take prompt and appropriate measures to exterminate pests.

[0972] (Application example 1)

[0973] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0974] When pests occur in a physical store, they must be exterminated quickly and effectively. Conventional methods lack sufficient means for identifying pests and quickly finding the optimal extermination method, which is time-consuming and labor-intensive. After discovering pests, purchasing the appropriate extermination products and learning how to use them also takes a lot of time. Furthermore, in stores that do not use fixed-point cameras for monitoring, it is difficult to grasp the occurrence of pests in real time. This can have a negative impact on the cleanliness of the physical store and business activities.

[0975] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0976] In this invention, the server includes: [means for a user to take a photo of a pest using a terminal and send it to the information device; [means for the information device to analyze the received photo of the pest using a generative AI model and identify the type of pest; and [means for the information device to generate an extermination method using the generative AI model based on the identified type of pest and related environmental information and send it to the user.] This makes it possible to quickly identify the type of pest and generate an optimal extermination method. Furthermore, users can instantly purchase extermination supplies through their terminal, enabling effective measures to maintain the cleanliness of physical stores. In addition, fixed cameras can be used to monitor pest occurrence in real time and implement extermination measures immediately.

[0977] A "terminal" is an electronic device used by a user to take photos of pests and send them to a server.

[0978] The "information device" is a central analysis system that receives photo data sent by users, analyzes the images using a generative AI model, and generates a removal method.

[0979] A "generative AI model" is a computational model that uses deep learning technology to identify pest species and suggest appropriate extermination methods.

[0980] A "fixed camera" is a surveillance camera device that is installed in a specific location and provides images in real time.

[0981] "Environmental information" refers to data provided by users, such as building or site temperature, humidity, and food types.

[0982] A "link" is information displayed on the device about a destination on the Internet for purchasing the appropriate pest control products.

[0983] The "extermination method" is the optimal pest extermination method suggested by the generative AI model based on the type of pest and environmental information.

[0984] A "User" is a person who uses the System to detect and eradicate pests in a physical store or building.

[0985] A "manager" is a person in charge of the operation and management of a building or store.

[0986] "Real-time" refers to the system's ability to immediately detect pest infestations and quickly implement appropriate extermination measures.

[0987] This invention describes a system that can quickly identify pest species in brick-and-mortar stores and propose optimal extermination methods. The system's main components include a terminal, an information device (server), a generative AI model, a fixed camera, and environmental sensors. It also includes a dedicated application for users.

[0988] Hardware and software used

[0989] 1. Device (smartphone or tablet):

[0990] This is a device that allows users to take photos of pests and send them to the information device.

[0991] 2. Information device (server):

[0992] It analyzes incoming photos of pests and uses a generative AI model to identify the type of pest.

[0993] Furthermore, the system generates an optimal removal method based on the identification results and related environmental information and sends it to the user.

[0994] 3. Generative AI Model:

[0995] Deep learning technology is used to identify the type of pest and generate the optimal extermination method.

[0996] Specifically, TensorFlow and PyTorch can be used.

[0997] 4. Fixed Camera:

[0998] These are surveillance cameras installed in specific locations within the store and provide real-time footage.

[0999] For example, a Raspberry Pi camera can be used.

[1000] 5. Environmental Sensors:

[1001] It is an IoT device for monitoring the temperature and humidity inside a store.

[1002] For example, using a DHT22 sensor.

[1003] Program processing explanation

[1004] 1. Take a photo of the pest and send it to us

[1005] The user takes a photo of the pest using a device and sends it to the information device via a dedicated application. The application is designed to send environmental information such as temperature and humidity along with the photo.

[1006] 2. Image analysis and pest identification

[1007] The information device analyzes the received photos and uses a generative AI model to identify the type of pest. The generative AI model uses deep learning technology and is capable of highly accurate identification based on past learning data. The analysis results of the generative AI model are stored in a database.

[1008] 3. Generating and notifying removal instructions

[1009] The information device uses a generative AI model to generate the optimal extermination method based on the identified pest species and environmental information provided by the user. The generated extermination method and recommended extermination products are then sent to the user's device, which also provides a function to instantly purchase extermination products based on this information.

[1010] 4. Real-time monitoring using fixed cameras

[1011] Fixed cameras monitor specific areas within a store in real time and transmit the images to an information device. The information device analyzes the video data and immediately notifies the manager if a pest infestation is detected. This notification also includes specific extermination methods.

[1012] Specific examples

[1013] For example, if a store staff member finds a cockroach in the kitchen, they can take a photo with their device and send it to the information device via a dedicated application. The information device then uses a generative AI model to identify the type of cockroach and, taking into account humidity and temperature information, suggests the optimal extermination spray and management method. The suggested method and product are instantly notified to the device, along with a link to purchase the extermination products.

[1014] In addition, if fixed cameras are monitoring the backyard, the information device will analyze the footage in real time and send out an alert if it detects any pests. For example, if a mouse is detected, it will also suggest appropriate locations to set traps and cleaning methods.

[1015] Prompt Sentence Examples

[1016] Input prompt: "Identify the type of pest in this photo."

[1017] Output: "This pest is cockroach species A."

[1018] Input prompt: "Please suggest the best method to control cockroach species A."

[1019] Output: "Use an insecticide spray appropriate for your current environment and take good care of your food."

[1020] This allows for fast and effective pest management in brick-and-mortar stores.

[1021] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1022] Step 1:

[1023] Users take photos of pests using their devices and send them to the server via a dedicated application, where the photos become input data.

[1024] Input: Pest photo, (optional) environmental information (temperature, humidity)

[1025] Output: Photos and environment information sent to the server

[1026] How it works: The user launches the application, uses the camera to take a photo of a pest, and then presses the send button to send the data to the server.

[1027] Step 2:

[1028] The photo data of pests received by the server is input into a generative AI model for analysis, and the type of pest is identified.

[1029] Input: User-submitted photo of a pest

[1030] Output: Pest species identification result

[1031] How it works: After the server receives the photo data, it sends it to the generative AI model, which analyzes the image, identifies the type of pest, and stores the results in a database.

[1032] Step 3:

[1033] The server obtains environmental information provided by the user and previously saved environmental data, and based on this, a generative AI model generates the optimal eradication method.

[1034] Input: Pest species, user's environmental information, historical environmental data

[1035] Output: Recommendation of the best removal method

[1036] How it works: The server retrieves historical environmental data from the database, combines it with current environmental information, and feeds it into a generative AI model. The model calculates the optimal removal method and generates a recommendation.

[1037] Step 4:

[1038] The server generates a proposed removal method and sends the associated removal product to the user's device.

[1039] Input: Best extermination method, relevant extermination product information

[1040] Output: Notification sent to the user's device

[1041] How it works: The server sends details of the proposed removal method and product to the user's device and sends a notification. The user can then review the information in the application and access a link to purchase the product.

[1042] Step 5:

[1043] Real-time video footage is sent from the fixed camera location to a server, which then analyzes the footage to detect the occurrence of pests.

[1044] Input: Real-time video from a fixed camera

[1045] Output: Pest infestation detection results, alarm

[1046] How it works: Fixed cameras continuously transmit video footage to a server, which then uses a generative AI model to analyze the video data in real time and send an alert to the administrator if a pest is detected.

[1047] Step 6:

[1048] The server notifies the administrator of immediate removal measures and suggests specific removal methods.

[1049] Input: Pest infestation detection results

[1050] Output: Notification to administrator, specific removal instructions

[1051] How it works: If the server detects a pest outbreak, it immediately notifies the administrator and provides details on the appropriate extermination method, allowing for a prompt response.

[1052] This series of steps enables quick identification of pests in physical stores and the suggestion of effective extermination methods.

[1053] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1054] This invention improves the user experience by incorporating an emotion engine that recognizes the user's emotions in addition to a system that identifies the type of pest and suggests the optimal extermination method based on the current environment. Specifically, the user takes a photo of the pest using their device and sends it to a server. The server then analyzes the received photo using a generative AI model to identify the type of pest. The server then generates the optimal extermination method based on the identification results and the user's environmental information and sends it to the user. The emotion engine also responds based on the user's emotional state.

[1055] System configuration

[1056] User device (smartphone, tablet, etc.)

[1057] Server (central analysis system)

[1058] Generative AI model (analysis model based on deep learning)

[1059] Fixed camera (optional surveillance camera)

[1060] IoT devices (for environmental monitoring in buildings, restaurants, etc.)

[1061] Emotion engine (a system that recognizes and analyzes user emotions)

[1062] Program processing explanation

[1063] 1. User-submitted photos of pests

[1064] First, the user launches the dedicated application on their device and takes a photo of the pest they find. Next, the user presses the send button in the application to send the photo to the server. If necessary, the user can also enter environmental information such as humidity and temperature at home or on site.

[1065] 2. Receiving and analyzing images on the server

[1066] The server receives photos of pests sent by users, records a log, assigns a unique identifier, and sends the received photos to a generative AI model, which analyzes the received images and identifies the type of pest. The identification results are stored in a database.

[1067] 3. Obtaining environmental information and generating optimal removal methods

[1068] The server receives environmental information provided by the user and previously saved data, and uses a generative AI model to generate the optimal extermination method based on the type of pest and environmental information.The server then sends the selected extermination method and product information to the user.

[1069] 4. User Emotion Recognition

[1070] The server uses an emotion engine to analyze the emotional state of the user's voice and text as they submit photos, determining their stress and anxiety levels and adjusting the content and format of their responses accordingly.

[1071] 5. Emotional response and notification to users

[1072] The server adapts the type of information it provides depending on the user's emotional state: for example, it provides more detailed explanations and step-by-step guides if the user is feeling highly stressed, or it offers additional support and options for consulting a specialist if the user is feeling particularly anxious.

[1073] Specific examples

[1074] User Case

[1075] When a user finds a cockroach in their kitchen, they take a photo of it using a dedicated app and send it to the server. The server identifies the type of cockroach, for example, recognizing it as "Cockroach Species A." Based on environmental information indicating that the home tends to be humid, the app then suggests effective ways to use cockroach extermination spray and food management. If the user is experiencing strong discomfort or stress, the emotion engine recognizes the situation and provides detailed instructions on extermination measures and additional support options.

[1076] Building maintenance case

[1077] Building managers use fixed cameras installed in the backyard of the building to monitor the environment in real time. The server analyzes the footage from the fixed cameras and detects rats. The server immediately sends an alert to the manager and suggests appropriate locations for traps and cleaning methods. If the manager is feeling stressed, the emotion engine recognizes this and flexibly adjusts the response, for example by providing detailed countermeasure procedures or additional support information.

[1078] In this way, the user experience can be improved by taking into account the user's emotions and responding to the process from detecting pests to providing effective countermeasures.

[1079] The processing flow will be explained below.

[1080] Step 1:

[1081] The user launches a dedicated application on their device and takes a photo of any pests they find.

[1082] Step 2:

[1083] The user presses the send button in the application to send the photograph of the pest and environmental information (e.g., humidity, temperature, etc.) to the server.

[1084] Step 3:

[1085] The server receives user-submitted photos of pests, logs them, assigns them unique identifiers, and sends the photos to a generative AI model.

[1086] Step 4:

[1087] The server uses the generative AI model to analyze the received pest photos and identify the pest species, and the analysis results are stored in a database.

[1088] Step 5:

[1089] The server retrieves environmental information provided by the user and previously saved data, including temperature, humidity, cleaning status, etc.

[1090] Step 6:

[1091] The server uses an AI model to generate the optimal extermination method based on the identified pest type and environmental information.

[1092] Step 7:

[1093] Formats the removal instructions and product recommendations generated by the server and prepares them for delivery to the user.

[1094] Step 8:

[1095] The server uses an emotion engine to analyze the voice and text inputs made by the user when they submit a photo and determine the user's emotional state.

[1096] Step 9:

[1097] The server tailors the notification content based on the user's emotional state, for example adding detailed step-by-step guides and support options if the user is feeling highly stressed.

[1098] Step 10:

[1099] The server sends the adjusted notification content to the user's terminal.

[1100] Step 11:

[1101] The user opens the application, reviews the received suggestions, and implements the suggested removal method or product.

[1102] Step 12:

[1103] When the terminal (an optional IoT device) is installed in a building or restaurant, it uses fixed cameras to continuously monitor the environment and transmits video data to a server.

[1104] Step 13:

[1105] The server analyzes the received video data in real time to detect pest infestations, and stores the analysis results in a database as needed.

[1106] Step 14:

[1107] When the server detects the occurrence of pests, it immediately sends an alert to the administrator and suggests the most appropriate extermination measures (for example, where to set traps and how to clean).

[1108] Step 15:

[1109] The administrator implements eradication measures according to instructions from the server and monitors the results again.

[1110] This allows for a fast and efficient process from pest detection to the implementation of effective countermeasures, and also enables responses that take into account the emotional state of users and administrators.

[1111] Example 2

[1112] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1113] Conventional pest control systems have difficulty accurately identifying pest species and providing appropriate extermination methods. In particular, providing information without considering the user's emotions or stress level can sometimes impair the user experience. Furthermore, building and store managers lack the means to detect pest infestations in real time and take prompt action.

[1114] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to take a photo of a pest using an information terminal and send it to the counting device, a means for the counting device to analyze the received photo of the pest using a generative AI model and identify the type of pest, a means for the counting device to generate an extermination method using the generative AI model based on environmental information related to the identified type of pest and send it to the user, and a means for the counting device to analyze the user's emotions from voice or text and adjust the response content. This makes it possible not only to accurately identify the type of pest and provide an appropriate extermination method, but also to respond according to the user's emotional state.

[1115] An "information terminal" is an electronic device that a user uses to take photos and send them to a server, and primarily includes devices such as smartphones and tablets.

[1116] "Aggregator" means the central system that receives, processes, and analyzes data sent by users.

[1117] A "generative AI model" is an artificial intelligence technology used to analyze received data, identify the type of pest, and generate optimal extermination methods.

[1118] "Environmental information" refers to data that indicates the local conditions, such as temperature, humidity, and light conditions, at the location where the pest has occurred.

[1119] A "fixed camera" is a surveillance camera that is installed in a specific location and continuously collects video data.

[1120] "Emotion analysis" is a technology that reads the user's emotional state from the voice and text in the data they send and determines their stress level and anxiety.

[1121] "Control methods" are the means and procedures for effectively eliminating pests, identified based on the type of pest and environmental information.

[1122] This invention is a system that allows a user to take a photo of a pest and send it to a server, which then identifies the type of pest and provides the optimal extermination method. It also has the ability to analyze the user's emotions and provide an appropriate response. This system is implemented primarily using the following hardware and software:

[1123] Hardware and Software Configuration

[1124] User device: An electronic device capable of taking photos and transmitting data, such as a smartphone or tablet.

[1125] Aggregation device (server): A central analysis system that receives, analyzes, and stores data, and runs generative AI models.

[1126] Generative AI model: An analytical model based on deep learning that identifies pest species and generates extermination methods.

[1127] Fixed camera: A surveillance camera installed in a facility or store that provides continuous video data.

[1128] Sentiment analysis engine: A system that reads emotions from a user's voice or text and tailors appropriate responses.

[1129] Specific example of system operation

[1130] 1. User-submitted photos of pests

[1131] Users launch the dedicated application on their smartphone or tablet, take a photo of any pests they find, and then press the send button in the application to send the photo to the server. If necessary, users can also enter environmental information such as humidity and temperature at home or on site.

[1132] Example: A user finds a cockroach in the kitchen, takes a photo with their smartphone, and submits it through an application.

[1133] 2. Receiving and analyzing images on the server

[1134] The server receives photos of pests sent by users, records a log, and assigns a unique identifier to each photo. The server then sends the photos to a generative AI model to identify the type of pest. The identification results are stored in a database.

[1135] Example: The server assigns an identifier such as "User ID_Photo ID_20231001" to a photo and sends the photo data to the generative AI model, which then stores the identification result, "This is cockroach species A," in the database.

[1136] 3. Obtaining environmental information and generating optimal removal methods

[1137] The server receives environmental information provided by the user and previously saved data, and generates the optimal removal method using a generative AI model. It then notifies the user of the selected removal method and product information.

[1138] Example: The server retrieves information from the database, such as "Kitchen humidity: 70%, temperature: 25°C," passes it to a generative AI model, and generates an extermination method, such as "Use an extermination spray suitable for cockroach species A and increase ventilation to reduce humidity," and notifies the user of this information.

[1139] 4. Recognizing and responding to user emotions

[1140] The server uses an emotion analysis engine to analyze the user's emotional state from the voice and text when they send a photo, determining their stress and anxiety levels and tailoring the response accordingly.

[1141] Example: If a user is feeling uncomfortable or stressed, the sentiment analysis engine will recognize the situation and provide detailed guidance such as "How to get rid of cockroach species A: 1) Get the spray. 2) Spray it directly on the cockroaches. 3) Clean the kitchen." or additional support information such as "If you would like to consult with an expert, please contact us here."

[1142] Prompt Sentence Examples

[1143] "I sent a photo of a cockroach I found in my kitchen. It's a cockroach species A. How can I get rid of it in a home with high humidity? How can you support users when they're feeling anxious?"

[1144]

[1145] "Fixed cameras have detected rats. Please provide advice on the appropriate trap locations and cleaning methods. What additional support do managers need if they are stressed?"

[1146] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1147] Step 1:

[1148] Users launch a dedicated application on their smartphone or tablet and take a photo of the pest. Then, they press the send button in the application to send the photo to the server. If necessary, they can enter environmental information such as humidity and temperature at home or on the job site.

[1149] (Input) Pest photos, environmental information

[1150] (Output) Photo data and environmental information sent to the server

[1151] (Specific operation) A user finds a cockroach in the kitchen, takes a photo of the cockroach with their smartphone, and taps the send button in the app to send the data to the server.

[1152] Step 2:

[1153] The server receives photos of pests submitted by users, assigns them unique identifiers, records logs, and stores the photo data and environmental information in a database.The server then sends the photos to a generative AI model to identify the type of pest.

[1154] (Input) Photo data and environmental information sent by the user

[1155] (Output) Data stored in the database, unique identifiers assigned, and pest species identification results

[1156] (Specific operation) The server assigns an identifier such as "User ID_Photo ID_20231001" to the photo and sends the photo data to the generative AI model for analysis.

[1157] Step 3:

[1158] The server uses a generative AI model to analyze the received photo data and identify the type of pest, storing the identification results in a database and using them to generate the next necessary extermination method.

[1159] (Input) Photo data, generative AI model

[1160] (Output) Pest identification results

[1161] (Specific operation) The generative AI model analyzes the photo, generates an identification result that says, "This is cockroach species A," and stores this in the database.

[1162] Step 4:

[1163] The server acquires environmental information provided by the user and previously saved environmental data, generates an extermination method suitable for the type of pest, and sends the generated extermination method and product information to the user.

[1164] (Input) Pest identification results, environmental information, past environmental data

[1165] (Output) The best method of eradication and its information

[1166] (Specific operation) The server generates notifications to the user based on data such as "The humidity in the home is high" and "In the case of cockroach species A, use an extermination spray."

[1167] Step 5:

[1168] The server uses the voice and text input from the user when sending a photo to analyze the user's emotional state using an emotion analysis engine, and adjusts the format and content of the information provided based on this.

[1169] (Input) User voice and text messages

[1170] (Output) Adjusting response content

[1171] (Specific operation) If a user sends a comment such as "It was really scary," the emotion analysis engine will determine that the emotional state is at a high stress level.

[1172] Step 6:

[1173] The server generates a response tailored to the user's emotional state and notifies the user, providing reassuring information such as detailed countermeasure procedures and options for consulting with an expert.

[1174] (Input) Emotion analysis results

[1175] (Output) A tailored response that is communicated to the user

[1176] (Specific actions) The server sends a detailed guide such as "How to get rid of cockroach species A: 1) Get the spray. 2) Spray it directly on the cockroaches. 3) Clean the kitchen." It also provides additional support information such as "If you would like to consult with an expert, please contact us here."

[1177] (Application example 2)

[1178] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1179] With conventional pest control systems, users often experienced a great deal of stress between discovering pests and finding the appropriate extermination method. Furthermore, the information provided about pest species and extermination methods was inconsistent, leaving room for improvement in the user experience. Furthermore, the system was unable to respond in a way that took the user's emotional state into account, resulting in low user satisfaction. To solve these problems, a system that could analyze the user's emotional state and respond flexibly based on that was needed.

[1180] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1181] In this invention, the server includes: [means for a user to take a photo of a pest using a terminal and send it to the server; [means for the server to analyze the received photo of the pest using a generative AI model and identify the type of pest; and [means for the server to generate an extermination method using the generative AI model based on the identified type of pest and related environmental information and send it to the user.] This allows the user to quickly and accurately identify the type of pest and provide the optimal extermination method, enabling effective pest extermination.

[1182] In addition, the server uses an emotion engine to analyze the user's emotional state from voice and text, and adjusts the extermination method provided, thereby reducing stress and anxiety when the user discovers a pest, improving the user experience.

[1183] A "terminal" is a device used by a user, such as a mobile information terminal, a smartphone, or a tablet.

[1184] A "server" is a central device that receives, analyzes, stores, and provides results on a computer network for image data and environmental information.

[1185] A "generative AI model" is an algorithm that uses artificial intelligence technologies such as deep learning to analyze image data and generate information on the type of pest and appropriate extermination methods.

[1186] An "emotion engine" is a system that analyzes a user's emotional state from their voice and text and provides appropriate responses based on that.

[1187] A "fixed camera" is a device such as a surveillance camera that is installed in a specific location such as a building or restaurant and provides video footage in real time.

[1188] "Environmental information" refers to information such as the temperature, humidity, lighting conditions, and food management status of the area where pests live.

[1189] "Extermination method" is a general term for procedures, chemicals, traps, etc. used to efficiently exterminate specific pests.

[1190] "Emotional state" refers to a user's psychological state, such as stress, anxiety, anger, or joy.

[1191] "User experience" refers to the overall satisfaction and emotional response a user feels when using a system or service.

[1192] This invention is a system for identifying pests found by users and supporting appropriate extermination methods. To improve the user experience, the system features an emotion engine that provides responses tailored to the user's emotional state. The system is composed of a combination of hardware and software, including a user terminal, a server, a generative AI model, an emotion engine, and a fixed camera.

[1193] 1. Send photos of pests

[1194] First, the user launches the dedicated application on their device (such as a smartphone or tablet) and takes a photo of the pest they find. After taking the photo, the user presses the send button in the application to send the photo to the server. If necessary, the user can also enter environmental information such as the temperature and humidity of their home or work site.

[1195] 2. Image Analysis

[1196] The server receives photos of pests sent by users. It then uses a generative AI model to analyze the received images and identify the type of pest. This generative AI model is built on deep learning technology and can identify the characteristics of pests with high accuracy. The analyzed information is stored in a database.

[1197] 3. Generating removal methods

[1198] The server generates the optimal extermination method based on the identified pest species and environmental information provided by the user. Using this generative AI model, an efficient and effective extermination method is automatically selected. The selected extermination method and related product information are then notified to the user.

[1199] 4. Emotion recognition

[1200] The emotion engine analyzes the voice and text information when users send photos to determine their emotional state. Developed based on an empirical psychological model, the engine accurately recognizes emotions such as stress, anxiety, and anger.

[1201] 5. Response and Notification

[1202] The application adapts the type of information it provides depending on the user's emotional state: for example, if the user is feeling highly stressed, it will provide more detailed removal instructions and step-by-step guides, while if the user is feeling anxious, it will offer additional support and options to consult with a specialist.

[1203] Specific examples

[1204] If a store staff member spots a rat in the backroom, they can take a photo of it using a dedicated smartphone app and send it to the server. The server will immediately identify the type of rat and suggest effective extermination methods and where to set traps. If a staff member is feeling anxious or stressed, the emotion engine will detect this and provide more detailed instructions or the option to consult with a specialist.

[1205] Prompt Sentence Examples

[1206] "Images taken with the app are sent to a server, and a generative AI model is used to identify the type of pest. The app then generates the optimal extermination method based on environmental information, and uses an emotion engine to provide a response that reflects the user's emotional state."

[1207] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1208] Step 1:

[1209] The user uses the device to take a photo of the pest.

[1210] Input: A user launches a smartphone app and takes a photo of a pest.

[1211] How it works: Users can take photos of pests in a dedicated application and manually enter the necessary environmental information (temperature, humidity, etc.).

[1212] Output: The captured photos of the pests and environmental information are saved on the device.

[1213] Step 2:

[1214] The device sends photos and environmental information to the server.

[1215] Input: Pest photos and environmental information stored on your device.

[1216] How it works: The user presses the send button in the application, sending the photo and environmental information to the server.

[1217] Output: The server receives the pest photos and environmental information.

[1218] Step 3:

[1219] The image received by the server is analyzed using a generative AI model to identify the type of pest.

[1220] Input: A photo of a pest received by the server.

[1221] How it works: The server inputs image data into a generative AI model, which uses deep learning to analyze and identify the type of pest. The identified data is then stored in a database.

[1222] Output: Pest species information.

[1223] Step 4:

[1224] The server generates the optimal extermination method based on the identified pest type and environmental information.

[1225] Input: Identified pest species information and environmental information.

[1226] How it works: The server uses a generative AI model to combine pest species and environmental information to generate the optimal extermination method.

[1227] Output: Information on the best removal methods and related products.

[1228] Step 5:

[1229] The server analyzes the user's emotional state from voice and text using an emotion engine.

[1230] Input: Voice and text data entered by the user when sending a photo.

[1231] How it works: The emotion engine allows the server to analyze the user's emotional state and identify emotions such as stress, anxiety, and anger.

[1232] Output: User's emotional state information.

[1233] Step 6:

[1234] The server adjusts its offerings of extermination methods based on the emotional state.

[1235] Input: optimal removal method and user emotional state information.

[1236] How it works: The server adapts the type and level of detail of the information it provides based on the user's emotional state, for example by adding detailed instructions or the option to consult with an expert.

[1237] Output: Adjusted removal instructions and supporting information.

[1238] Step 7:

[1239] The server sends coordinated removal instructions and support information to the user.

[1240] Input: Coordinated removal instructions and support information.

[1241] Operation: The server sends the generated removal instructions and support information to the user's device.

[1242] Output: Removal instructions and support information are made available to the user on their device.

[1243] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1244] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1245] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1246] [Fourth embodiment]

[1247] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1248] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1249] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1250] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1251] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1252] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1253] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1254] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1255] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1256] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1257] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1258] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1259] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1260] This invention relates to a system that identifies pest species and proposes optimal extermination methods based on the current environment. Specifically, a user takes a photo of the pest using a device and sends it to a server. The server then analyzes the received photo using a generative AI model to identify the type of pest. The server then generates an optimal extermination method based on the identification results and the user's environmental information, enabling it to send the method to the user.

[1261] System configuration

[1262] User device (smartphone, tablet, etc.)

[1263] Server (central analysis system)

[1264] Generative AI model (analysis model based on deep learning)

[1265] Fixed camera (optional surveillance camera)

[1266] IoT devices (for environmental monitoring in buildings, restaurants, etc.)

[1267] Program processing explanation

[1268] 1. User-submitted photos of pests

[1269] First, the user launches the dedicated application on their device and takes a photo of the pest they find. Next, the user presses the send button in the application to send the photo to the server. If necessary, the user can also enter environmental information such as humidity and temperature at home or on site.

[1270] 2. Receiving and analyzing images on the server

[1271] The server receives photos of pests sent by users, records a log, assigns a unique identifier, and sends the received photos to a generative AI model, which analyzes the received images and identifies the type of pest. The identification results are stored in a database.

[1272] 3. Obtaining environmental information and generating optimal removal methods

[1273] The server receives environmental information provided by the user and previously saved data, and uses a generative AI model to generate the optimal extermination method based on the type of pest and environmental information.The server then sends the selected extermination method and product information to the user.

[1274] 4. Notice to Users

[1275] The server then sends the generated recommendations to the user's device, where the user can view details of the suggested removal methods and products and act accordingly through the application.

[1276] Specific examples

[1277] User Case

[1278] When a user finds a cockroach in their kitchen, they take a photo of it using a dedicated app and send it to the server. The server identifies the type of cockroach, for example, "Cockroach Species A." Based on environmental information, such as the tendency for humidity in the home, the app then suggests effective ways to use cockroach extermination spray and how to manage grains and food. The user can then click a link within the app to purchase the extermination spray.

[1279] Building maintenance case

[1280] The building manager uses fixed cameras installed in the backyard of the building to monitor the environment in real time. The server analyzes the footage from the fixed cameras and detects rats. The server immediately sends an alert to the manager and suggests appropriate trap locations and cleaning methods. The manager implements countermeasures according to the instructions and then monitors again.

[1281] This system allows for the rapid detection of pests and the implementation of effective countermeasures, enabling users and administrators to take appropriate measures.

[1282] The processing flow will be explained below.

[1283] Step 1:

[1284] The user uses the device to launch a dedicated application and take a photo of the pest.

[1285] Step 2:

[1286] The user presses the send button in the application to send the photograph of the pest they have taken to the server. If necessary, the user can also input environmental information (humidity, temperature, etc.) and send it at the same time.

[1287] Step 3:

[1288] The server receives the pest photos sent by the user, logs them, and assigns them a unique identifier.

[1289] Step 4:

[1290] The server sends the received photo of the pest to the generative AI model, which then analyzes the image and identifies the type of pest.

[1291] Step 5:

[1292] The server stores the analysis results in a database, which includes the type of pest, the date and time the photo was taken, location information (if necessary), and environmental information.

[1293] Step 6:

[1294] The server acquires environmental information provided by the user and previously saved data. The acquired environmental information includes the temperature, humidity, cleaning status, etc. at a specific location.

[1295] Step 7:

[1296] The server uses a generative AI model to generate the optimal extermination method based on the identified pest species and the acquired environmental information. This generative AI model determines effective extermination methods based on the pest species and environmental conditions.

[1297] Step 8:

[1298] The server then formats the generated removal instructions for the user, generating content that includes removal instructions details, recommended product information, and links to where to purchase them.

[1299] Step 9:

[1300] The server sends the formatted proposal to the user's terminal.

[1301] Step 10:

[1302] The user opens the application and reviews the received suggestions. The user implements the suggested action (e.g., purchases the recommended product and follows its instructions).

[1303] Step 11:

[1304] Terminals (IoT device option) are installed in buildings and restaurants, and video data is acquired using fixed cameras. This video data is continuously sent to a server.

[1305] Step 12:

[1306] The server analyzes the received video data in real time and monitors for pest infestations. Based on the analysis results, an alert is generated immediately if a pest infestation is detected.

[1307] Step 13:

[1308] The server notifies the administrator of detected pest information and immediately suggests effective extermination measures, including where to set traps and how to clean the area.

[1309] Step 14:

[1310] Based on the notification received from the server, the administrator will implement the specified disinfection measures and cleaning, then monitor again and feed the results back to the server.

[1311] These steps result in a system where the process from pest detection to providing effective treatment is carried out quickly and efficiently.

[1312] Example 1

[1313] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1314] The present invention relates to a system that quickly and accurately identifies pest species and proposes optimal extermination methods to users or administrators. Conventional pest control methods rely heavily on human labor, which takes time to identify and select appropriate extermination methods, and can lead to incorrect responses. In particular, implementing optimal measures that take environmental information and past data into account requires specialized knowledge, which is difficult for average users to understand. This has led to problems such as delayed pest control and insufficient measures.

[1315] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1316] In this invention, the server includes: [means for a user to take a photo of a pest using a terminal and send it to the server; [means for the server to preprocess the received photo of the pest and analyze it with a generative AI model to identify the type of pest; and [means for the server to generate an optimal extermination method using the generative AI model based on current environmental information and past data related to the identified type of pest and send it to the user.] This makes it possible to quickly and accurately identify the type of pest and suggest the optimal extermination method.

[1317] A "terminal" is an electronic device operated by a user, and primarily refers to a portable information terminal such as a smartphone or tablet.

[1318] "Server" refers to the central processing unit that receives, processes, and stores data sent by users and analyzes it using the generative AI model.

[1319] A "generative AI model" is an analytical algorithm based on deep learning technology that identifies the type of pest based on input data and generates the optimal extermination method.

[1320] "Environmental information" refers to data provided by users, including information such as temperature, humidity, and geographic location, that is used to determine pest occurrence and optimize extermination methods.

[1321] A "fixed camera" refers to a surveillance camera installed in a building or facility that provides video data in real time.

[1322] "Control method" refers to the specific means or product application that effectively eliminates the targeted pest.

[1323] A "log" refers to a record of a series of operations and events that occur within a system, including received image data and its processing status.

[1324] "Preprocessing" refers to the processing performed to make the received image data easier to analyze by the generative AI model, and refers to operations such as resizing and noise removal.

[1325] "Identification Results" refers to information about the type of pest identified by the generative AI model.

[1326] A "database" refers to a collection of data that systematically stores identification results, environmental information, past data, etc., and makes them searchable and accessible as needed.

[1327] This invention is a system that allows users to take photos of pests and send them to a server, and then uses a generative AI model to identify the type of pest and suggest the optimal extermination method. Specifically, the system uses the following hardware and software:

[1328] Hardware and software used

[1329] User terminal: Refers to a device such as a smartphone or tablet used to take photographs and operate applications.

[1330] Server: The central analysis system that receives, processes, and analyzes data.

[1331] Generative AI model: An analytical model based on deep learning that identifies the type of pest and generates the optimal extermination method.

[1332] Fixed camera (optional): A surveillance camera installed in a building or facility that transmits video data to a server in real time.

[1333] IoT devices (optional): Refers to sensors used for environmental monitoring in buildings, restaurants, etc.

[1334] System operation explanation

[1335] User-submitted photos of pests

[1336] The user launches a dedicated application on their device and takes a photo of the pest they find. For example, they can use the camera function of their smartphone to capture a high-resolution image. They then press the send button in the application to send the captured photo data to the server. At this time, the user can also enter environmental information (temperature, humidity, etc.) if necessary.

[1337] Image reception and preprocessing on the server

[1338] The server receives photo data sent by users, records logs, assigns a unique identifier, checks the quality of the received images, and performs preprocessing (size adjustment, noise removal, etc.). The preprocessed image data is then sent to the generative AI model.

[1339] Pest species identification using generative AI models

[1340] The generative AI model receives preprocessed images sent from the server and uses deep learning techniques to identify the type of pest. Specifically, it extracts features from the image and identifies the type of pest through comparison with existing labeled data. The identification results are sent back to the server and stored in a database.

[1341] Acquiring and analyzing environmental information on the server

[1342] The server analyzes the environmental information provided by the user and similar environmental data retrieved from a past database. This evaluates the pest occurrence trends and the effectiveness of extermination methods. Based on the analysis results, a generative AI model is used to generate the optimal extermination method.

[1343] Generate and send optimal removal instructions

[1344] The server then notifies the user of the optimal removal method obtained from the generative AI model. Specifically, it creates a detailed proposal including the selected removal method and product information and sends it to the user's device. The user can then review the proposal through the application and implement countermeasures according to the instructions.

[1345] Specific examples

[1346] User Case

[1347] When a user finds a cockroach in their kitchen, they take a photo of it using a dedicated app and send it to the server. The server identifies the type of cockroach, for example, recognizing it as "Cockroach Species A." Based on the user's environmental information, such as the fact that humidity in the home tends to be high, the app then suggests effective ways to use cockroach extermination spray and how to manage grains and food. The user can then click on a link within the app to purchase the extermination spray.

[1348] Building maintenance case

[1349] The building manager monitors the environment in real time using fixed cameras installed in the backyard of the building. The server analyzes the footage from the fixed cameras and detects rats. The server immediately sends an alert to the manager and suggests appropriate locations for traps and cleaning methods. The manager implements countermeasures according to the instructions and then monitors again. This system allows for rapid detection of pests and the implementation of effective countermeasures, enabling users and managers to take appropriate measures.

[1350] Prompt Sentence Examples

[1351] 1. "I want to take photos of pests with my smartphone, send them to a server, and receive analysis results and recommendations for the best extermination method."

[1352] 2. "A system that analyzes footage from surveillance cameras installed in office buildings, immediately notifies the operator when pests are discovered, and suggests the most appropriate countermeasures."

[1353] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1354] Step 1:

[1355] A user takes a photo of the pest and submits it

[1356] Specific operation: The user launches the dedicated application using a device such as a smartphone or tablet. They take a photo of the pest they find using the device's camera. After taking the photo, they press the send button in the application to send the photo data and, if necessary, environmental information (temperature, humidity, etc.) to the server.

[1357] Input: User-taken photo data of the pest, and optionally environmental information.

[1358] Output: Photo data and environment information sent to the server.

[1359] Step 2:

[1360] The server receives the photo data and performs preprocessing.

[1361] Specific operation: The server receives photo data sent by users, logs the received data, assigns a unique identifier, checks the quality of the received image, and performs image preprocessing (e.g., resizing, noise reduction) if necessary.

[1362] Input: Photo data submitted by the user.

[1363] Output: Preprocessed image data.

[1364] Step 3:

[1365] A generative AI model analyzes image data to identify the type of pest.

[1366] How it works: The server sends the preprocessed image data to the generative AI model, which uses deep learning techniques to extract features from the image and identify the type of pest by matching it with existing labeled data. The identification results are stored in a database.

[1367] Input: Preprocessed image data.

[1368] Output: Identification results specifying the pest species.

[1369] Step 4:

[1370] The server acquires and analyzes current environmental information and past data.

[1371] Specific operation: The server obtains current environmental information provided by the user and past environmental data stored in the database. Based on this information, it evaluates and analyzes trends in pest occurrence and the effectiveness of extermination methods.

[1372] Input: Current environmental information and historical environmental data.

[1373] Output: Analysis results on pest occurrence trends and effectiveness of control methods.

[1374] Step 5:

[1375] A generative AI model generates the optimal removal method and notifies the user.

[1376] Specific operation: The server uses a generative AI model to generate the optimal extermination method based on the identified pest species and analyzed environmental information. It then creates a detailed proposal including the generated extermination method and product information and sends it to the user's device. The user can then review the proposal through the application and implement countermeasures according to the instructions.

[1377] Input: Identified pest species and analyzed environmental information.

[1378] Output: Notification of best removal method and product information, recommendations.

[1379] This allows the user to take prompt and appropriate measures to exterminate pests.

[1380] (Application example 1)

[1381] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1382] When pests occur in a physical store, they must be exterminated quickly and effectively. Conventional methods lack sufficient means for identifying pests and quickly finding the optimal extermination method, which is time-consuming and labor-intensive. After discovering pests, purchasing the appropriate extermination products and learning how to use them also takes a lot of time. Furthermore, in stores that do not use fixed-point cameras for monitoring, it is difficult to grasp the occurrence of pests in real time. This can have a negative impact on the cleanliness of the physical store and business activities.

[1383] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1384] In this invention, the server includes: [means for a user to take a photo of a pest using a terminal and send it to the information device; [means for the information device to analyze the received photo of the pest using a generative AI model and identify the type of pest; and [means for the information device to generate an extermination method using the generative AI model based on the identified type of pest and related environmental information and send it to the user.] This makes it possible to quickly identify the type of pest and generate an optimal extermination method. Furthermore, users can instantly purchase extermination supplies through their terminal, enabling effective measures to maintain the cleanliness of physical stores. In addition, fixed cameras can be used to monitor pest occurrence in real time and implement extermination measures immediately.

[1385] A "terminal" is an electronic device used by a user to take photos of pests and send them to a server.

[1386] The "information device" is a central analysis system that receives photo data sent by users, analyzes the images using a generative AI model, and generates a removal method.

[1387] A "generative AI model" is a computational model that uses deep learning technology to identify pest species and suggest appropriate extermination methods.

[1388] A "fixed camera" is a surveillance camera device that is installed in a specific location and provides images in real time.

[1389] "Environmental information" refers to data provided by users, such as building or site temperature, humidity, and food types.

[1390] A "link" is information displayed on the device about a destination on the Internet for purchasing the appropriate pest control products.

[1391] The "extermination method" is the optimal pest extermination method suggested by the generative AI model based on the type of pest and environmental information.

[1392] A "User" is a person who uses the System to detect and eradicate pests in a physical store or building.

[1393] A "manager" is a person in charge of the operation and management of a building or store.

[1394] "Real-time" refers to the system's ability to immediately detect pest infestations and quickly implement appropriate extermination measures.

[1395] This invention describes a system that can quickly identify pest species in brick-and-mortar stores and propose optimal extermination methods. The system's main components include a terminal, an information device (server), a generative AI model, a fixed camera, and environmental sensors. It also includes a dedicated application for users.

[1396] Hardware and software used

[1397] 1. Device (smartphone or tablet):

[1398] This is a device that allows users to take photos of pests and send them to the information device.

[1399] 2. Information device (server):

[1400] It analyzes incoming photos of pests and uses a generative AI model to identify the type of pest.

[1401] Furthermore, the system generates an optimal removal method based on the identification results and related environmental information and sends it to the user.

[1402] 3. Generative AI Model:

[1403] Deep learning technology is used to identify the type of pest and generate the optimal extermination method.

[1404] Specifically, TensorFlow and PyTorch can be used.

[1405] 4. Fixed Camera:

[1406] These are surveillance cameras installed in specific locations within the store and provide real-time footage.

[1407] For example, a Raspberry Pi camera can be used.

[1408] 5. Environmental Sensors:

[1409] It is an IoT device for monitoring the temperature and humidity inside a store.

[1410] For example, using a DHT22 sensor.

[1411] Program processing explanation

[1412] 1. Take a photo of the pest and send it to us

[1413] The user takes a photo of the pest using a device and sends it to the information device via a dedicated application. The application is designed to send environmental information such as temperature and humidity along with the photo.

[1414] 2. Image analysis and pest identification

[1415] The information device analyzes the received photos and uses a generative AI model to identify the type of pest. The generative AI model uses deep learning technology and is capable of highly accurate identification based on past learning data. The analysis results of the generative AI model are stored in a database.

[1416] 3. Generating and notifying removal instructions

[1417] The information device uses a generative AI model to generate the optimal extermination method based on the identified pest species and environmental information provided by the user. The generated extermination method and recommended extermination products are then sent to the user's device, which also provides a function to instantly purchase extermination products based on this information.

[1418] 4. Real-time monitoring using fixed cameras

[1419] Fixed cameras monitor specific areas within a store in real time and transmit the images to an information device. The information device analyzes the video data and immediately notifies the manager if a pest infestation is detected. This notification also includes specific extermination methods.

[1420] Specific examples

[1421] For example, if a store staff member finds a cockroach in the kitchen, they can take a photo with their device and send it to the information device via a dedicated application. The information device then uses a generative AI model to identify the type of cockroach and, taking into account humidity and temperature information, suggests the optimal extermination spray and management method. The suggested method and product are instantly notified to the device, along with a link to purchase the extermination products.

[1422] In addition, if fixed cameras are monitoring the backyard, the information device will analyze the footage in real time and send out an alert if it detects any pests. For example, if a mouse is detected, it will also suggest appropriate locations to set traps and cleaning methods.

[1423] Prompt Sentence Examples

[1424] Input prompt: "Identify the type of pest in this photo."

[1425] Output: "This pest is cockroach species A."

[1426] Input prompt: "Please suggest the best method to control cockroach species A."

[1427] Output: "Use an insecticide spray appropriate for your current environment and take good care of your food."

[1428] This allows for fast and effective pest management in brick-and-mortar stores.

[1429] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1430] Step 1:

[1431] Users take photos of pests using their devices and send them to the server via a dedicated application, where the photos become input data.

[1432] Input: Pest photo, (optional) environmental information (temperature, humidity)

[1433] Output: Photos and environment information sent to the server

[1434] How it works: The user launches the application, uses the camera to take a photo of a pest, and then presses the send button to send the data to the server.

[1435] Step 2:

[1436] The photo data of pests received by the server is input into a generative AI model for analysis, and the type of pest is identified.

[1437] Input: User-submitted photo of a pest

[1438] Output: Pest species identification result

[1439] How it works: After the server receives the photo data, it sends it to the generative AI model, which analyzes the image, identifies the type of pest, and stores the results in a database.

[1440] Step 3:

[1441] The server obtains environmental information provided by the user and previously saved environmental data, and based on this, a generative AI model generates the optimal eradication method.

[1442] Input: Pest species, user's environmental information, historical environmental data

[1443] Output: Recommendation of the best removal method

[1444] How it works: The server retrieves historical environmental data from the database, combines it with current environmental information, and feeds it into a generative AI model. The model calculates the optimal removal method and generates a recommendation.

[1445] Step 4:

[1446] The server generates a proposed removal method and sends the associated removal product to the user's device.

[1447] Input: Best extermination method, relevant extermination product information

[1448] Output: Notification sent to the user's device

[1449] How it works: The server sends details of the proposed removal method and product to the user's device and sends a notification. The user can then review the information in the application and access a link to purchase the product.

[1450] Step 5:

[1451] Real-time video footage is sent from the fixed camera location to a server, which then analyzes the footage to detect the occurrence of pests.

[1452] Input: Real-time video from a fixed camera

[1453] Output: Pest infestation detection results, alarm

[1454] How it works: Fixed cameras continuously transmit video footage to a server, which then uses a generative AI model to analyze the video data in real time and send an alert to the administrator if a pest is detected.

[1455] Step 6:

[1456] The server notifies the administrator of immediate removal measures and suggests specific removal methods.

[1457] Input: Pest infestation detection results

[1458] Output: Notification to administrator, specific removal instructions

[1459] How it works: If the server detects a pest outbreak, it immediately notifies the administrator and provides details on the appropriate extermination method, allowing for a prompt response.

[1460] This series of steps enables quick identification of pests in physical stores and the suggestion of effective extermination methods.

[1461] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1462] This invention improves the user experience by incorporating an emotion engine that recognizes the user's emotions in addition to a system that identifies the type of pest and suggests the optimal extermination method based on the current environment. Specifically, the user takes a photo of the pest using their device and sends it to a server. The server then analyzes the received photo using a generative AI model to identify the type of pest. The server then generates the optimal extermination method based on the identification results and the user's environmental information and sends it to the user. The emotion engine also responds based on the user's emotional state.

[1463] System configuration

[1464] User device (smartphone, tablet, etc.)

[1465] Server (central analysis system)

[1466] Generative AI model (analysis model based on deep learning)

[1467] Fixed camera (optional surveillance camera)

[1468] IoT devices (for environmental monitoring in buildings, restaurants, etc.)

[1469] Emotion engine (a system that recognizes and analyzes user emotions)

[1470] Program processing explanation

[1471] 1. User-submitted photos of pests

[1472] First, the user launches the dedicated application on their device and takes a photo of the pest they find. Next, the user presses the send button in the application to send the photo to the server. If necessary, the user can also enter environmental information such as humidity and temperature at home or on site.

[1473] 2. Receiving and analyzing images on the server

[1474] The server receives photos of pests sent by users, records a log, assigns a unique identifier, and sends the received photos to a generative AI model, which analyzes the received images and identifies the type of pest. The identification results are stored in a database.

[1475] 3. Obtaining environmental information and generating optimal removal methods

[1476] The server receives environmental information provided by the user and previously saved data, and uses a generative AI model to generate the optimal extermination method based on the type of pest and environmental information.The server then sends the selected extermination method and product information to the user.

[1477] 4. User Emotion Recognition

[1478] The server uses an emotion engine to analyze the emotional state of the user's voice and text as they submit photos, determining their stress and anxiety levels and adjusting the content and format of their responses accordingly.

[1479] 5. Emotional response and notification to users

[1480] The server adapts the type of information it provides depending on the user's emotional state: for example, it provides more detailed explanations and step-by-step guides if the user is feeling highly stressed, or it offers additional support and options for consulting a specialist if the user is feeling particularly anxious.

[1481] Specific examples

[1482] User Case

[1483] When a user finds a cockroach in their kitchen, they take a photo of it using a dedicated app and send it to the server. The server identifies the type of cockroach, for example, recognizing it as "Cockroach Species A." Based on environmental information indicating that the home tends to be humid, the app then suggests effective ways to use cockroach extermination spray and food management. If the user is experiencing strong discomfort or stress, the emotion engine recognizes the situation and provides detailed instructions on extermination measures and additional support options.

[1484] Building maintenance case

[1485] Building managers use fixed cameras installed in the backyard of the building to monitor the environment in real time. The server analyzes the footage from the fixed cameras and detects rats. The server immediately sends an alert to the manager and suggests appropriate locations for traps and cleaning methods. If the manager is feeling stressed, the emotion engine recognizes this and flexibly adjusts the response, for example by providing detailed countermeasure procedures or additional support information.

[1486] In this way, the user experience can be improved by responding to the process from pest detection to the provision of effective countermeasures while taking into account the user's emotions.

[1487] The processing flow will be explained below.

[1488] Step 1:

[1489] The user launches a dedicated application on their device and takes a photo of any pests they find.

[1490] Step 2:

[1491] The user presses the send button in the application to send the photograph of the pest and environmental information (e.g., humidity, temperature, etc.) to the server.

[1492] Step 3:

[1493] The server receives user-submitted photos of pests, logs them, assigns them unique identifiers, and sends the photos to a generative AI model.

[1494] Step 4:

[1495] The server uses the generative AI model to analyze the received pest photos and identify the pest species, and the analysis results are stored in a database.

[1496] Step 5:

[1497] The server retrieves environmental information provided by the user and previously saved data, including temperature, humidity, cleaning status, etc.

[1498] Step 6:

[1499] The server uses an AI model to generate the optimal extermination method based on the identified pest type and environmental information.

[1500] Step 7:

[1501] Formats the removal instructions and product recommendations generated by the server and prepares them for delivery to the user.

[1502] Step 8:

[1503] The server uses an emotion engine to analyze the voice and text inputs made by the user when they submit a photo and determine the user's emotional state.

[1504] Step 9:

[1505] The server tailors the notification content based on the user's emotional state, for example adding detailed step-by-step guides and support options if the user is feeling highly stressed.

[1506] Step 10:

[1507] The server sends the adjusted notification content to the user's terminal.

[1508] Step 11:

[1509] The user opens the application, reviews the received suggestions, and implements the suggested removal method or product.

[1510] Step 12:

[1511] When the terminal (an optional IoT device) is installed in a building or restaurant, it uses fixed cameras to continuously monitor the environment and transmits video data to a server.

[1512] Step 13:

[1513] The server analyzes the received video data in real time to detect pest infestations, and stores the analysis results in a database as needed.

[1514] Step 14:

[1515] When the server detects the occurrence of pests, it immediately sends an alert to the administrator and suggests the most appropriate extermination measures (for example, where to set traps and how to clean).

[1516] Step 15:

[1517] The administrator implements eradication measures according to instructions from the server and monitors the results again.

[1518] This allows for a fast and efficient process from pest detection to the implementation of effective countermeasures, and also enables responses that take into account the emotional state of users and administrators.

[1519] Example 2

[1520] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1521] Conventional pest control systems have difficulty accurately identifying pest species and providing appropriate extermination methods. In particular, providing information without considering the user's emotions or stress level can sometimes impair the user experience. Furthermore, building and store managers lack the means to detect pest infestations in real time and take prompt action.

[1522] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for a user to take a photo of a pest using an information terminal and send it to the counting device, a means for the counting device to analyze the received photo of the pest using a generative AI model and identify the type of pest, a means for the counting device to generate an extermination method using the generative AI model based on environmental information related to the identified type of pest and send it to the user, and a means for the counting device to analyze the user's emotions from voice or text and adjust the response content. This makes it possible not only to accurately identify the type of pest and provide an appropriate extermination method, but also to respond according to the user's emotional state.

[1523] An "information terminal" is an electronic device that a user uses to take photos and send them to a server, and primarily includes devices such as smartphones and tablets.

[1524] "Aggregator" means the central system that receives, processes, and analyzes data sent by users.

[1525] A "generative AI model" is an artificial intelligence technology used to analyze received data, identify the type of pest, and generate optimal extermination methods.

[1526] "Environmental information" refers to data that indicates the local conditions, such as temperature, humidity, and light conditions, at the location where the pest has occurred.

[1527] A "fixed camera" is a surveillance camera that is installed in a specific location and continuously collects video data.

[1528] "Emotion analysis" is a technology that reads the user's emotional state from the voice and text in the data they send and determines their stress level and anxiety.

[1529] "Control methods" are the means and procedures for effectively eliminating pests, identified based on the type of pest and environmental information.

[1530] This invention is a system that allows a user to take a photo of a pest and send it to a server, which then identifies the type of pest and provides the optimal extermination method. It also has the ability to analyze the user's emotions and provide an appropriate response. This system is implemented primarily using the following hardware and software:

[1531] Hardware and Software Configuration

[1532] User device: An electronic device capable of taking photos and transmitting data, such as a smartphone or tablet.

[1533] Aggregation device (server): A central analysis system that receives, analyzes, and stores data, and runs generative AI models.

[1534] Generative AI model: An analytical model based on deep learning that identifies pest species and generates extermination methods.

[1535] Fixed camera: A surveillance camera installed in a facility or store that provides continuous video data.

[1536] Sentiment analysis engine: A system that reads emotions from a user's voice or text and tailors appropriate responses.

[1537] Specific example of system operation

[1538] 1. User-submitted photos of pests

[1539] Users launch the dedicated application on their smartphone or tablet, take a photo of any pests they find, and then press the send button in the application to send the photo to the server. If necessary, users can also enter environmental information such as humidity and temperature at home or on site.

[1540] Example: A user finds a cockroach in the kitchen, takes a photo with their smartphone, and submits it through an application.

[1541] 2. Receiving and analyzing images on the server

[1542] The server receives photos of pests sent by users, records a log, and assigns a unique identifier to each photo. The server then sends the photos to a generative AI model to identify the type of pest. The identification results are stored in a database.

[1543] Example: The server assigns an identifier such as "User ID_Photo ID_20231001" to a photo and sends the photo data to the generative AI model, which then stores the identification result, "This is cockroach species A," in the database.

[1544] 3. Obtaining environmental information and generating optimal removal methods

[1545] The server receives environmental information provided by the user and previously saved data, and generates the optimal removal method using a generative AI model. It then notifies the user of the selected removal method and product information.

[1546] Example: The server retrieves information from the database, such as "Kitchen humidity: 70%, temperature: 25°C," passes it to a generative AI model, and generates an extermination method, such as "Use an extermination spray suitable for cockroach species A and increase ventilation to reduce humidity," and notifies the user of this information.

[1547] 4. Recognizing and responding to user emotions

[1548] The server uses an emotion analysis engine to analyze the user's emotional state from the voice and text when they send a photo, determining their stress and anxiety levels and tailoring the response accordingly.

[1549] Example: If a user is feeling uncomfortable or stressed, the sentiment analysis engine will recognize the situation and provide detailed guidance such as "How to get rid of cockroach species A: 1) Get the spray. 2) Spray it directly on the cockroaches. 3) Clean the kitchen." or additional support information such as "If you would like to consult with an expert, please contact us here."

[1550] Prompt Sentence Examples

[1551] "I sent a photo of a cockroach I found in my kitchen. It's a cockroach species A. How can I get rid of it in a home with high humidity? How can you support users when they're feeling anxious?"

[1552]

[1553] "Fixed cameras have detected rats. Please provide advice on the appropriate trap locations and cleaning methods. What additional support do managers need if they are stressed?"

[1554] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1555] Step 1:

[1556] Users launch a dedicated application on their smartphone or tablet and take a photo of the pest. Then, they press the send button in the application to send the photo to the server. If necessary, they can enter environmental information such as humidity and temperature at home or on the job site.

[1557] (Input) Pest photos, environmental information

[1558] (Output) Photo data and environmental information sent to the server

[1559] (Specific operation) A user finds a cockroach in the kitchen, takes a photo of the cockroach with their smartphone, and taps the send button in the app to send the data to the server.

[1560] Step 2:

[1561] The server receives photos of pests submitted by users, assigns them unique identifiers, records logs, and stores the photo data and environmental information in a database.The server then sends the photos to a generative AI model to identify the type of pest.

[1562] (Input) Photo data and environmental information sent by the user

[1563] (Output) Data stored in the database, unique identifiers assigned, and pest species identification results

[1564] (Specific operation) The server assigns an identifier such as "User ID_Photo ID_20231001" to the photo and sends the photo data to the generative AI model for analysis.

[1565] Step 3:

[1566] The server uses a generative AI model to analyze the received photo data and identify the type of pest, storing the identification results in a database and using them to generate the next necessary extermination method.

[1567] (Input) Photo data, generative AI model

[1568] (Output) Pest identification results

[1569] (Specific operation) The generative AI model analyzes the photo, generates an identification result that says, "This is cockroach species A," and stores this in the database.

[1570] Step 4:

[1571] The server acquires environmental information provided by the user and previously saved environmental data, generates an extermination method suitable for the type of pest, and sends the generated extermination method and product information to the user.

[1572] (Input) Pest identification results, environmental information, past environmental data

[1573] (Output) The best method of eradication and its information

[1574] (Specific operation) The server generates notifications to the user based on data such as "The humidity in the home is high" and "In the case of cockroach species A, use an extermination spray."

[1575] Step 5:

[1576] The server uses the voice and text input from the user when sending a photo to analyze the user's emotional state using an emotion analysis engine, and adjusts the format and content of the information provided based on this.

[1577] (Input) User voice and text messages

[1578] (Output) Adjusting response content

[1579] (Specific operation) If a user sends a comment such as "It was really scary," the emotion analysis engine will determine that the emotional state is at a high stress level.

[1580] Step 6:

[1581] The server generates a response tailored to the user's emotional state and notifies the user, providing reassuring information such as detailed countermeasure procedures and options for consulting with an expert.

[1582] (Input) Emotion analysis results

[1583] (Output) A tailored response that is communicated to the user

[1584] (Specific actions) The server sends a detailed guide such as "How to get rid of cockroach species A: 1) Get the spray. 2) Spray it directly on the cockroaches. 3) Clean the kitchen." It also provides additional support information such as "If you would like to consult with an expert, please contact us here."

[1585] (Application example 2)

[1586] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1587] With conventional pest control systems, users often experienced a great deal of stress between discovering pests and finding the appropriate extermination method. Furthermore, the information provided about pest species and extermination methods was inconsistent, leaving room for improvement in the user experience. Furthermore, the system was unable to respond in a way that took the user's emotional state into account, resulting in low user satisfaction. To solve these problems, a system that could analyze the user's emotional state and respond flexibly based on that was needed.

[1588] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1589] In this invention, the server includes: [means for a user to take a photo of a pest using a terminal and send it to the server; [means for the server to analyze the received photo of the pest using a generative AI model and identify the type of pest; and [means for the server to generate an extermination method using the generative AI model based on the identified type of pest and related environmental information and send it to the user.] This allows the user to quickly and accurately identify the type of pest and provide the optimal extermination method, enabling effective pest extermination.

[1590] In addition, the server uses an emotion engine to analyze the user's emotional state from voice and text, and adjusts the extermination method provided, thereby reducing stress and anxiety when the user discovers a pest, improving the user experience.

[1591] A "terminal" is a device used by a user, such as a mobile information terminal, a smartphone, or a tablet.

[1592] A "server" is a central device that receives, analyzes, stores, and provides results on a computer network for image data and environmental information.

[1593] A "generative AI model" is an algorithm that uses artificial intelligence technologies such as deep learning to analyze image data and generate information on the type of pest and appropriate extermination methods.

[1594] An "emotion engine" is a system that analyzes a user's emotional state from their voice and text and provides appropriate responses based on that.

[1595] A "fixed camera" is a device such as a surveillance camera that is installed in a specific location such as a building or restaurant and provides video footage in real time.

[1596] "Environmental information" refers to information such as the temperature, humidity, lighting conditions, and food management status of the area where pests live.

[1597] "Extermination method" is a general term for procedures, chemicals, traps, etc. used to efficiently exterminate specific pests.

[1598] "Emotional state" refers to a user's psychological state, such as stress, anxiety, anger, or joy.

[1599] "User experience" refers to the overall satisfaction and emotional response a user feels when using a system or service.

[1600] This invention is a system for identifying pests found by users and supporting appropriate extermination methods. To improve the user experience, the system features an emotion engine that provides responses tailored to the user's emotional state. The system is composed of a combination of hardware and software, including a user terminal, a server, a generative AI model, an emotion engine, and a fixed camera.

[1601] 1. Send photos of pests

[1602] First, the user launches the dedicated application on their device (such as a smartphone or tablet) and takes a photo of the pest they find. After taking the photo, the user presses the send button in the application to send the photo to the server. If necessary, the user can also enter environmental information such as the temperature and humidity of their home or work site.

[1603] 2. Image Analysis

[1604] The server receives photos of pests sent by users. It then uses a generative AI model to analyze the received images and identify the type of pest. This generative AI model is built on deep learning technology and can identify the characteristics of pests with high accuracy. The analyzed information is stored in a database.

[1605] 3. Generating removal methods

[1606] The server generates the optimal extermination method based on the identified pest species and environmental information provided by the user. Using this generative AI model, an efficient and effective extermination method is automatically selected. The selected extermination method and related product information are then notified to the user.

[1607] 4. Emotion recognition

[1608] The emotion engine analyzes the voice and text information when users send photos to determine their emotional state. Developed based on an empirical psychological model, the engine accurately recognizes emotions such as stress, anxiety, and anger.

[1609] 5. Response and Notification

[1610] The application adapts the type of information it provides depending on the user's emotional state: for example, if the user is feeling highly stressed, it will provide more detailed removal instructions and step-by-step guides, while if the user is feeling anxious, it will offer additional support and options to consult with a specialist.

[1611] Specific examples

[1612] If a store staff member spots a rat in the backroom, they can take a photo of it using a dedicated smartphone app and send it to the server. The server will immediately identify the type of rat and suggest effective extermination methods and where to set traps. If a staff member is feeling anxious or stressed, the emotion engine will detect this and provide more detailed instructions or the option to consult with a specialist.

[1613] Prompt Sentence Examples

[1614] "Images taken with the app are sent to a server, and a generative AI model is used to identify the type of pest. The app then generates the optimal extermination method based on environmental information, and uses an emotion engine to provide a response that reflects the user's emotional state."

[1615] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1616] Step 1:

[1617] The user uses the device to take a photo of the pest.

[1618] Input: A user launches a smartphone app and takes a photo of a pest.

[1619] How it works: Users can take photos of pests in a dedicated application and manually enter the necessary environmental information (temperature, humidity, etc.).

[1620] Output: The captured photos of the pests and environmental information are saved on the device.

[1621] Step 2:

[1622] The device sends photos and environmental information to the server.

[1623] Input: Pest photos and environmental information stored on your device.

[1624] How it works: The user presses the send button in the application, sending the photo and environmental information to the server.

[1625] Output: The server receives the pest photos and environmental information.

[1626] Step 3:

[1627] The image received by the server is analyzed using a generative AI model to identify the type of pest.

[1628] Input: A photo of a pest received by the server.

[1629] How it works: The server inputs image data into a generative AI model, which uses deep learning to analyze and identify the type of pest. The identified data is then stored in a database.

[1630] Output: Pest species information.

[1631] Step 4:

[1632] The server generates the optimal extermination method based on the identified pest type and environmental information.

[1633] Input: Identified pest species information and environmental information.

[1634] How it works: The server uses a generative AI model to combine pest species and environmental information to generate the optimal extermination method.

[1635] Output: Information on the best removal methods and related products.

[1636] Step 5:

[1637] The server analyzes the user's emotional state from voice and text using an emotion engine.

[1638] Input: Voice and text data entered by the user when sending a photo.

[1639] How it works: The emotion engine allows the server to analyze the user's emotional state and identify emotions such as stress, anxiety, and anger.

[1640] Output: User's emotional state information.

[1641] Step 6:

[1642] The server adjusts its offerings of extermination methods based on the emotional state.

[1643] Input: optimal removal method and user emotional state information.

[1644] How it works: The server adapts the type and level of detail of the information it provides based on the user's emotional state, for example by adding detailed instructions or the option to consult with an expert.

[1645] Output: Adjusted removal instructions and supporting information.

[1646] Step 7:

[1647] The server sends coordinated removal instructions and support information to the user.

[1648] Input: Coordinated removal instructions and support information.

[1649] Operation: The server sends the generated removal instructions and support information to the user's device.

[1650] Output: Removal instructions and support information are made available to the user on their device.

[1651] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1652] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1653] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1654] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1655] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1656] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1657] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1658] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1659] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1660] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1661] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1662] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1663] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1664] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1665] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1666] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1667] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1668] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1669] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1670] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1671] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1672] The following is further disclosed regarding the above embodiment.

[1673] (Claim 1)

[1674] [Means for a user to take a photo of a pest using a terminal and send it to a server;

[1675] [Means for analyzing the photos of pests received by the server using a generative AI model and identifying the type of pest;

[1676] [Means for the server to generate an extermination method using a generative AI model based on the identified pest type and related environmental information and send it to the user;

[1677] A system including:

[1678] (Claim 2)

[1679] [The server acquires video data from fixed cameras installed in buildings and restaurants, and detects the occurrence of pests in real time.

[1680] [Means for the server to notify the administrator of immediate eradication measures based on the detected pest outbreak;

[1681] The system of claim 1 further comprising:

[1682] (Claim 3)

[1683] [Means for the server to retrieve previously stored environmental data and combine it with current environmental information to optimize pest control methods;

[1684] The system of claim 1 further comprising:

[1685] "Example 1"

[1686] (Claim 1)

[1687] [Means for a user to take a photo of a pest using a terminal and send it to a server;

[1688] [Means for preprocessing the photos of pests received by the server and analyzing them with a generative AI model to identify the type of pest;

[1689] [The server generates an optimal extermination method using a generation AI model based on the identified pest type and related current environmental information and past data, and transmits the method to the user;

[1690] A system including:

[1691] (Claim 2)

[1692] [A server acquires video data from fixed cameras installed in buildings and facilities, and detects pest occurrence in real time,

[1693] [Means for the server to notify the administrator of immediate eradication measures based on the detected pest outbreak;

[1694] The system of claim 1 further comprising:

[1695] (Claim 3)

[1696] [The server retrieves previously stored environmental data and combines it with current environmental information to optimize pest control methods]

[1697] Means and

[1698] The system of claim 1 further comprising:

[1699] "Application Example 1"

[1700] (Claim 1)

[1701] [Means for a user to take a photo of a pest using a terminal and transmit the photo to an information device;

[1702] [Means for analyzing the photos of pests received by the information device using a generative AI model and identifying the type of pest;

[1703] [Means for generating an extermination method using a generative AI model based on the identified pest type and related environmental information by the information device and transmitting the method to the user;

[1704] [A functional means to purchase appropriate extermination supplies through the link provided on your device;

[1705] ...

[1706] A system including:

[1707] (Claim 2)

[1708] [Means for an information device to acquire video data from fixed cameras installed in buildings and stores and detect the occurrence of pests in real time;

[1709] The system of claim 1, wherein the information device notifies the administrator of immediate extermination measures based on the detected infestation of pests.

[1710] (Claim 3)

[1711] The system of claim 1, wherein the information device acquires previously stored environmental data and combines it with current environmental information to optimize pest control methods.

[1712] "Example 2: Combining Emotion Engines"

[1713] (Claim 1)

[1714] [Means for a user to take a photo of a pest using an information terminal and transmit the photo to a counting device;

[1715] [Means for analyzing the photos of pests received by the aggregation device using a generative AI model and identifying the type of pest;

[1716] [Means for generating an extermination method using a generative AI model based on the type of pest identified by the aggregation device and related environmental information, and sending the method to the user;

[1717] [Means for the aggregation device to analyze the user's emotions from the voice or text and adjust the response content;

[1718] A system including:

[1719] (Claim 2)

[1720] [The system of claim 1, wherein the aggregation device acquires image data from a fixed camera installed in a facility or store and detects the occurrence of pests in real time.

[1721] (Claim 3)

[1722] [The system of claim 1, wherein the aggregation device acquires previously stored environmental data and combines it with current environmental information to optimize pest control methods.

[1723] "Application example 2 when combining emotion engines"

[1724] (Claim 1)

[1725] [Means for a user to take a photo of a pest using a terminal and send it to a server;

[1726] [Means for analyzing the photos of pests received by the server using a generative AI model and identifying the type of pest;

[1727] [Means for the server to generate an extermination method using a generative AI model based on the identified pest type and related environmental information and send it to the user;

[1728] [The server analyzes the user's emotional state from voice and text using an emotion engine and adjusts the content of the removal method provided;

[1729] A system including:

[1730] (Claim 2)

[1731] [The server acquires video data from fixed cameras installed in buildings and restaurants, and detects the occurrence of pests in real time.

[1732] [Means for the server to notify the administrator of immediate extermination measures based on the detected pest outbreak and to respond according to the emotional state using an emotion engine;

[1733] The system of claim 1 further comprising:

[1734] (Claim 3)

[1735] [Means for the server to acquire environmental data stored in the past, combine it with current environmental information to optimize pest control methods, and provide support content according to the user's emotional state using an emotion engine;

[1736] The system of claim 1 further comprising: [Explanation of symbols]

[1737] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for a user to take a photo of a pest using a terminal and transmit the photo to a server; A means for analyzing the photos of pests received by the server using a generative AI model to identify the type of pest; A means for the server to generate an extermination method using a generative AI model based on the identified pest type and related environmental information and transmit the method to the user; A system including:

2. The server acquires video data from fixed cameras installed in buildings and restaurants, and detects the occurrence of pests in real time. A means for the server to notify the administrator of immediate extermination measures based on the detected pest outbreak; The system of claim 1 further comprising:

3. A means for the server to retrieve previously stored environmental data and combine it with current environmental information to optimize pest control methods; The system of claim 1 further comprising:

Images