system

Abstract

Provide a system. The present invention provides a method for collecting ultrasound, microRNA, and volatile substances from plants; means for pre-processing the collected data; a means for using generative AI to analyze the pre-processed data and estimate the plant condition; means for transmitting information about plant growth to a user based on the analysis results; The system includes a means for receiving user feedback to improve the accuracy of the generative AI.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Signals emitted by plants, such as ultrasound, microRNA, and volatile substances, are important sources of information that indicate the plant's condition and needs. However, until now, it has been difficult for humans to directly understand these signals and implement appropriate responses. As a result, it has been difficult to determine the optimal flowering and pollination times, fruit thinning, and harvesting times, and there is also the possibility of inadequate use of pesticides and avoidance of insect damage. This has been a problem that hinders improvements in agricultural productivity and food self-sufficiency. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for collecting ultrasound, microRNA, and volatile substances from plants, a means for preprocessing the collected data, a means for analyzing the preprocessed data and using a generating AI to estimate the plant's condition, a means for sending information about plant growth to a user based on the analysis results, and a means for receiving feedback from the user and improving the accuracy of the generating AI. This system enables more accurate determination of flowering time, pollination timing, fruit thinning, and harvesting timing, making it easier to use pesticides appropriately and avoid insect damage, and is expected to improve agricultural productivity and food self-sufficiency.

[0006] "Ultrasound" is a high-frequency sound wave that exceeds the frequency of normal sound (20 kHz or less), and is a type of signal emitted by plants and other living organisms to indicate a specific state.

[0007] "MicroRNAs" are small non-coding RNA molecules that regulate gene expression in plants and other organisms and may indicate specific physiological states or stress responses.

[0008] "Volatile substances" is a general term for chemical substances that easily evaporate in the air and are released by plants when they respond to or communicate with the external environment.

[0009] "Data preprocessing" is the process of detecting missing or outliers in collected raw data and complementing or removing them as necessary; it is a preparatory step to improve the accuracy of analysis.

[0010] "Generative AI" is a type of artificial intelligence technology used to analyze collected and preprocessed data to estimate the condition and needs of plants.

[0011] "Estimation" refers to identifying the current and future state of a plant and the measures required based on collected and analyzed data.

[0012] "Feedback" refers to reporting the results of actions taken by the user and any observed changes to the server, providing information that will help improve the accuracy of the analysis.

[0013] The "flowering period" refers to the specific period during which a plant blooms, and is an important timing in which proper management can have a significant impact on harvest yield and quality.

[0014] "Pollination timing" refers to the specific period during which a plant can receive pollen and become fertilized, and whether or not this happens has a significant impact on the success of the harvest.

[0015] "Fruit thinning" is an agricultural technique in which fruits and vegetables are selected and removed during development to increase yield and quality.

[0016] "Appropriate use" refers to the method of using pesticides, fertilizers, and other materials effectively and safely, and means promoting healthy crop growth by avoiding excessive or insufficient use.

[0017] "Insect damage" refers to the phenomenon in which agricultural crops are damaged by insects, which leads to reduced yields and deterioration of quality, so appropriate measures are necessary. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The system of the present invention analyzes signals emitted by plants, such as ultrasonic waves, microRNA, and volatile substances, estimates the plant's condition, and based on that estimates, suggests appropriate actions to the user. Specific processing of the program for implementing this system is explained below in natural language.

[0040] server

[0041] 1. Data Collection:

[0042] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances from plants. These signals are important data that indicate the timing of plant flowering and pollination, as well as damage caused by pests and diseases.

[0043] 2. Data Preprocessing:

[0044] The server detects missing or outliers in the collected data and performs imputation or removal as necessary. Preprocessing improves the reliability of the data and the accuracy of analysis.

[0045] 3. Data Analysis:

[0046] The server inputs the preprocessed data into the generative AI, which estimates the plant's condition and needs. The generative AI takes into account past data and environmental conditions to predict the optimal growing conditions for the plant.

[0047] 4. Information Transmission:

[0048] Based on the analysis results, the server notifies the user of specific information such as flowering time, pollination timing, and fruit thinning timing, allowing the user to carry out tasks at the appropriate time.

[0049] 5. Feedback received:

[0050] The server receives feedback from users to improve the accuracy of the generative AI, including the results of their work and observations.

[0051] Terminal (user's device)

[0052] 1. Receiving information:

[0053] The device receives analysis results and action plans sent from the server in real time, allowing users to understand the condition and needs of their plants at any given time.

[0054] 2. User Interface:

[0055] The device then displays the received information in an easy-to-understand format. For example, if the flowering period is approaching, it will display the specific pollination timing and procedure, and encourage the user to take appropriate action.

[0056] User

[0057] 1. Information verification:

[0058] Based on the information provided by the device, users can check specific actions for plant growth, allowing them to take necessary action before it is too late.

[0059] 2. Take action:

[0060] Based on the received information, the user can take appropriate action on-site, such as manually pollinating the crops based on the notified pollination timing.

[0061] 3. Providing Feedback:

[0062] The user inputs the results of their execution and any changes they observe into their device and sends them to the server. This feedback information is used to improve the analysis accuracy of the generative AI.

[0063] Specific examples

[0064] For example, the server receives a signal from a sensor indicating that "flowering is approaching," analyzes the signal, and notifies the user that "the time for pollination is approaching." The user checks the notification and pollinates at the appropriate time. The results of this work are reported to the server as feedback, contributing to improving the accuracy of the next prediction.

[0065] Furthermore, if the server detects signals of damage caused by pests, it will send an alert to the device saying, "Insect damage has occurred. Urgent action is required." Based on this, the user can quickly take measures such as spraying pesticides. The results are then sent to the server as feedback.

[0066] The processing flow will be explained below.

[0067] Server Processing Steps

[0068] Step 1: Data collection

[0069] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances from plants, which reflect the physiological state and stress status of plants.

[0070] Step 2: Data Preprocessing

[0071] The server preprocesses the collected data, specifically detecting missing and outlier values ​​and imputing or removing them as necessary, a crucial step to ensure the reliability and accuracy of the data.

[0072] Step 3: Data analysis

[0073] The pre-processed data is fed into generative AI to estimate the plant's condition and needs, including predicting flowering time and risk of pest and disease damage.

[0074] Step 4: Send information

[0075] Based on the analysis results, the server creates a specific action plan and notifies the user, including specific instructions and recommendations for plant growth.

[0076] Step 5: Receiving feedback

[0077] The server receives feedback from users, which includes work results and observations, and is important for improving the analysis accuracy of the generative AI.

[0078] Terminal (user device) processing steps

[0079] Step 1: Receiving information

[0080] The terminal receives the prediction results and action plans sent from the server in real time.

[0081] Step 2: Display information

[0082] The device then displays the received information in an easy-to-understand format for the user. For example, if the flowering period is approaching, it will display specific pollination timing and procedures.

[0083] Step 3: Operation Guide

[0084] If necessary, provide the user with additional information and instructions to assist with specific operations.

[0085] User processing steps

[0086] Step 1: Verify the information

[0087] The user checks the information provided by the terminal and understands specific actions to take regarding plant growth.

[0088] Step 2: Take action

[0089] The user then takes appropriate action based on instructions from the device, such as manually pollinating the plants based on the notified pollination timing.

[0090] Step 3: Provide feedback

[0091] The user inputs the results of their execution and any changes they observe into their device and sends them to the server. This feedback information is used to improve the analytical accuracy of the generative AI.

[0092] Through the above steps, the server, terminal, and user can work together to efficiently and effectively manage plant growth.

[0093] Example 1

[0094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] Accurately understanding the condition of plants and taking appropriate action in a timely manner is extremely important for efficient plant cultivation and pest prevention. However, current technology does not yet fully establish a system for monitoring plant conditions in real time and taking appropriate measures. In particular, there is room for improvement in the accuracy of data collection and analysis, the timing of user notifications, and how feedback is utilized.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0097] In this invention, the server includes means for collecting ultrasound, microRNA, and volatile substances from plants, means for preprocessing the collected data, means for inputting the preprocessed data into a generative AI model to estimate the plant's condition, means for transmitting information about plant growth to a user's device based on the analysis results, means for receiving user feedback and improving the accuracy of the generative AI model, means for visually displaying information through a user interface and notifying the user in real time, and means for retraining the generative AI model based on the feedback. This makes it possible to monitor and analyze the plant's condition in real time and provide the user with necessary action information at the appropriate time.

[0098] "Means for collecting ultrasound, microRNA, and volatile substances from plants" is a general term for devices and sensors that detect ultrasound, microRNA, and volatile substances emitted by plants and obtain this information as data.

[0099] "Means for preprocessing collected data" refers to processes and algorithms for detecting, complementing, or removing missing or outlier values ​​in collected signal data from plants, thereby improving the reliability of the data and the accuracy of the analysis.

[0100] "Means of inputting preprocessed data into a generative AI model to estimate plant condition" refers to a system or algorithm that inputs data that has undergone preprocessing into a machine learning model and uses that data to estimate the current condition and future needs of a plant.

[0101] "Means for sending information about plant growth to the user's device based on the analysis results" refers to a notification system or communication protocol that notifies the user of the results of the generative AI model's analysis, allowing the user to take the necessary action at the appropriate time.

[0102] "Means for receiving user feedback and improving the accuracy of generative AI models" refers to systems and processes for collecting work results and observational information from users and using it as training data for generative AI models to improve the accuracy of future predictions and analyses.

[0103] "Means for visually displaying information through a user interface and providing notifications in real time" refers to an interface for displaying analysis results and action plans in a format that is easily understandable to users, and a system or software for notifying users of necessary information in real time.

[0104] "Means for retraining a generative AI model based on feedback" refers to a process or algorithm that uses feedback information obtained from users to retrain a generative AI model and improve the model's accuracy and reliability.

[0105] The system of the present invention includes a series of means for analyzing signals emitted by plants in real time and providing specific cultivation actions to the user based on the analysis results. The hardware and software used in this system are as follows.

[0106] Hardware

[0107] Ultrasonic sensor: A device that detects ultrasonic waves emitted by plants and collects that data.

[0108] MicroRNA detection device: A device that detects microRNAs in plants and collects the data.

[0109] Volatile substance detector: A sensor that detects volatile organic compounds emitted by plants.

[0110] User's smart device: The device that receives notifications and displays the user interface (e.g., smartphone, tablet).

[0111] software

[0112] Generative AI model: A machine learning model for predicting and analyzing plant conditions. Built using frameworks such as TENSORFLOW (registered trademark).

[0113] Data preprocessing program: A script using a programming language such as Python to clean the collected data and impute missing values.

[0114] Notification system: A system that uses services such as Firebase to send notifications to users' devices in real time.

[0115] Server Operation

[0116] The server uses various sensors and specialized equipment to collect ultrasonic, microRNA, and volatile chemical signals from plants in real time. The collected data is then stored in a database, and missing and outliers are detected and imputed or removed. The preprocessed data is then fed into a generative AI model, which takes into account past data and current environmental conditions to predict the plant's condition and necessary actions.

[0117] Device behavior

[0118] The device receives the analysis results sent from the server in real time and displays them visually through the user interface. For example, if the flowering period is approaching, the device will display the specific pollination timing and procedure, prompting the user to take appropriate action. The notification system uses Firebase and other services to quickly deliver the analysis results to the user's device.

[0119] User Actions

[0120] Based on the information provided by the device, the user plans and executes specific actions related to plant growth. For example, if the user receives a notification that "pollination time is approaching," the user prepares for pollination and carries it out at the appropriate time. The results of the actual work and any observed changes in the plant are entered into the device, and the feedback is sent to the server. This feedback is used to retrain the generative AI model, contributing to improving the accuracy of analysis in future generations.

[0121] Specific examples

[0122] For example, the server receives a signal from a sensor that "flowering is approaching," analyzes the signal, and notifies the user that "the time for pollination is approaching." The user checks this notification and performs pollination at the appropriate time. The results of this work are reported to the server as feedback, contributing to improved prediction accuracy for the next time. Furthermore, if the server detects a signal of damage caused by pests, it will send an alert to the device saying, "Insect damage has occurred. Urgent action is required." Based on this, the user can quickly take measures such as spraying pesticides, and send the results of this to the server as feedback.

[0123] Prompt Sentence Examples

[0124] You are designing a system that analyzes signals emitted by plants and provides information about their status to the user. These signals include ultrasound, microRNA, and volatile substances. Please specify the hardware and software used in this system and explain the specific data processing and data calculation methods.

[0125] These processes enable real-time monitoring and analysis of plant conditions, providing users with timely information on the actions they need to take.

[0126] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0127] Server Processing Steps

[0128] Step 1: Data collection

[0129] Input: Signal data from ultrasonic sensors, microRNA detectors, and volatile substance detectors.

[0130] How it works: The server receives real-time signal data about the plant's condition from these various sensors. The sensors record data every second and send it to the server. The server stores the received data in a database.

[0131] Output: The raw data collected.

[0132] Step 2: Data Preprocessing

[0133] Input: Raw data collected.

[0134] Specific operation: The server detects missing values ​​and outliers in the received data and processes them (complements or removes them). For complementation, it uses the most recent valid data, and for outliers, it corrects or removes values ​​that are out of range.

[0135] Output: Preprocessed clean data.

[0136] Step 3: Data analysis

[0137] Input: Preprocessed clean data.

[0138] Specific operation: The server inputs the preprocessed data into the generative AI model, which then predicts the plant's condition and growth based on past data and current environmental conditions.

[0139] Output: Analysis results (e.g., flowering time, pollination timing, risk of pests and diseases, etc.).

[0140] Step 4: Send information

[0141] Input: Analysis results.

[0142] Specific operation: Based on the analysis results of the generation AI, the server notifies the user of necessary action information (e.g., flowering time, pollination timing). This information is sent to the user's device via a notification service such as Firebase.

[0143] Output: Notifications sent to the user's device.

[0144] Step 5: Receiving feedback

[0145] Input: Feedback data from users.

[0146] Specific operation: Receives work results and observation data entered by the user into the device, stores this feedback data in a database, and creates a new learning set to retrain the generative AI model.

[0147] Output: Feedback data, improved generative AI model.

[0148] Terminal processing steps

[0149] Step 1: Receiving information

[0150] Input: Analysis results and action plans sent from the server.

[0151] What it does: The device maintains a connection with the server and periodically checks for new notifications. If any notifications are available, they are immediately downloaded and displayed to the user.

[0152] Output: Notification data waiting to be displayed.

[0153] Step 2: User Interface Display

[0154] Input: Notification data.

[0155] Specific operation: The device analyzes the received notification and displays it on the user interface. It provides instructions in visually appealing graphs and text, making it easy for the user to understand.

[0156] Output: A screen display for the user to see.

[0157] User processing steps

[0158] Step 1: Verify the information

[0159] Input: Notifications from your device.

[0160] Specific behavior: The user taps the notification on their device to view more information and plan appropriate actions based on the information displayed.

[0161] Output: A confirmed action plan.

[0162] Step 2: Take action

[0163] Input: Confirmed action plan.

[0164] Specific actions: The user performs the necessary work on site based on the notification. For example, if pollination is required, prepare pollination tools and perform pollination at the appropriate time.

[0165] Output: The training work carried out.

[0166] Step 3: Provide feedback

[0167] Input: Development activities performed and observation data.

[0168] Specific operation: The user inputs the results of their work and observed changes into the terminal and sends them to the server, which improves the analysis accuracy of the entire system.

[0169] Output: Feedback data sent to the server.

[0170] (Application example 1)

[0171] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0172] Conventional plant growth management systems are required to not only grasp the state of plants, but also to formulate and execute appropriate action plans, but there are many challenges to achieving this. Specifically, they require accurate data collection and analysis to accurately analyze signals from plants and provide optimal growth conditions, as well as the autonomy to take appropriate actions based on that data. Conventional methods lack sufficient analytical accuracy and automation of actions, making effective plant growth management difficult.

[0173] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0174] In this invention, the server includes means for collecting ultrasound, microRNA, and volatile substances from plants, means for preprocessing the collected data, means for analyzing the preprocessed data and using a generating AI to estimate the state of the plant, means for sending information about plant growth to a user based on the analysis results, means for receiving feedback from the user and improving the accuracy of the generating AI, and means for controlling a factory robot that executes an action plan required for plant growth based on the analysis results. This realizes a system that can perform everything from data analysis to action execution in an integrated manner, enabling optimal plant growth management.

[0175] "Ultrasound" refers to sound waves with a frequency of 20 kilohertz or higher, which contain unique signals emitted by plants.

[0176] "MicroRNAs" are small RNA molecules that regulate gene expression in plants and are important biomarkers that reflect the health and stress status of plants.

[0177] "Volatiles" are volatile compounds emitted by plants that indicate the plant's condition or environmental stress.

[0178] "Generative AI" is an artificial intelligence model that estimates the condition of plants based on collected data and suggests optimal growth conditions and action plans.

[0179] An "action plan" is a plan that includes specific actions and procedures required for the growth and protection of plants, based on the analysis results of the generative AI.

[0180] A "factory robot" is a mechanical device that operates autonomously or semi-autonomously within a factory and executes action plans related to plant growth management.

[0181] "Data preprocessing" is the process of detecting missing or outliers in collected plant data and imputing or removing them appropriately.

[0182] "Feedback" is information provided by the user about execution results and observed changes, which is used to improve the accuracy of the generative AI.

[0183] This invention is a system for understanding the state of plants in real time and implementing appropriate cultivation actions based on that information. The main components of the system are a server, terminals, and factory robots.

[0184] server

[0185] The server includes the following means:

[0186] 1. Data collection: The server collects data about the plants in real time from various sensors, such as ultrasonic sensors, microRNA analyzers, and volatile substance detectors.

[0187] 2. Data preprocessing: Detect missing or outliers in the collected data, and impute or remove them as necessary to improve the reliability of the data and the accuracy of the analysis.

[0188] 3. Generative AI analysis: The preprocessed data is input into the generative AI, which takes into account past data and environmental conditions to estimate the plant's condition and necessary growing conditions.

[0189] 4. Information transmission: Based on the analysis results, specific information such as flowering time, pollination timing, and fruit thinning timing is notified to the user. In addition, an action plan generated based on the analysis results is transmitted to the factory robot.

[0190] 5. Feedback reception: Receive feedback from the user on the actions taken to improve the analytical accuracy of the generative AI.

[0191] Terminal (user's device)

[0192] The terminal includes the following means:

[0193] 1. Information reception: Receive analysis results and action plans sent from the server in real time.

[0194] 2. User interface: The received information is displayed to the user in an easy-to-understand format, showing the notified pollination timing and procedures.

[0195] Factory robots

[0196] The factory robot includes the following means:

[0197] 1. Action plan execution: According to the action plan sent from the server, appropriate cultivation tasks are carried out automatically or semi-automatically, such as watering, fertilizing, and pest control.

[0198] Overall processing flow

[0199] The server first collects data about the plants from various sensors. The collected data is preprocessed, and missing or outliers are filled in or removed. The preprocessed data is analyzed by the generative AI, which estimates the plant's condition and the conditions necessary for its growth. The analysis results are notified to the user's device, and an appropriate action plan is sent to the factory robot. The factory robot carries out the growth work based on this action plan. The user provides feedback on actual work results and observations, which are sent to the server and used to improve the accuracy of the generative AI.

[0200] Specific examples

[0201] For example, the server collects ultrasonic data, microRNA data, and volatile substance data from a plant and analyzes it to determine that the plant is nearing flowering. As a result, the user's device is notified that "the time for pollination is approaching." At the same time, an action plan for carrying out "pollination" is sent to the factory robot. The user checks the notification, and the robot carries out pollination at the appropriate time. The results are then reported to the server as feedback.

[0202] Example prompts for generative AI models

[0203] Enter your plant's ultrasound, microRNA, and volatile data, and we'll estimate your next steps and suggest an appropriate course of action.

[0204] Example data:

[0205] Ultrasonic: 10kHz

[0206] MicroRNA: [RNA_sequence_A, RNA_sequence_B]

[0207] Volatile substances: [Compound A, Compound B]

[0208] In this way, the entire system works together to achieve optimal plant growth.

[0209] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0210] Step 1:

[0211] Data collection

[0212] The server collects real-time plant data from sensors such as ultrasonic sensors, microRNA analyzers, and volatile substance detectors. In this step, raw data from each sensor is input and output in its raw format.

[0213] Step 2:

[0214] Data Preprocessing

[0215] The server detects missing or outliers in the collected data and imputes or removes them as necessary. Here, the raw data collected in the previous step is used as input, and reliable data is output with outliers replaced with default values ​​such as "default_value." Specific operations include checking the consistency of the data and filtering outliers.

[0216] Step 3:

[0217] Generative AI analysis

[0218] The server inputs the preprocessed data and uses generative AI to estimate the plant's condition. The output is an analysis result that clarifies the plant's current condition and necessary growing conditions. Input data to the generative AI model includes values ​​of ultrasound, microRNA, and volatile substances.

[0219] Step 4:

[0220] Information transmission

[0221] Based on the analysis results of the generated AI, the server sends information about plant growth to the user's device. It also sends an action plan to the factory robot. The analysis results are used as input, and the output is instructions and action plans received by the device and robot. The specific operation involves data communication via a network.

[0222] Step 5:

[0223] Information reception (terminal)

[0224] The terminal receives analysis results and action plans sent from the server in real time. The input is the analysis results and action plans from the server, and the output is display information converted into a format that can be viewed by the user. Specific operations include analyzing the received data and displaying it on the user interface.

[0225] Step 6:

[0226] Action plan execution (factory robot)

[0227] Factory robots automatically or semi-automatically perform appropriate cultivation tasks based on an action plan sent from a server. The action plan from the server is given as input, and the output is the actual physical work, such as watering, fertilizing, and pest control. Specific operations include robot motion control and the use of sensors.

[0228] Step 7:

[0229] Feedback provided by users

[0230] The user inputs feedback about the tasks performed and the changes observed into the terminal, which then sends it to the server. The input is feedback data from the user, and the output is data used to update the generative AI. Specific operations include data input via the user interface and feedback transmission via the network.

[0231] Step 8:

[0232] Feedback reception (server)

[0233] The server receives feedback from users and uses it to improve the accuracy of the generative AI. The input is the user feedback data, and the output is an updated, more accurate generative AI model. Specific operations include analyzing the feedback data and training the generative AI model.

[0234] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0235] The present invention is a system that analyzes signals emitted by plants, such as ultrasound, microRNA, and volatile substances, to estimate the plant's condition and notify the user of appropriate actions based on the results. This system incorporates an emotion engine that recognizes the user's emotions, and also has the function of adjusting the content and timing of notifications based on the user's emotions and making suggestions to reduce the user's stress level. Specific processing of the program is described below.

[0236] server

[0237] 1. Data Collection:

[0238] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances from plants, which reflect the physiological state and stress status of plants.

[0239] 2. Data Preprocessing:

[0240] The server preprocesses the collected data, specifically detecting missing and outlier values ​​and imputing or removing them as necessary, which is an important step to ensure the reliability and accuracy of the data.

[0241] 3. Data Analysis:

[0242] The pre-processed data is fed into generative AI to estimate the plant's condition and needs, including predicting flowering time and risk of pest and disease damage.

[0243] 4. Emotion analysis:

[0244] The server uses an emotion engine to analyze the user's emotional data sent from the device, thereby understanding the user's current emotional state and stress level.

[0245] 5. Information Transmission:

[0246] Based on the analysis results, the server creates a specific action plan, adjusting the content and timing of notifications according to the user's emotional state to notify the user in the most optimal way.

[0247] 6. Feedback received:

[0248] The server receives feedback from users to improve the accuracy of the generative AI. The feedback information is the results of work and observations, and is useful for future analysis.

[0249] Terminal (user's device)

[0250] 1. Receiving information:

[0251] The terminal receives the prediction results and action plans sent from the server in real time.

[0252] 2. Emotional Data Collection:

[0253] The device uses a camera, microphone, and other sensors to collect information such as the user's facial expressions, voice, and heart rate, and transmits this information to a server as emotional data.

[0254] 3. Information display:

[0255] The device then displays the received information in an easy-to-understand format for the user. For example, if the flowering period is approaching, it will display specific pollination timing and procedures.

[0256] 4. Operation Guide:

[0257] If necessary, provide the user with additional information and instructions to assist with specific operations.

[0258] User

[0259] 1. Information verification:

[0260] The user checks the information provided by the terminal and understands specific actions to take regarding plant growth.

[0261] 2. Take action:

[0262] The user then takes appropriate action based on instructions from the device, such as manually pollinating the plants based on the notified pollination timing.

[0263] 3. Record your emotions:

[0264] Users record their emotions and stress levels on their devices and send them to a server, where the emotional data is used for analysis and action optimization.

[0265] 4. Providing Feedback:

[0266] The user inputs the results of their actions and any changes they observe into their device and sends them to the server. This feedback information is used to improve the analytical accuracy of the generative AI.

[0267] Specific examples

[0268] For example, if the server receives a signal from a sensor that "flowering is approaching" and analyzes it, it notifies the user that "the time for pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification will be sent as is, but if it detects that the user is in a high stress state, the message will be adjusted to include a warning such as "The important time for pollination is approaching, but please proceed with the work without overdoing it."

[0269] Furthermore, if the server detects signals of damage caused by pests, it sends an alert to the device saying, "Insect damage has occurred. Immediate action is required." If the user feels high stress in response to this alert, the emotion engine sends a message emphasizing how to deal with the situation, such as, "Insect damage has occurred. Please calmly take action by following the procedures." Based on this, the user can quickly take action, such as spraying pesticides. The results are then sent to the server as feedback.

[0270] In this way, the server, terminals, and users can work together to efficiently and effectively manage plant cultivation. In addition, the combination of the emotion engine reduces the psychological burden on users and improves work efficiency.

[0271] The processing flow will be explained below.

[0272] Server Processing Steps

[0273] Step 1: Data collection

[0274] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances, to detect the physiological and stress states of plants.

[0275] Step 2: Data Preprocessing

[0276] The server detects whether there are any missing or outliers in the collected data and performs imputation or removal as necessary, thereby improving the reliability of the data and the accuracy of the analysis.

[0277] Step 3: Data analysis

[0278] The preprocessed data is input into a generative AI that estimates the plant's condition and needs, and the generative AI predicts flowering time and pest and disease risk based on past data and environmental conditions.

[0279] Step 4: Sentiment Analysis

[0280] The server uses an emotion engine to analyze the user's emotional data received from the device, thereby understanding the user's current emotional state and stress level.

[0281] Step 5: Send information

[0282] Based on the analysis results, the server creates a specific action plan, adjusting the content and timing of notifications according to the user's emotional state to notify the user in the most optimal way.

[0283] Step 6: Receiving feedback

[0284] The server receives feedback from users, including the results of tasks performed and observations, and uses this feedback to improve the analytical accuracy of the generative AI.

[0285] Terminal processing steps

[0286] Step 1: Receiving information

[0287] The terminal receives the analysis results and action plans sent from the server in real time.

[0288] Step 2: Collecting Emotional Data

[0289] The device uses a camera, microphone, and other sensors to collect emotional data such as the user's facial expressions, voice, and heart rate, and sends this data to a server.

[0290] Step 3: Display information

[0291] The device then displays the received information in an easy-to-understand format for the user. For example, if the flowering period is approaching, it will display specific pollination timing and procedures.

[0292] User processing steps

[0293] Step 1: Verify the information

[0294] The user checks the information provided by the terminal and understands specific actions to take regarding plant growth.

[0295] Step 2: Take action

[0296] The user can then take appropriate action based on instructions from the device, such as manually pollinating the plants based on the notified pollination timing.

[0297] Step 3: Record your emotions

[0298] Users record their emotions and stress levels on their devices and send them to a server, where the emotional data is used for analysis and action optimization.

[0299] Step 4: Provide feedback

[0300] The user inputs the results of their execution and any observed changes into their device and sends them to the server. The feedback information is used to improve the analysis accuracy of the generative AI.

[0301] Specific examples

[0302] For example, if the server receives a signal from a sensor that "flowering is approaching" and analyzes it, it notifies the user that "the time for pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification will be sent as is, but if it detects that the user is in a high stress state, the message will be adjusted to include a warning such as "The important time for pollination is approaching, but please proceed with the work without overdoing it."

[0303] When the server detects a signal of pest damage, it sends an alert to the device saying, "Insect damage has occurred. Immediate action is required." If the user feels high stress in response to this alert, the emotion engine sends a message emphasizing how to deal with the situation, such as, "Insect damage has occurred, but please calmly follow the procedures and take action." Based on this, the user can quickly take action, such as spraying pesticides, and send the results of that action to the server as feedback.

[0304] In this way, the server, terminals, and users can work together to efficiently and effectively manage plant cultivation. In addition, the combination of the emotion engine reduces the psychological burden on users and improves work efficiency.

[0305] Example 2

[0306] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0307] Conventional agricultural systems have had difficulty accurately understanding the physiological state and stress levels of plants and taking appropriate measures based on that information. Furthermore, they do not provide information that takes into account the user's emotional state, making it impossible to reduce the user's psychological burden. Furthermore, continuous system optimization based on feedback is insufficient.

[0308] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting ultrasound, microRNA, and volatile substances from plants, means for preprocessing the collected data, means for analyzing the preprocessed data and using a generation AI to estimate the plant's condition, means for analyzing user emotion data, means for sending information about plant growth to the user based on the analysis results and the emotion data, and means for receiving feedback from the user and improving the accuracy of the generation AI. This makes it possible to accurately grasp the plant's condition and take appropriate measures based on that understanding, taking the user's emotional state into consideration. Furthermore, the system can be continuously optimized by reflecting user feedback.

[0309] "Ultrasound" is a type of sound wave that usually has a frequency of 20 kHz or higher and is inaudible to the human ear.

[0310] "MicroRNA" is a small RNA molecule that regulates gene expression and has the function of suppressing the production of specific proteins.

[0311] "Volatile substances" are substances that easily evaporate into gas at room temperature and serve as indicators of the physiological state and stress of plants.

[0312] "Preprocessing" refers to a series of processes that prepare collected data in a form suitable for analysis, and includes operations such as filling in and removing missing or outlier values ​​in the data.

[0313] "Generative AI" is an artificial intelligence technology used to analyze collected data and predict specific situations or conditions.

[0314] "Emotion data" is data including the user's facial expression, voice, heart rate, etc., and indicates the user's current emotional state and stress level.

[0315] "Feedback" is information that records the results of operations performed by the user and observed changes and returns them to the system.

[0316] "Information regarding plant cultivation" includes information regarding the flowering period of plants, timing of pollination, fruit thinning, harvesting time, proper use of pesticides, and avoidance of insect damage.

[0317] "Missing data" refers to values ​​that are not recorded because measurements or collection were not performed correctly.

[0318] An "outlier" is a data value that deviates from the normal range and may affect the analysis results.

[0319] The "emotional state of the user" indicates the state of emotions or moods that the user is feeling, and specifically includes joy, sadness, surprise, anger, fear, and the like.

[0320] The "stress level" indicates the degree of mental tension or pressure that the user feels.

[0321] The "emotion engine" is software that analyzes a user's emotional data and evaluates their current emotional state and stress level.

[0322] MODE FOR CARRYING OUT THE INVENTION

[0323] This invention is a system that analyzes signals emitted by plants, such as ultrasonic waves, microRNA, and volatile substances, to estimate the plant's condition and notify the user of appropriate actions based on the results. This system incorporates an emotion engine that recognizes the user's emotions, and also has the function of adjusting the content and timing of notifications based on the user's emotions and making suggestions to reduce the user's stress level.

[0324] Hardware and software used

[0325] server:

[0326] Hardware: High-performance server (e.g., server with Intel Xeon processor)

[0327] Software: Python, TensorFlow, Pandas, Numpy, IBM Watson® Sentiment Analysis API, Microsoft® Azure®

[0328] Terminal (user's device):

[0329] Hardware: Smartphone (ANDROID (registered trademark) or iOS), wearable device (smartwatch with heart rate monitor)

[0330] Software: Dedicated application (Android / iOS app), Bluetooth module

[0331] Data collection

[0332] The server collects real-time signal data of ultrasonic waves, microRNAs, and volatile substances from multiple sensors (e.g., ultrasonic sensors, RNA sequencers, and gas chromatographs) in plants. These signal data reflect the physiological state and stress status of plants.

[0333] Data Preprocessing

[0334] The server preprocesses the collected data using the Python Pandas library to detect missing and outliers, and impute or remove them as needed. Data smoothing and normalization are also performed at this stage.

[0335] Data analysis

[0336] The preprocessed data is then fed into a generative AI model, a deep learning model built using TensorFlow, which analyzes the plant's condition, predicting flowering time and assessing pest and disease risk, and creating a specific action plan.

[0337] Emotion analysis

[0338] The server analyzes the user's emotional data sent from the device using IBM Watson's emotion analysis API, which allows it to understand the user's current emotional state and stress level. Emotional data is collected from the device's camera, microphone, and wearable device.

[0339] Information transmission

[0340] Based on the analysis results and emotional data, the server notifies the user of the appropriate action: for users in a low-stress state, it sends a simple notification such as "The time for pollination is approaching," and for users in a high-stress state, it sends a notification with a warning such as "Please proceed with the pollination work without overdoing it."

[0341] Receive feedback

[0342] The server receives feedback from users, who send the results of their work and observed changes as feedback to the server through a dedicated application. This feedback information is recorded in a database and used to improve the accuracy of the generative AI model.

[0343] Specific examples

[0344] For example, if a plant is about to flower, the server analyzes the data from the sensor and notifies the user that "pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification is sent as is, but if the user is in a high stress state, an adjusted message is sent saying, "The important pollination time is approaching, but please proceed with your work without overdoing it."

[0345] When signals of pest damage are detected, the server sends an alert to the device saying, "Insect damage has occurred. Immediate action is required." If the user feels high stress in response to this alert, the emotion engine sends a message emphasizing countermeasures, such as, "Insect damage has occurred, but please calmly follow the procedures and take action." Based on this, the user can quickly take measures such as spraying pesticides.

[0346] In this way, the server, terminals, and users can work together to efficiently and effectively manage plant cultivation. Furthermore, by utilizing the emotion engine, the psychological burden on users can be reduced and work efficiency can be improved.

[0347] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0348] Program processing flow

[0349] Step 1: Data collection

[0350] How it works: The server collects data in real time from multiple sensors (ultrasonic sensors, RNA sequencers, and gas chromatographs) attached to plants, including the sounds of plant growth (ultrasonic waves), microRNAs (gene expression status), and volatile substances (vaporized chemical components).

[0351] Input: Signal data from the sensor.

[0352] Output: Raw, unprocessed data.

[0353] Step 2: Data Preprocessing

[0354] Specific operation: The server preprocesses the collected raw data. At this stage, it creates a data frame using Python's Pandas library, detects and imputes missing values ​​and outliers, and normalizes the data. Specifically, it performs line imputation, outlier removal, and smoothing.

[0355] Input: Raw, unprocessed data.

[0356] Output: A formatted dataset.

[0357] Step 3: Data analysis

[0358] How it works: The server inputs the preprocessed data into a generative AI model (e.g., a deep learning model using TensorFlow) to analyze the plant's condition and needs. This analysis includes predicting flowering time, assessing pest and disease risk, and generating a specific action plan.

[0359] Input: The formatted dataset.

[0360] Output: Plant status, risk assessment and action plan generated.

[0361] Step 4: Collect emotional data

[0362] Specific operation: The device uses sensors such as a camera, microphone, and smartwatch with a heart rate monitor to collect the user's facial expressions, voice, and heart rate, and transmits them to the server, thereby obtaining the user's emotional data.

[0363] Input: User biometric information (facial expression, voice, heart rate).

[0364] Output: User emotion data.

[0365] Step 5: Sentiment Analysis

[0366] Specific operation: The server analyzes the emotion data received from the device using an emotion engine (e.g., IBM Watson's emotion analysis API), thereby evaluating the user's emotional state and stress level.

[0367] Input: User emotion data.

[0368] Output: Evaluation of emotional state and stress level.

[0369] Step 6: Send information

[0370] Specific operation: Based on the analysis results and emotional data, the server notifies the user of appropriate actions. Notifications are sent via email or push notification, and the content is adjusted according to the user's emotional state. Users in a low-stress state will receive a simple notification such as "The time for pollination is approaching," while users in a high-stress state will receive a notification with a warning such as "Please proceed with the pollination work without overdoing it."

[0371] Input: Analysis results, emotional state and stress level.

[0372] Output: The adjusted notification content.

[0373] Step 7: Receive and display information

[0374] Specific operation: The device receives the notification sent from the server and displays it in a dedicated application. The displayed content includes specific response procedures and timing.

[0375] Input: Notification from the server.

[0376] Output: The action plan that is displayed to the user.

[0377] Step 8: Take Action

[0378] Specific operation: The user checks the information provided by the device and performs actual farm work, such as manually pollinating crops based on the notified pollination timing.

[0379] Input: Information provided by the device.

[0380] Output: The action that was performed.

[0381] Step 9: Provide feedback

[0382] How it works: The user inputs the results of their actions and observed changes into the device and sends them to the server. This feedback is stored in a database and used to improve the accuracy of the generative AI model.

[0383] Input: Execution results, observed changes.

[0384] Output: Feedback data.

[0385] Through this series of processing steps, the system can accurately grasp the plant's growth status, notify the user of appropriate actions, and reduce the user's psychological burden by taking into account the user's emotional state. Furthermore, the system can continuously optimize the entire system by utilizing feedback.

[0386] (Application example 2)

[0387] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0388] Currently, there are systems that closely monitor the condition of plants and provide advice on their cultivation, but there are no notification systems that take into account the user's emotions and stress level. This increases the psychological burden on the user, resulting in inefficient plant cultivation management. The present invention aims to solve these problems.

[0389] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting ultrasound, microRNA, and volatile substances from plants; means for preprocessing the collected data; means for analyzing the preprocessed data and using a generation AI to estimate the plant's condition; means for sending information about plant growth to the user based on the analysis results; means including an emotion engine that recognizes the user's emotions and adjusts the content and timing of notifications; and means for receiving feedback from the user and improving the accuracy of the generation AI. This enables efficient plant growth management while reducing the user's psychological burden by suggesting appropriate actions according to the plant's condition and providing notifications that take the user's emotional state into consideration.

[0390] "Ultrasound" is a sound wave with a vibration frequency of 20,000 Hz or higher, which is inaudible to the human ear but is used to detect the internal structure of materials.

[0391] "MicroRNA" is a short non-coding RNA molecule approximately 22 bases in length that functions to regulate gene expression.

[0392] "Volatiles" are substances that evaporate readily at room temperature and may be released by plants to indicate stress or other physiological conditions.

[0393] "Preprocessing" refers to the process of removing missing values ​​and outliers from collected data and preparing it for analysis.

[0394] "Generative AI" is a type of artificial intelligence technology that has the ability to generate new information and predictions through data analysis.

[0395] The "emotion engine" is a system that recognizes the user's emotions and stress level, and adjusts the content and timing of notifications based on that information.

[0396] "Notification content" is information that notifies the user of the analysis results and specific action plans based on those results.

[0397] "Feedback" is information provided by the user about the execution results and their evaluation, which helps improve the accuracy of the system.

[0398] "Plant condition" refers to comprehensive information about the plant's health, growth process, stress status, etc.

[0399] The system embodying this invention collects ultrasonic waves, microRNAs, and volatile substances from plants, preprocesses and analyzes them, and notifies the user of appropriate actions based on the results. It also incorporates an emotion engine that recognizes the user's emotions and adjusts the content and timing of notifications.

[0400] System hardware configuration

[0401] The system hardware is as follows:

[0402] Plant signal sensors: using ultrasonic sensors, microRNA sensors, and volatile substance sensors.

[0403] User emotion recognition devices: Use cameras, microphones, heart rate sensors, etc.

[0404] Devices: Smartphones, tablets, head-mounted displays, etc.

[0405] System software configuration

[0406] The main software used is as follows:

[0407] Generative AI models: perform data analysis and predictions.

[0408] Emotion recognition engine: Analyzes user emotional data and adjusts notification content.

[0409] Data preprocessing module: Imputes and removes missing and outlier values.

[0410] How the system is implemented

[0411] 1. Data Collection:

[0412] The server collects real-time data from various sensors attached to the plants, including ultrasonic waves, microRNAs, and volatile compounds, which reflect the physiological and stress states of the plants.

[0413] 2. Data Preprocessing:

[0414] The server detects missing or outliers in the collected data and performs imputation or removal as necessary, thereby ensuring the reliability and accuracy of the data.

[0415] 3. Data Analysis:

[0416] The pre-processed data is fed into a generative AI model to estimate the plant's condition and needs, such as predicting flowering time and risk of pest and disease damage.

[0417] 4. Emotion recognition:

[0418] The device uses a camera, microphone, and heart rate sensor to collect user emotional data and transmit it to a server, which then uses an emotion engine to analyze the data and determine the user's current emotional state and stress level.

[0419] 5. Information Transmission:

[0420] The server creates a specific action plan based on the analysis results and adjusts the content and timing of notifications according to the user's emotional state. For example, it may notify the user that pollination is approaching, or send a message containing a warning if the user is under high stress.

[0421] 6. Feedback received:

[0422] The server receives feedback from users and uses it to improve the accuracy of the generative AI. The feedback information is the results of the tasks performed and observations.

[0423] Specific examples

[0424] For example, if the server receives a signal from a sensor that "flowering is approaching" and analyzes it, it notifies the user that "the time for pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification will be sent as is, but if it detects that the user is in a high stress state, the message will be adjusted to include a warning that "the important time for pollination is approaching, but please proceed with the work without overdoing it."

[0425] Prompt Sentence Examples

[0426] "Sensor URL: http: / / api.plant_sensors.com / data"

[0427] "User data: including user emotional data"

[0428] This system allows for efficient plant growth management while reducing the psychological burden on users by suggesting appropriate actions based on the condition of the plant and providing notifications that take into account the user's emotional state.

[0429] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0430] Step 1:

[0431] Data collection

[0432] The server collects ultrasonic, microRNA, and volatile data in real time from sensors attached to plants. The input is the signal data from the sensors, and the output is the collected raw data. This data reflects the physiological state and stress status of the plants.

[0433] Step 2:

[0434] Data Preprocessing

[0435] The server detects missing values ​​and outliers in the collected data and imputes or removes them as necessary. The input is the collected raw data, and after passing through the data cleaning and imputation process, it outputs more accurate and reliable clean data.

[0436] Step 3:

[0437] Data analysis

[0438] The server inputs the preprocessed data into a generative AI model to estimate the plant's condition and needs. The input is preprocessed clean data, and the output is the plant's condition (e.g., flowering time, risk of pests, etc.). Specifically, the AI ​​model analyzes the data and makes predictions and diagnoses.

[0439] Step 4:

[0440] Emotion recognition data collection

[0441] The device uses a camera, microphone, and heart rate sensor to collect emotional data such as the user's facial expressions, voice, and heart rate. The input is sensory data from the camera and microphone, and the output is the user's emotional data (e.g., stress level, emotional state).

[0442] Step 5:

[0443] Emotion analysis

[0444] The server analyzes the user's emotional data sent from the device to understand the user's current emotional state and stress level. The input is the collected user's emotional data, and the output is the analyzed emotional state. The emotion engine runs and analyzes this.

[0445] Step 6:

[0446] Information transmission and notification coordination

[0447] The server creates a specific action plan based on the analysis results and adjusts the content and timing of notifications according to the user's emotional state. The input is the plant's condition and the user's emotional state, and the output is an adjusted notification message. Notifications are given taking into account the user's psychological state, such as warnings about pollination timing or insect damage.

[0448] Step 7:

[0449] Receive feedback

[0450] The server receives feedback provided by users (e.g., the results of actual work performed or observation data) and improves the accuracy of the generative AI model. The input is the feedback information from the user, and the output is an improved AI model. The feedback is analyzed and the system performance is improved.

[0451] This step not only suggests appropriate actions based on the condition of the plant, but also reduces the user's psychological burden by providing notifications that take into account the user's emotional state, thereby enabling efficient plant growth management.

[0452] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0453] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0454] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0455] [Second embodiment]

[0456] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0457] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0458] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0459] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0460] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0461] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0462] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0463] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0464] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0465] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0466] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0467] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0468] The system of the present invention analyzes signals emitted by plants, such as ultrasonic waves, microRNA, and volatile substances, estimates the plant's condition, and based on that estimates, suggests appropriate actions to the user. Specific processing of the program for implementing this system is explained below in natural language.

[0469] server

[0470] 1. Data Collection:

[0471] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances from plants. These signals are important data that indicate the timing of plant flowering and pollination, as well as damage caused by pests and diseases.

[0472] 2. Data Preprocessing:

[0473] The server detects missing or outliers in the collected data and performs imputation or removal as necessary. Preprocessing improves the reliability of the data and the accuracy of analysis.

[0474] 3. Data Analysis:

[0475] The server inputs the preprocessed data into the generative AI, which estimates the plant's condition and needs. The generative AI takes into account past data and environmental conditions to predict the optimal growing conditions for the plant.

[0476] 4. Information Transmission:

[0477] Based on the analysis results, the server notifies the user of specific information such as flowering time, pollination timing, and fruit thinning timing, allowing the user to carry out tasks at the appropriate time.

[0478] 5. Feedback received:

[0479] The server receives feedback from users to improve the accuracy of the generative AI, including the results of their work and observations.

[0480] Terminal (user's device)

[0481] 1. Receiving information:

[0482] The device receives analysis results and action plans sent from the server in real time, allowing users to understand the condition and needs of their plants at any given time.

[0483] 2. User Interface:

[0484] The device then displays the received information in an easy-to-understand format. For example, if the flowering period is approaching, it will display the specific pollination timing and procedure, and encourage the user to take appropriate action.

[0485] User

[0486] 1. Information verification:

[0487] Based on the information provided by the device, users can check specific actions for plant growth, allowing them to take necessary action before it is too late.

[0488] 2. Take action:

[0489] Based on the received information, the user can take appropriate action on-site, such as manually pollinating the crops based on the notified pollination timing.

[0490] 3. Providing Feedback:

[0491] The user inputs the results of their execution and any changes they observe into their device and sends them to the server. This feedback information is used to improve the analysis accuracy of the generative AI.

[0492] Specific examples

[0493] For example, the server receives a signal from a sensor indicating that "flowering is approaching," analyzes the signal, and notifies the user that "the time for pollination is approaching." The user checks the notification and pollinates at the appropriate time. The results of this work are reported to the server as feedback, contributing to improving the accuracy of the next prediction.

[0494] Furthermore, if the server detects signals of damage caused by pests, it will send an alert to the device saying, "Insect damage has occurred. Urgent action is required." Based on this, the user can quickly take measures such as spraying pesticides. The results are then sent to the server as feedback.

[0495] The processing flow will be explained below.

[0496] Server Processing Steps

[0497] Step 1: Data collection

[0498] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances from plants, which reflect the physiological state and stress status of plants.

[0499] Step 2: Data Preprocessing

[0500] The server preprocesses the collected data, specifically detecting missing and outlier values ​​and imputing or removing them as necessary, a crucial step to ensure the reliability and accuracy of the data.

[0501] Step 3: Data analysis

[0502] The pre-processed data is fed into generative AI to estimate the plant's condition and needs, including predicting flowering time and risk of pest and disease damage.

[0503] Step 4: Send information

[0504] Based on the analysis results, the server creates a specific action plan and notifies the user, including specific instructions and recommendations for plant growth.

[0505] Step 5: Receiving feedback

[0506] The server receives feedback from users, which includes work results and observations, and is important for improving the analysis accuracy of the generative AI.

[0507] Terminal (user device) processing steps

[0508] Step 1: Receiving information

[0509] The terminal receives the prediction results and action plans sent from the server in real time.

[0510] Step 2: Display information

[0511] The device then displays the received information in an easy-to-understand format for the user. For example, if the flowering period is approaching, it will display specific pollination timing and procedures.

[0512] Step 3: Operation Guide

[0513] If necessary, provide the user with additional information and instructions to assist with specific operations.

[0514] User processing steps

[0515] Step 1: Verify the information

[0516] The user checks the information provided by the terminal and understands specific actions to take regarding plant growth.

[0517] Step 2: Take action

[0518] The user then takes appropriate action based on instructions from the device, such as manually pollinating the plants based on the notified pollination timing.

[0519] Step 3: Provide feedback

[0520] The user inputs the results of their execution and any changes they observe into their device and sends them to the server. This feedback information is used to improve the analytical accuracy of the generative AI.

[0521] Through the above steps, the server, terminal, and user can work together to efficiently and effectively manage plant growth.

[0522] Example 1

[0523] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0524] Accurately understanding the condition of plants and taking appropriate action in a timely manner is extremely important for efficient plant cultivation and pest prevention. However, current technology does not yet fully establish a system for monitoring plant conditions in real time and taking appropriate measures. In particular, there is room for improvement in the accuracy of data collection and analysis, the timing of user notifications, and how feedback is utilized.

[0525] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0526] In this invention, the server includes means for collecting ultrasound, microRNA, and volatile substances from plants, means for preprocessing the collected data, means for inputting the preprocessed data into a generative AI model to estimate the plant's condition, means for transmitting information about plant growth to a user's device based on the analysis results, means for receiving user feedback and improving the accuracy of the generative AI model, means for visually displaying information through a user interface and notifying the user in real time, and means for retraining the generative AI model based on the feedback. This makes it possible to monitor and analyze the plant's condition in real time and provide the user with necessary action information at the appropriate time.

[0527] "Means for collecting ultrasound, microRNA, and volatile substances from plants" is a general term for devices and sensors that detect ultrasound, microRNA, and volatile substances emitted by plants and obtain this information as data.

[0528] "Means for preprocessing collected data" refers to processes and algorithms for detecting, complementing, or removing missing or outlier values ​​in collected signal data from plants, thereby improving the reliability of the data and the accuracy of the analysis.

[0529] "Means of inputting preprocessed data into a generative AI model to estimate plant condition" refers to a system or algorithm that inputs data that has undergone preprocessing into a machine learning model and uses that data to estimate the current condition and future needs of a plant.

[0530] "Means for sending information about plant growth to the user's device based on the analysis results" refers to a notification system or communication protocol that notifies the user of the results of the generative AI model's analysis, allowing the user to take the necessary action at the appropriate time.

[0531] "Means for receiving user feedback and improving the accuracy of generative AI models" refers to systems and processes for collecting work results and observational information from users and using it as training data for generative AI models to improve the accuracy of future predictions and analyses.

[0532] "Means for visually displaying information through a user interface and providing notifications in real time" refers to an interface for displaying analysis results and action plans in a format that is easily understandable to users, and a system or software for notifying users of necessary information in real time.

[0533] "Means for retraining a generative AI model based on feedback" refers to a process or algorithm that uses feedback information obtained from users to retrain a generative AI model and improve the model's accuracy and reliability.

[0534] The system of the present invention includes a series of means for analyzing signals emitted by plants in real time and providing specific cultivation actions to the user based on the analysis results. The hardware and software used in this system are as follows.

[0535] Hardware

[0536] Ultrasonic sensor: A device that detects ultrasonic waves emitted by plants and collects that data.

[0537] MicroRNA detection device: A device that detects microRNAs in plants and collects the data.

[0538] Volatile substance detector: A sensor that detects volatile organic compounds emitted by plants.

[0539] User's smart device: The device that receives notifications and displays the user interface (e.g., smartphone, tablet).

[0540] software

[0541] Generative AI model: A machine learning model for predicting and analyzing plant conditions. Built using frameworks such as TensorFlow.

[0542] Data preprocessing program: A script using a programming language such as Python to clean the collected data and impute missing values.

[0543] Notification system: A system that uses services such as Firebase to send notifications to users' devices in real time.

[0544] Server Operation

[0545] The server uses various sensors and specialized equipment to collect ultrasonic, microRNA, and volatile chemical signals from plants in real time. The collected data is then stored in a database, and missing and outliers are detected and imputed or removed. The preprocessed data is then fed into a generative AI model, which takes into account past data and current environmental conditions to predict the plant's condition and necessary actions.

[0546] Device behavior

[0547] The device receives the analysis results sent from the server in real time and displays them visually through the user interface. For example, if the flowering period is approaching, the device will display the specific pollination timing and procedure, prompting the user to take appropriate action. The notification system uses Firebase and other services to quickly deliver the analysis results to the user's device.

[0548] User Actions

[0549] Based on the information provided by the device, the user plans and executes specific actions related to plant growth. For example, if the user receives a notification that "pollination time is approaching," the user prepares for pollination and carries it out at the appropriate time. The results of the actual work and any observed changes in the plant are entered into the device, and the feedback is sent to the server. This feedback is used to retrain the generative AI model, contributing to improving the accuracy of analysis in future generations.

[0550] Specific examples

[0551] For example, the server receives a signal from a sensor that "flowering is approaching," analyzes the signal, and notifies the user that "the time for pollination is approaching." The user checks this notification and performs pollination at the appropriate time. The results of this work are reported to the server as feedback, contributing to improved prediction accuracy for the next time. Furthermore, if the server detects a signal of damage caused by pests, it will send an alert to the device saying, "Insect damage has occurred. Urgent action is required." Based on this, the user can quickly take measures such as spraying pesticides, and send the results of this to the server as feedback.

[0552] Prompt Sentence Examples

[0553] You are designing a system that analyzes signals emitted by plants and provides information about their status to the user. These signals include ultrasound, microRNA, and volatile substances. Please specify the hardware and software used in this system and explain the specific data processing and data calculation methods.

[0554] These processes enable real-time monitoring and analysis of plant conditions, providing users with timely information on the actions they need to take.

[0555] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0556] Server Processing Steps

[0557] Step 1: Data collection

[0558] Input: Signal data from ultrasonic sensors, microRNA detectors, and volatile substance detectors.

[0559] How it works: The server receives real-time signal data about the plant's condition from these various sensors. The sensors record data every second and send it to the server. The server stores the received data in a database.

[0560] Output: The raw data collected.

[0561] Step 2: Data Preprocessing

[0562] Input: Raw data collected.

[0563] Specific operation: The server detects missing values ​​and outliers in the received data and processes them (complements or removes them). For complementation, it uses the most recent valid data, and for outliers, it corrects or removes values ​​that are out of range.

[0564] Output: Preprocessed clean data.

[0565] Step 3: Data analysis

[0566] Input: Preprocessed clean data.

[0567] Specific operation: The server inputs the preprocessed data into the generative AI model, which then predicts the plant's condition and growth based on past data and current environmental conditions.

[0568] Output: Analysis results (e.g., flowering time, pollination timing, risk of pests and diseases, etc.).

[0569] Step 4: Send information

[0570] Input: Analysis results.

[0571] Specific operation: Based on the analysis results of the generation AI, the server notifies the user of necessary action information (e.g., flowering time, pollination timing). This information is sent to the user's device via a notification service such as Firebase.

[0572] Output: Notifications sent to the user's device.

[0573] Step 5: Receiving feedback

[0574] Input: Feedback data from users.

[0575] Specific operation: Receives work results and observation data entered by the user into the device, stores this feedback data in a database, and creates a new learning set to retrain the generative AI model.

[0576] Output: Feedback data, improved generative AI model.

[0577] Terminal processing steps

[0578] Step 1: Receiving information

[0579] Input: Analysis results and action plans sent from the server.

[0580] What it does: The device maintains a connection with the server and periodically checks for new notifications. If any notifications are available, they are immediately downloaded and displayed to the user.

[0581] Output: Notification data waiting to be displayed.

[0582] Step 2: User Interface Display

[0583] Input: Notification data.

[0584] Specific operation: The device analyzes the received notification and displays it on the user interface. It provides instructions in visually appealing graphs and text, making it easy for the user to understand.

[0585] Output: A screen display for the user to see.

[0586] User processing steps

[0587] Step 1: Verify the information

[0588] Input: Notifications from your device.

[0589] Specific behavior: The user taps the notification on their device to view more information and plan appropriate actions based on the information displayed.

[0590] Output: A confirmed action plan.

[0591] Step 2: Take action

[0592] Input: Confirmed action plan.

[0593] Specific actions: The user performs the necessary work on site based on the notification. For example, if pollination is required, prepare pollination tools and perform pollination at the appropriate time.

[0594] Output: The training work carried out.

[0595] Step 3: Provide feedback

[0596] Input: Development activities performed and observation data.

[0597] Specific operation: The user inputs the results of their work and observed changes into the terminal and sends them to the server, which improves the analysis accuracy of the entire system.

[0598] Output: Feedback data sent to the server.

[0599] (Application example 1)

[0600] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0601] Conventional plant growth management systems are required to not only grasp the state of plants, but also to formulate and execute appropriate action plans, but there are many challenges to achieving this. Specifically, they require accurate data collection and analysis to accurately analyze signals from plants and provide optimal growth conditions, as well as the autonomy to take appropriate actions based on that data. Conventional methods lack sufficient analytical accuracy and automation of actions, making effective plant growth management difficult.

[0602] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0603] In this invention, the server includes means for collecting ultrasound, microRNA, and volatile substances from plants, means for preprocessing the collected data, means for analyzing the preprocessed data and using a generating AI to estimate the state of the plant, means for sending information about plant growth to a user based on the analysis results, means for receiving feedback from the user and improving the accuracy of the generating AI, and means for controlling a factory robot that executes an action plan required for plant growth based on the analysis results. This realizes a system that can perform everything from data analysis to action execution in an integrated manner, enabling optimal plant growth management.

[0604] "Ultrasound" refers to sound waves with a frequency of 20 kilohertz or higher, which contain unique signals emitted by plants.

[0605] "MicroRNAs" are small RNA molecules that regulate gene expression in plants and are important biomarkers that reflect the health and stress status of plants.

[0606] "Volatiles" are volatile compounds emitted by plants that indicate the plant's condition or environmental stress.

[0607] "Generative AI" is an artificial intelligence model that estimates the condition of plants based on collected data and suggests optimal growth conditions and action plans.

[0608] An "action plan" is a plan that includes specific actions and procedures required for the growth and protection of plants, based on the analysis results of the generative AI.

[0609] A "factory robot" is a mechanical device that operates autonomously or semi-autonomously within a factory and executes action plans related to plant growth management.

[0610] "Data preprocessing" is the process of detecting missing or outliers in collected plant data and imputing or removing them appropriately.

[0611] "Feedback" is information provided by the user about execution results and observed changes, which is used to improve the accuracy of the generative AI.

[0612] This invention is a system for understanding the state of plants in real time and implementing appropriate cultivation actions based on that information. The main components of the system are a server, terminals, and factory robots.

[0613] server

[0614] The server includes the following means:

[0615] 1. Data collection: The server collects data about the plants in real time from various sensors, such as ultrasonic sensors, microRNA analyzers, and volatile substance detectors.

[0616] 2. Data preprocessing: Detect missing or outliers in the collected data, and impute or remove them as necessary to improve the reliability of the data and the accuracy of the analysis.

[0617] 3. Generative AI analysis: The preprocessed data is input into the generative AI, which takes into account past data and environmental conditions to estimate the plant's condition and necessary growing conditions.

[0618] 4. Information transmission: Based on the analysis results, specific information such as flowering time, pollination timing, and fruit thinning timing is notified to the user. In addition, an action plan generated based on the analysis results is transmitted to the factory robot.

[0619] 5. Feedback reception: Receive feedback from the user on the actions taken to improve the analytical accuracy of the generative AI.

[0620] Terminal (user's device)

[0621] The terminal includes the following means:

[0622] 1. Information reception: Receive analysis results and action plans sent from the server in real time.

[0623] 2. User interface: The received information is displayed to the user in an easy-to-understand format, showing the notified pollination timing and procedures.

[0624] Factory robots

[0625] The factory robot includes the following means:

[0626] 1. Action plan execution: According to the action plan sent from the server, appropriate cultivation tasks are carried out automatically or semi-automatically, such as watering, fertilizing, and pest control.

[0627] Overall processing flow

[0628] The server first collects data about the plants from various sensors. The collected data is preprocessed, and missing or outliers are filled in or removed. The preprocessed data is analyzed by the generative AI, which estimates the plant's condition and the conditions necessary for its growth. The analysis results are notified to the user's device, and an appropriate action plan is sent to the factory robot. The factory robot carries out the growth work based on this action plan. The user provides feedback on actual work results and observations, which are sent to the server and used to improve the accuracy of the generative AI.

[0629] Specific examples

[0630] For example, the server collects ultrasonic data, microRNA data, and volatile substance data from a plant and analyzes it to determine that the plant is nearing flowering. As a result, the user's device is notified that "the time for pollination is approaching." At the same time, an action plan for carrying out "pollination" is sent to the factory robot. The user checks the notification, and the robot carries out pollination at the appropriate time. The results are then reported to the server as feedback.

[0631] Example prompts for generative AI models

[0632] Enter your plant's ultrasound, microRNA, and volatile data, and we'll estimate your next steps and suggest an appropriate course of action.

[0633] Example data:

[0634] Ultrasonic: 10kHz

[0635] MicroRNA: [RNA_sequence_A, RNA_sequence_B]

[0636] Volatile substances: [Compound A, Compound B]

[0637] In this way, the entire system works together to achieve optimal plant growth.

[0638] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0639] Step 1:

[0640] Data collection

[0641] The server collects real-time plant data from sensors such as ultrasonic sensors, microRNA analyzers, and volatile substance detectors. In this step, raw data from each sensor is input and output in its raw format.

[0642] Step 2:

[0643] Data Preprocessing

[0644] The server detects missing or outliers in the collected data and imputes or removes them as necessary. Here, the raw data collected in the previous step is used as input, and reliable data is output with outliers replaced with default values ​​such as "default_value." Specific operations include checking the consistency of the data and filtering outliers.

[0645] Step 3:

[0646] Generative AI analysis

[0647] The server inputs the preprocessed data and uses generative AI to estimate the plant's condition. The output is an analysis result that clarifies the plant's current condition and necessary growing conditions. Input data to the generative AI model includes values ​​of ultrasound, microRNA, and volatile substances.

[0648] Step 4:

[0649] Information transmission

[0650] Based on the analysis results of the generated AI, the server sends information about plant growth to the user's device. It also sends an action plan to the factory robot. The analysis results are used as input, and the output is instructions and action plans received by the device and robot. The specific operation involves data communication via a network.

[0651] Step 5:

[0652] Information reception (terminal)

[0653] The terminal receives analysis results and action plans sent from the server in real time. The input is the analysis results and action plans from the server, and the output is display information converted into a format that can be viewed by the user. Specific operations include analyzing the received data and displaying it on the user interface.

[0654] Step 6:

[0655] Action plan execution (factory robot)

[0656] Factory robots automatically or semi-automatically perform appropriate cultivation tasks based on an action plan sent from a server. The action plan from the server is given as input, and the output is the actual physical work, such as watering, fertilizing, and pest control. Specific operations include robot motion control and the use of sensors.

[0657] Step 7:

[0658] Feedback provided by users

[0659] The user inputs feedback about the tasks performed and the changes observed into the terminal, which then sends it to the server. The input is feedback data from the user, and the output is data used to update the generative AI. Specific operations include data input via the user interface and feedback transmission via the network.

[0660] Step 8:

[0661] Feedback reception (server)

[0662] The server receives feedback from users and uses it to improve the accuracy of the generative AI. The input is the user feedback data, and the output is an updated, more accurate generative AI model. Specific operations include analyzing the feedback data and training the generative AI model.

[0663] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0664] The present invention is a system that analyzes signals emitted by plants, such as ultrasound, microRNA, and volatile substances, to estimate the plant's condition and notify the user of appropriate actions based on the results. This system incorporates an emotion engine that recognizes the user's emotions, and also has the function of adjusting the content and timing of notifications based on the user's emotions and making suggestions to reduce the user's stress level. Specific processing of the program is described below.

[0665] server

[0666] 1. Data Collection:

[0667] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances from plants, which reflect the physiological state and stress status of plants.

[0668] 2. Data Preprocessing:

[0669] The server preprocesses the collected data, specifically detecting missing and outlier values ​​and imputing or removing them as necessary, which is an important step to ensure the reliability and accuracy of the data.

[0670] 3. Data Analysis:

[0671] The pre-processed data is fed into generative AI to estimate the plant's condition and needs, including predicting flowering time and risk of pest and disease damage.

[0672] 4. Emotion analysis:

[0673] The server uses an emotion engine to analyze the user's emotional data sent from the device, thereby understanding the user's current emotional state and stress level.

[0674] 5. Information Transmission:

[0675] Based on the analysis results, the server creates a specific action plan, adjusting the content and timing of notifications according to the user's emotional state to notify the user in the most optimal way.

[0676] 6. Feedback received:

[0677] The server receives feedback from users to improve the accuracy of the generative AI. The feedback information is the results of work and observations, and is useful for future analysis.

[0678] Terminal (user's device)

[0679] 1. Receiving information:

[0680] The terminal receives the prediction results and action plans sent from the server in real time.

[0681] 2. Emotional Data Collection:

[0682] The device uses a camera, microphone, and other sensors to collect information such as the user's facial expressions, voice, and heart rate, and transmits this information to a server as emotional data.

[0683] 3. Information display:

[0684] The device then displays the received information in an easy-to-understand format for the user. For example, if the flowering period is approaching, it will display specific pollination timing and procedures.

[0685] 4. Operation Guide:

[0686] If necessary, provide the user with additional information and instructions to assist with specific operations.

[0687] User

[0688] 1. Information verification:

[0689] The user checks the information provided by the terminal and understands specific actions to take regarding plant growth.

[0690] 2. Take action:

[0691] The user then takes appropriate action based on instructions from the device, such as manually pollinating the plants based on the notified pollination timing.

[0692] 3. Record your emotions:

[0693] Users record their emotions and stress levels on their devices and send them to a server, where the emotional data is used for analysis and action optimization.

[0694] 4. Providing Feedback:

[0695] The user inputs the results of their actions and any changes they observe into their device and sends them to the server. This feedback information is used to improve the analytical accuracy of the generative AI.

[0696] Specific examples

[0697] For example, if the server receives a signal from a sensor that "flowering is approaching" and analyzes it, it notifies the user that "the time for pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification will be sent as is, but if it detects that the user is in a high stress state, the message will be adjusted to include a warning such as "The important time for pollination is approaching, but please proceed with the work without overdoing it."

[0698] Furthermore, if the server detects signals of damage caused by pests, it sends an alert to the device saying, "Insect damage has occurred. Immediate action is required." If the user feels high stress in response to this alert, the emotion engine sends a message emphasizing how to deal with the situation, such as, "Insect damage has occurred. Please calmly take action by following the procedures." Based on this, the user can quickly take action, such as spraying pesticides. The results are then sent to the server as feedback.

[0699] In this way, the server, terminals, and users can work together to efficiently and effectively manage plant cultivation. In addition, the combination of the emotion engine reduces the psychological burden on users and improves work efficiency.

[0700] The processing flow will be explained below.

[0701] Server Processing Steps

[0702] Step 1: Data collection

[0703] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances, to detect the physiological and stress states of plants.

[0704] Step 2: Data Preprocessing

[0705] The server detects whether there are any missing or outliers in the collected data and performs imputation or removal as necessary, thereby improving the reliability of the data and the accuracy of the analysis.

[0706] Step 3: Data analysis

[0707] The preprocessed data is input into a generative AI that estimates the plant's condition and needs, and the generative AI predicts flowering time and pest and disease risk based on past data and environmental conditions.

[0708] Step 4: Sentiment Analysis

[0709] The server uses an emotion engine to analyze the user's emotional data received from the device, thereby understanding the user's current emotional state and stress level.

[0710] Step 5: Send information

[0711] Based on the analysis results, the server creates a specific action plan, adjusting the content and timing of notifications according to the user's emotional state to notify the user in the most optimal way.

[0712] Step 6: Receiving feedback

[0713] The server receives feedback from users, including the results of tasks performed and observations, and uses this feedback to improve the analytical accuracy of the generative AI.

[0714] Terminal processing steps

[0715] Step 1: Receiving information

[0716] The terminal receives the analysis results and action plans sent from the server in real time.

[0717] Step 2: Collecting Emotional Data

[0718] The device uses a camera, microphone, and other sensors to collect emotional data such as the user's facial expressions, voice, and heart rate, and sends this data to a server.

[0719] Step 3: Display information

[0720] The device then displays the received information in an easy-to-understand format for the user. For example, if the flowering period is approaching, it will display specific pollination timing and procedures.

[0721] User processing steps

[0722] Step 1: Verify the information

[0723] The user checks the information provided by the terminal and understands specific actions to take regarding plant growth.

[0724] Step 2: Take action

[0725] The user can then take appropriate action based on instructions from the device, such as manually pollinating the plants based on the notified pollination timing.

[0726] Step 3: Record your emotions

[0727] Users record their emotions and stress levels on their devices and send them to a server, where the emotional data is used for analysis and action optimization.

[0728] Step 4: Provide feedback

[0729] The user inputs the results of their execution and any observed changes into their device and sends them to the server. The feedback information is used to improve the analysis accuracy of the generative AI.

[0730] Specific examples

[0731] For example, if the server receives a signal from a sensor that "flowering is approaching" and analyzes it, it notifies the user that "the time for pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification will be sent as is, but if it detects that the user is in a high stress state, the message will be adjusted to include a warning such as "The important time for pollination is approaching, but please proceed with the work without overdoing it."

[0732] When the server detects a signal of pest damage, it sends an alert to the device saying, "Insect damage has occurred. Immediate action is required." If the user feels high stress in response to this alert, the emotion engine sends a message emphasizing how to deal with the situation, such as, "Insect damage has occurred, but please calmly follow the procedures and take action." Based on this, the user can quickly take action, such as spraying pesticides, and send the results of that action to the server as feedback.

[0733] In this way, the server, terminals, and users can work together to efficiently and effectively manage plant cultivation. In addition, the combination of the emotion engine reduces the psychological burden on users and improves work efficiency.

[0734] Example 2

[0735] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0736] Conventional agricultural systems have had difficulty accurately understanding the physiological state and stress levels of plants and taking appropriate measures based on that information. Furthermore, they do not provide information that takes into account the user's emotional state, making it impossible to reduce the user's psychological burden. Furthermore, continuous system optimization based on feedback is insufficient.

[0737] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting ultrasound, microRNA, and volatile substances from plants, means for preprocessing the collected data, means for analyzing the preprocessed data and using a generation AI to estimate the plant's condition, means for analyzing user emotion data, means for sending information about plant growth to the user based on the analysis results and the emotion data, and means for receiving feedback from the user and improving the accuracy of the generation AI. This makes it possible to accurately grasp the plant's condition and take appropriate measures based on that understanding, taking the user's emotional state into consideration. Furthermore, the system can be continuously optimized by reflecting user feedback.

[0738] "Ultrasound" is a type of sound wave that usually has a frequency of 20 kHz or higher and is inaudible to the human ear.

[0739] "MicroRNA" is a small RNA molecule that regulates gene expression and has the function of suppressing the production of specific proteins.

[0740] "Volatile substances" are substances that easily evaporate into gas at room temperature and serve as indicators of the physiological state and stress of plants.

[0741] "Preprocessing" refers to a series of processes that prepare collected data in a form suitable for analysis, and includes operations such as filling in and removing missing or outlier values ​​in the data.

[0742] "Generative AI" is an artificial intelligence technology used to analyze collected data and predict specific situations or conditions.

[0743] "Emotion data" is data including the user's facial expression, voice, heart rate, etc., and indicates the user's current emotional state and stress level.

[0744] "Feedback" is information that records the results of operations performed by the user and observed changes and returns them to the system.

[0745] "Information regarding plant cultivation" includes information regarding the flowering period of plants, timing of pollination, fruit thinning, harvesting time, proper use of pesticides, and avoidance of insect damage.

[0746] "Missing data" refers to values ​​that are not recorded because measurements or collection were not performed correctly.

[0747] An "outlier" is a data value that deviates from the normal range and may affect the analysis results.

[0748] The "emotional state of the user" indicates the state of emotions or moods that the user is feeling, and specifically includes joy, sadness, surprise, anger, fear, and the like.

[0749] The "stress level" indicates the degree of mental tension or pressure that the user feels.

[0750] The "emotion engine" is software that analyzes a user's emotional data and evaluates their current emotional state and stress level.

[0751] MODE FOR CARRYING OUT THE INVENTION

[0752] This invention is a system that analyzes signals emitted by plants, such as ultrasonic waves, microRNA, and volatile substances, to estimate the plant's condition and notify the user of appropriate actions based on the results. This system incorporates an emotion engine that recognizes the user's emotions, and also has the function of adjusting the content and timing of notifications based on the user's emotions and making suggestions to reduce the user's stress level.

[0753] Hardware and software used

[0754] server:

[0755] Hardware: High-performance server (e.g., server with Intel Xeon processor)

[0756] Software: Python, TensorFlow, Pandas, Numpy, IBM Watson Sentiment Analysis API, Microsoft Azure

[0757] Terminal (user's device):

[0758] Hardware: Smartphone (Android or iOS), wearable device (smartwatch with heart rate monitor)

[0759] Software: Dedicated application (Android / iOS app), Bluetooth module

[0760] Data collection

[0761] The server collects real-time signal data of ultrasonic waves, microRNAs, and volatile substances from multiple sensors (e.g., ultrasonic sensors, RNA sequencers, and gas chromatographs) in plants. These signal data reflect the physiological state and stress status of plants.

[0762] Data Preprocessing

[0763] The server preprocesses the collected data using the Python Pandas library to detect missing and outliers, and impute or remove them as needed. Data smoothing and normalization are also performed at this stage.

[0764] Data analysis

[0765] The preprocessed data is then fed into a generative AI model, a deep learning model built using TensorFlow, which analyzes the plant's condition, predicting flowering time and assessing pest and disease risk, and creating a specific action plan.

[0766] Emotion analysis

[0767] The server analyzes the user's emotional data sent from the device using IBM Watson's emotion analysis API, which allows it to understand the user's current emotional state and stress level. Emotional data is collected from the device's camera, microphone, and wearable device.

[0768] Information transmission

[0769] Based on the analysis results and emotional data, the server notifies the user of the appropriate action: for users in a low-stress state, it sends a simple notification such as "The time for pollination is approaching," and for users in a high-stress state, it sends a notification with a warning such as "Please proceed with the pollination work without overdoing it."

[0770] Receive feedback

[0771] The server receives feedback from users, who send the results of their work and observed changes as feedback to the server through a dedicated application. This feedback information is recorded in a database and used to improve the accuracy of the generative AI model.

[0772] Specific examples

[0773] For example, if a plant is about to flower, the server analyzes the data from the sensor and notifies the user that "pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification is sent as is, but if the user is in a high stress state, an adjusted message is sent saying, "The important pollination time is approaching, but please proceed with your work without overdoing it."

[0774] When signals of pest damage are detected, the server sends an alert to the device saying, "Insect damage has occurred. Immediate action is required." If the user feels high stress in response to this alert, the emotion engine sends a message emphasizing countermeasures, such as, "Insect damage has occurred, but please calmly follow the procedures and take action." Based on this, the user can quickly take measures such as spraying pesticides.

[0775] In this way, the server, terminals, and users can work together to efficiently and effectively manage plant cultivation. Furthermore, by utilizing the emotion engine, the psychological burden on users can be reduced and work efficiency can be improved.

[0776] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0777] Program processing flow

[0778] Step 1: Data collection

[0779] How it works: The server collects data in real time from multiple sensors (ultrasonic sensors, RNA sequencers, and gas chromatographs) attached to plants, including the sounds of plant growth (ultrasonic waves), microRNAs (gene expression status), and volatile substances (vaporized chemical components).

[0780] Input: Signal data from the sensor.

[0781] Output: Raw, unprocessed data.

[0782] Step 2: Data Preprocessing

[0783] Specific operation: The server preprocesses the collected raw data. At this stage, it creates a data frame using Python's Pandas library, detects and imputes missing values ​​and outliers, and normalizes the data. Specifically, it performs line imputation, outlier removal, and smoothing.

[0784] Input: Raw, unprocessed data.

[0785] Output: A formatted dataset.

[0786] Step 3: Data analysis

[0787] How it works: The server inputs the preprocessed data into a generative AI model (e.g., a deep learning model using TensorFlow) to analyze the plant's condition and needs. This analysis includes predicting flowering time, assessing pest and disease risk, and generating a specific action plan.

[0788] Input: The formatted dataset.

[0789] Output: Plant status, risk assessment and action plan generated.

[0790] Step 4: Collect emotional data

[0791] Specific operation: The device uses sensors such as a camera, microphone, and smartwatch with a heart rate monitor to collect the user's facial expressions, voice, and heart rate, and transmits them to the server, thereby obtaining the user's emotional data.

[0792] Input: User biometric information (facial expression, voice, heart rate).

[0793] Output: User emotion data.

[0794] Step 5: Sentiment Analysis

[0795] Specific operation: The server analyzes the emotion data received from the device using an emotion engine (e.g., IBM Watson's emotion analysis API), thereby evaluating the user's emotional state and stress level.

[0796] Input: User emotion data.

[0797] Output: Evaluation of emotional state and stress level.

[0798] Step 6: Send information

[0799] Specific operation: Based on the analysis results and emotional data, the server notifies the user of appropriate actions. Notifications are sent via email or push notification, and the content is adjusted according to the user's emotional state. Users in a low-stress state will receive a simple notification such as "The time for pollination is approaching," while users in a high-stress state will receive a notification with a warning such as "Please proceed with the pollination work without overdoing it."

[0800] Input: Analysis results, emotional state and stress level.

[0801] Output: The adjusted notification content.

[0802] Step 7: Receive and display information

[0803] Specific operation: The device receives the notification sent from the server and displays it in a dedicated application. The displayed content includes specific response procedures and timing.

[0804] Input: Notification from the server.

[0805] Output: The action plan that is displayed to the user.

[0806] Step 8: Take Action

[0807] Specific operation: The user checks the information provided by the device and performs actual farm work, such as manually pollinating crops based on the notified pollination timing.

[0808] Input: Information provided by the device.

[0809] Output: The action that was performed.

[0810] Step 9: Provide feedback

[0811] How it works: The user inputs the results of their actions and observed changes into the device and sends them to the server. This feedback is stored in a database and used to improve the accuracy of the generative AI model.

[0812] Input: Execution results, observed changes.

[0813] Output: Feedback data.

[0814] Through this series of processing steps, the system can accurately grasp the plant's growth status, notify the user of appropriate actions, and reduce the user's psychological burden by taking into account the user's emotional state. Furthermore, the system can continuously optimize the entire system by utilizing feedback.

[0815] (Application example 2)

[0816] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0817] Currently, there are systems that closely monitor the condition of plants and provide advice on their cultivation, but there are no notification systems that take into account the user's emotions and stress level. This increases the psychological burden on the user, resulting in inefficient plant cultivation management. The present invention aims to solve these problems.

[0818] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting ultrasound, microRNA, and volatile substances from plants; means for preprocessing the collected data; means for analyzing the preprocessed data and using a generation AI to estimate the plant's condition; means for sending information about plant growth to the user based on the analysis results; means including an emotion engine that recognizes the user's emotions and adjusts the content and timing of notifications; and means for receiving feedback from the user and improving the accuracy of the generation AI. This enables efficient plant growth management while reducing the user's psychological burden by suggesting appropriate actions according to the plant's condition and providing notifications that take the user's emotional state into consideration.

[0819] "Ultrasound" is a sound wave with a vibration frequency of 20,000 Hz or higher, which is inaudible to the human ear but is used to detect the internal structure of materials.

[0820] "MicroRNA" is a short non-coding RNA molecule approximately 22 bases in length that functions to regulate gene expression.

[0821] "Volatiles" are substances that evaporate readily at room temperature and may be released by plants to indicate stress or other physiological conditions.

[0822] "Preprocessing" refers to the process of removing missing values ​​and outliers from collected data and preparing it for analysis.

[0823] "Generative AI" is a type of artificial intelligence technology that has the ability to generate new information and predictions through data analysis.

[0824] The "emotion engine" is a system that recognizes the user's emotions and stress level, and adjusts the content and timing of notifications based on that information.

[0825] "Notification content" is information that notifies the user of the analysis results and specific action plans based on those results.

[0826] "Feedback" is information provided by the user about the execution results and their evaluation, which helps improve the accuracy of the system.

[0827] "Plant condition" refers to comprehensive information about the plant's health, growth process, stress status, etc.

[0828] The system embodying this invention collects ultrasonic waves, microRNAs, and volatile substances from plants, preprocesses and analyzes them, and notifies the user of appropriate actions based on the results. It also incorporates an emotion engine that recognizes the user's emotions and adjusts the content and timing of notifications.

[0829] System hardware configuration

[0830] The system hardware is as follows:

[0831] Plant signal sensors: using ultrasonic sensors, microRNA sensors, and volatile substance sensors.

[0832] User emotion recognition devices: Use cameras, microphones, heart rate sensors, etc.

[0833] Devices: Smartphones, tablets, head-mounted displays, etc.

[0834] System software configuration

[0835] The main software used is as follows:

[0836] Generative AI models: perform data analysis and predictions.

[0837] Emotion recognition engine: Analyzes user emotional data and adjusts notification content.

[0838] Data preprocessing module: Imputes and removes missing and outlier values.

[0839] How the system is implemented

[0840] 1. Data Collection:

[0841] The server collects real-time data from various sensors attached to the plants, including ultrasonic waves, microRNAs, and volatile compounds, which reflect the physiological and stress states of the plants.

[0842] 2. Data Preprocessing:

[0843] The server detects missing or outliers in the collected data and performs imputation or removal as necessary, thereby ensuring the reliability and accuracy of the data.

[0844] 3. Data Analysis:

[0845] The pre-processed data is fed into a generative AI model to estimate the plant's condition and needs, such as predicting flowering time and risk of pest and disease damage.

[0846] 4. Emotion recognition:

[0847] The device uses a camera, microphone, and heart rate sensor to collect user emotional data and transmit it to a server, which then uses an emotion engine to analyze the data and determine the user's current emotional state and stress level.

[0848] 5. Information Transmission:

[0849] The server creates a specific action plan based on the analysis results and adjusts the content and timing of notifications according to the user's emotional state. For example, it may notify the user that pollination is approaching, or send a message containing a warning if the user is under high stress.

[0850] 6. Feedback received:

[0851] The server receives feedback from users and uses it to improve the accuracy of the generative AI. The feedback information is the results of the tasks performed and observations.

[0852] Specific examples

[0853] For example, if the server receives a signal from a sensor that "flowering is approaching" and analyzes it, it notifies the user that "the time for pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification will be sent as is, but if it detects that the user is in a high stress state, the message will be adjusted to include a warning that "the important time for pollination is approaching, but please proceed with the work without overdoing it."

[0854] Prompt Sentence Examples

[0855] "Sensor URL: http: / / api.plant_sensors.com / data"

[0856] "User data: including user emotional data"

[0857] This system allows for efficient plant growth management while reducing the psychological burden on users by suggesting appropriate actions based on the condition of the plant and providing notifications that take into account the user's emotional state.

[0858] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0859] Step 1:

[0860] Data collection

[0861] The server collects ultrasonic, microRNA, and volatile data in real time from sensors attached to plants. The input is the signal data from the sensors, and the output is the collected raw data. This data reflects the physiological state and stress status of the plants.

[0862] Step 2:

[0863] Data Preprocessing

[0864] The server detects missing values ​​and outliers in the collected data and imputes or removes them as necessary. The input is the collected raw data, and after passing through the data cleaning and imputation process, it outputs more accurate and reliable clean data.

[0865] Step 3:

[0866] Data analysis

[0867] The server inputs the preprocessed data into a generative AI model to estimate the plant's condition and needs. The input is preprocessed clean data, and the output is the plant's condition (e.g., flowering time, risk of pests, etc.). Specifically, the AI ​​model analyzes the data and makes predictions and diagnoses.

[0868] Step 4:

[0869] Emotion recognition data collection

[0870] The device uses a camera, microphone, and heart rate sensor to collect emotional data such as the user's facial expressions, voice, and heart rate. The input is sensory data from the camera and microphone, and the output is the user's emotional data (e.g., stress level, emotional state).

[0871] Step 5:

[0872] Emotion analysis

[0873] The server analyzes the user's emotional data sent from the device to understand the user's current emotional state and stress level. The input is the collected user's emotional data, and the output is the analyzed emotional state. The emotion engine runs and analyzes this.

[0874] Step 6:

[0875] Information transmission and notification coordination

[0876] The server creates a specific action plan based on the analysis results and adjusts the content and timing of notifications according to the user's emotional state. The input is the plant's condition and the user's emotional state, and the output is an adjusted notification message. Notifications are given taking into account the user's psychological state, such as warnings about pollination timing or insect damage.

[0877] Step 7:

[0878] Receive feedback

[0879] The server receives feedback provided by users (e.g., the results of actual work performed or observation data) and improves the accuracy of the generative AI model. The input is the feedback information from the user, and the output is an improved AI model. The feedback is analyzed and the system performance is improved.

[0880] This step not only suggests appropriate actions based on the condition of the plant, but also reduces the user's psychological burden by providing notifications that take into account the user's emotional state, thereby enabling efficient plant growth management.

[0881] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0882] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0883] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0884] [Third embodiment]

[0885] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0886] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0887] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0888] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0889] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0890] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0891] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0892] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0893] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0894] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0895] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0896] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0897] The system of the present invention analyzes signals emitted by plants, such as ultrasonic waves, microRNA, and volatile substances, estimates the plant's condition, and based on that estimates, suggests appropriate actions to the user. Specific processing of the program for implementing this system is explained below in natural language.

[0898] server

[0899] 1. Data Collection:

[0900] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances from plants. These signals are important data that indicate the timing of plant flowering and pollination, as well as damage caused by pests and diseases.

[0901] 2. Data Preprocessing:

[0902] The server detects missing or outliers in the collected data and performs imputation or removal as necessary. Preprocessing improves the reliability of the data and the accuracy of analysis.

[0903] 3. Data Analysis:

[0904] The server inputs the preprocessed data into the generative AI, which estimates the plant's condition and needs. The generative AI takes into account past data and environmental conditions to predict the optimal growing conditions for the plant.

[0905] 4. Information Transmission:

[0906] Based on the analysis results, the server notifies the user of specific information such as flowering time, pollination timing, and fruit thinning timing, allowing the user to carry out tasks at the appropriate time.

[0907] 5. Feedback received:

[0908] The server receives feedback from users to improve the accuracy of the generative AI, including the results of their work and observations.

[0909] Terminal (user's device)

[0910] 1. Receiving information:

[0911] The device receives analysis results and action plans sent from the server in real time, allowing users to understand the condition and needs of their plants at any given time.

[0912] 2. User Interface:

[0913] The device then displays the received information in an easy-to-understand format. For example, if the flowering period is approaching, it will display the specific pollination timing and procedure, and encourage the user to take appropriate action.

[0914] User

[0915] 1. Information verification:

[0916] Based on the information provided by the device, users can check specific actions for plant growth, allowing them to take necessary action before it is too late.

[0917] 2. Take action:

[0918] Based on the received information, the user can take appropriate action on-site, such as manually pollinating the crops based on the notified pollination timing.

[0919] 3. Providing Feedback:

[0920] The user inputs the results of their execution and any changes they observe into their device and sends them to the server. This feedback information is used to improve the analysis accuracy of the generative AI.

[0921] Specific examples

[0922] For example, the server receives a signal from a sensor indicating that "flowering is approaching," analyzes the signal, and notifies the user that "the time for pollination is approaching." The user checks the notification and pollinates at the appropriate time. The results of this work are reported to the server as feedback, contributing to improving the accuracy of the next prediction.

[0923] Furthermore, if the server detects signals of damage caused by pests, it will send an alert to the device saying, "Insect damage has occurred. Urgent action is required." Based on this, the user can quickly take measures such as spraying pesticides. The results are then sent to the server as feedback.

[0924] The processing flow will be explained below.

[0925] Server Processing Steps

[0926] Step 1: Data collection

[0927] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances from plants, which reflect the physiological state and stress status of plants.

[0928] Step 2: Data Preprocessing

[0929] The server preprocesses the collected data, specifically detecting missing and outlier values ​​and imputing or removing them as necessary, a crucial step to ensure the reliability and accuracy of the data.

[0930] Step 3: Data analysis

[0931] The pre-processed data is fed into generative AI to estimate the plant's condition and needs, including predicting flowering time and risk of pest and disease damage.

[0932] Step 4: Send information

[0933] Based on the analysis results, the server creates a specific action plan and notifies the user, including specific instructions and recommendations for plant growth.

[0934] Step 5: Receiving feedback

[0935] The server receives feedback from users, which includes work results and observations, and is important for improving the analysis accuracy of the generative AI.

[0936] Terminal (user device) processing steps

[0937] Step 1: Receiving information

[0938] The terminal receives the prediction results and action plans sent from the server in real time.

[0939] Step 2: Display information

[0940] The device then displays the received information in an easy-to-understand format for the user. For example, if the flowering period is approaching, it will display specific pollination timing and procedures.

[0941] Step 3: Operation Guide

[0942] If necessary, provide the user with additional information and instructions to assist with specific operations.

[0943] User processing steps

[0944] Step 1: Verify the information

[0945] The user checks the information provided by the terminal and understands specific actions to take regarding plant growth.

[0946] Step 2: Take action

[0947] The user then takes appropriate action based on instructions from the device, such as manually pollinating the plants based on the notified pollination timing.

[0948] Step 3: Provide feedback

[0949] The user inputs the results of their execution and any changes they observe into their device and sends them to the server. This feedback information is used to improve the analytical accuracy of the generative AI.

[0950] Through the above steps, the server, terminal, and user can work together to efficiently and effectively manage plant growth.

[0951] Example 1

[0952] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0953] Accurately understanding the condition of plants and taking appropriate action in a timely manner is extremely important for efficient plant cultivation and pest prevention. However, current technology does not yet fully establish a system for monitoring plant conditions in real time and taking appropriate measures. In particular, there is room for improvement in the accuracy of data collection and analysis, the timing of user notifications, and how feedback is utilized.

[0954] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0955] In this invention, the server includes means for collecting ultrasound, microRNA, and volatile substances from plants, means for preprocessing the collected data, means for inputting the preprocessed data into a generative AI model to estimate the plant's condition, means for transmitting information about plant growth to a user's device based on the analysis results, means for receiving user feedback and improving the accuracy of the generative AI model, means for visually displaying information through a user interface and notifying the user in real time, and means for retraining the generative AI model based on the feedback. This makes it possible to monitor and analyze the plant's condition in real time and provide the user with necessary action information at the appropriate time.

[0956] "Means for collecting ultrasound, microRNA, and volatile substances from plants" is a general term for devices and sensors that detect ultrasound, microRNA, and volatile substances emitted by plants and obtain this information as data.

[0957] "Means for preprocessing collected data" refers to processes and algorithms for detecting, complementing, or removing missing or outlier values ​​in collected signal data from plants, thereby improving the reliability of the data and the accuracy of the analysis.

[0958] "Means of inputting preprocessed data into a generative AI model to estimate plant condition" refers to a system or algorithm that inputs data that has undergone preprocessing into a machine learning model and uses that data to estimate the current condition and future needs of a plant.

[0959] "Means for sending information about plant growth to the user's device based on the analysis results" refers to a notification system or communication protocol that notifies the user of the results of the generative AI model's analysis, allowing the user to take the necessary action at the appropriate time.

[0960] "Means for receiving user feedback and improving the accuracy of generative AI models" refers to systems and processes for collecting work results and observational information from users and using it as training data for generative AI models to improve the accuracy of future predictions and analyses.

[0961] "Means for visually displaying information through a user interface and providing notifications in real time" refers to an interface for displaying analysis results and action plans in a format that is easily understandable to users, and a system or software for notifying users of necessary information in real time.

[0962] "Means for retraining a generative AI model based on feedback" refers to a process or algorithm that uses feedback information obtained from users to retrain a generative AI model and improve the model's accuracy and reliability.

[0963] The system of the present invention includes a series of means for analyzing signals emitted by plants in real time and providing specific cultivation actions to the user based on the analysis results. The hardware and software used in this system are as follows.

[0964] Hardware

[0965] Ultrasonic sensor: A device that detects ultrasonic waves emitted by plants and collects that data.

[0966] MicroRNA detection device: A device that detects microRNAs in plants and collects the data.

[0967] Volatile substance detector: A sensor that detects volatile organic compounds emitted by plants.

[0968] User's smart device: The device that receives notifications and displays the user interface (e.g., smartphone, tablet).

[0969] software

[0970] Generative AI model: A machine learning model for predicting and analyzing plant conditions. Built using frameworks such as TensorFlow.

[0971] Data preprocessing program: A script using a programming language such as Python to clean the collected data and impute missing values.

[0972] Notification system: A system that uses services such as Firebase to send notifications to users' devices in real time.

[0973] Server Operation

[0974] The server uses various sensors and specialized equipment to collect ultrasonic, microRNA, and volatile chemical signals from plants in real time. The collected data is then stored in a database, and missing and outliers are detected and imputed or removed. The preprocessed data is then fed into a generative AI model, which takes into account past data and current environmental conditions to predict the plant's condition and necessary actions.

[0975] Device behavior

[0976] The device receives the analysis results sent from the server in real time and displays them visually through the user interface. For example, if the flowering period is approaching, the device will display the specific pollination timing and procedure, prompting the user to take appropriate action. The notification system uses Firebase and other services to quickly deliver the analysis results to the user's device.

[0977] User Actions

[0978] Based on the information provided by the device, the user plans and executes specific actions related to plant growth. For example, if the user receives a notification that "pollination time is approaching," the user prepares for pollination and carries it out at the appropriate time. The results of the actual work and any observed changes in the plant are entered into the device, and the feedback is sent to the server. This feedback is used to retrain the generative AI model, contributing to improving the accuracy of analysis in future generations.

[0979] Specific examples

[0980] For example, the server receives a signal from a sensor that "flowering is approaching," analyzes the signal, and notifies the user that "the time for pollination is approaching." The user checks this notification and performs pollination at the appropriate time. The results of this work are reported to the server as feedback, contributing to improved prediction accuracy for the next time. Furthermore, if the server detects a signal of damage caused by pests, it will send an alert to the device saying, "Insect damage has occurred. Urgent action is required." Based on this, the user can quickly take measures such as spraying pesticides, and send the results of this to the server as feedback.

[0981] Prompt Sentence Examples

[0982] You are designing a system that analyzes signals emitted by plants and provides information about their status to the user. These signals include ultrasound, microRNA, and volatile substances. Please specify the hardware and software used in this system and explain the specific data processing and data calculation methods.

[0983] These processes enable real-time monitoring and analysis of plant conditions, providing users with timely information on the actions they need to take.

[0984] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0985] Server Processing Steps

[0986] Step 1: Data collection

[0987] Input: Signal data from ultrasonic sensors, microRNA detectors, and volatile substance detectors.

[0988] How it works: The server receives real-time signal data about the plant's condition from these various sensors. The sensors record data every second and send it to the server. The server stores the received data in a database.

[0989] Output: The raw data collected.

[0990] Step 2: Data Preprocessing

[0991] Input: Raw data collected.

[0992] Specific operation: The server detects missing values ​​and outliers in the received data and processes them (complements or removes them). For complementation, it uses the most recent valid data, and for outliers, it corrects or removes values ​​that are out of range.

[0993] Output: Preprocessed clean data.

[0994] Step 3: Data analysis

[0995] Input: Preprocessed clean data.

[0996] Specific operation: The server inputs the preprocessed data into the generative AI model, which then predicts the plant's condition and growth based on past data and current environmental conditions.

[0997] Output: Analysis results (e.g., flowering time, pollination timing, risk of pests and diseases, etc.).

[0998] Step 4: Send information

[0999] Input: Analysis results.

[1000] Specific operation: Based on the analysis results of the generation AI, the server notifies the user of necessary action information (e.g., flowering time, pollination timing). This information is sent to the user's device via a notification service such as Firebase.

[1001] Output: Notifications sent to the user's device.

[1002] Step 5: Receiving feedback

[1003] Input: Feedback data from users.

[1004] Specific operation: Receives work results and observation data entered by the user into the device, stores this feedback data in a database, and creates a new learning set to retrain the generative AI model.

[1005] Output: Feedback data, improved generative AI model.

[1006] Terminal processing steps

[1007] Step 1: Receiving information

[1008] Input: Analysis results and action plans sent from the server.

[1009] What it does: The device maintains a connection with the server and periodically checks for new notifications. If any notifications are available, they are immediately downloaded and displayed to the user.

[1010] Output: Notification data waiting to be displayed.

[1011] Step 2: User Interface Display

[1012] Input: Notification data.

[1013] Specific operation: The device analyzes the received notification and displays it on the user interface. It provides instructions in visually appealing graphs and text, making it easy for the user to understand.

[1014] Output: A screen display for the user to see.

[1015] User processing steps

[1016] Step 1: Verify the information

[1017] Input: Notifications from your device.

[1018] Specific behavior: The user taps the notification on their device to view more information and plan appropriate actions based on the information displayed.

[1019] Output: A confirmed action plan.

[1020] Step 2: Take action

[1021] Input: Confirmed action plan.

[1022] Specific actions: The user performs the necessary work on site based on the notification. For example, if pollination is required, prepare pollination tools and perform pollination at the appropriate time.

[1023] Output: The training work carried out.

[1024] Step 3: Provide feedback

[1025] Input: Development activities performed and observation data.

[1026] Specific operation: The user inputs the results of their work and observed changes into the terminal and sends them to the server, which improves the analysis accuracy of the entire system.

[1027] Output: Feedback data sent to the server.

[1028] (Application example 1)

[1029] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1030] Conventional plant growth management systems are required to not only grasp the state of plants, but also to formulate and execute appropriate action plans, but there are many challenges to achieving this. Specifically, they require accurate data collection and analysis to accurately analyze signals from plants and provide optimal growth conditions, as well as the autonomy to take appropriate actions based on that data. Conventional methods lack sufficient analytical accuracy and automation of actions, making effective plant growth management difficult.

[1031] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1032] In this invention, the server includes means for collecting ultrasound, microRNA, and volatile substances from plants, means for preprocessing the collected data, means for analyzing the preprocessed data and using a generating AI to estimate the state of the plant, means for sending information about plant growth to a user based on the analysis results, means for receiving feedback from the user and improving the accuracy of the generating AI, and means for controlling a factory robot that executes an action plan required for plant growth based on the analysis results. This realizes a system that can perform everything from data analysis to action execution in an integrated manner, enabling optimal plant growth management.

[1033] "Ultrasound" refers to sound waves with a frequency of 20 kilohertz or higher, which contain unique signals emitted by plants.

[1034] "MicroRNAs" are small RNA molecules that regulate gene expression in plants and are important biomarkers that reflect the health and stress status of plants.

[1035] "Volatiles" are volatile compounds emitted by plants that indicate the plant's condition or environmental stress.

[1036] "Generative AI" is an artificial intelligence model that estimates the condition of plants based on collected data and suggests optimal growth conditions and action plans.

[1037] An "action plan" is a plan that includes specific actions and procedures required for the growth and protection of plants, based on the analysis results of the generative AI.

[1038] A "factory robot" is a mechanical device that operates autonomously or semi-autonomously within a factory and executes action plans related to plant growth management.

[1039] "Data preprocessing" is the process of detecting missing or outliers in collected plant data and imputing or removing them appropriately.

[1040] "Feedback" is information provided by the user about execution results and observed changes, which is used to improve the accuracy of the generative AI.

[1041] This invention is a system for understanding the state of plants in real time and implementing appropriate cultivation actions based on that information. The main components of the system are a server, terminals, and factory robots.

[1042] server

[1043] The server includes the following means:

[1044] 1. Data collection: The server collects data about the plants in real time from various sensors, such as ultrasonic sensors, microRNA analyzers, and volatile substance detectors.

[1045] 2. Data preprocessing: Detect missing or outliers in the collected data, and impute or remove them as necessary to improve the reliability of the data and the accuracy of the analysis.

[1046] 3. Generative AI analysis: The preprocessed data is input into the generative AI, which takes into account past data and environmental conditions to estimate the plant's condition and necessary growing conditions.

[1047] 4. Information transmission: Based on the analysis results, specific information such as flowering time, pollination timing, and fruit thinning timing is notified to the user. In addition, an action plan generated based on the analysis results is transmitted to the factory robot.

[1048] 5. Feedback reception: Receive feedback from the user on the actions taken to improve the analytical accuracy of the generative AI.

[1049] Terminal (user's device)

[1050] The terminal includes the following means:

[1051] 1. Information reception: Receive analysis results and action plans sent from the server in real time.

[1052] 2. User interface: The received information is displayed to the user in an easy-to-understand format, showing the notified pollination timing and procedures.

[1053] Factory robots

[1054] The factory robot includes the following means:

[1055] 1. Action plan execution: According to the action plan sent from the server, appropriate cultivation tasks are carried out automatically or semi-automatically, such as watering, fertilizing, and pest control.

[1056] Overall processing flow

[1057] The server first collects data about the plants from various sensors. The collected data is preprocessed, and missing or outliers are filled in or removed. The preprocessed data is analyzed by the generative AI, which estimates the plant's condition and the conditions necessary for its growth. The analysis results are notified to the user's device, and an appropriate action plan is sent to the factory robot. The factory robot carries out the growth work based on this action plan. The user provides feedback on actual work results and observations, which are sent to the server and used to improve the accuracy of the generative AI.

[1058] Specific examples

[1059] For example, the server collects ultrasonic data, microRNA data, and volatile substance data from a plant and analyzes it to determine that the plant is nearing flowering. As a result, the user's device is notified that "the time for pollination is approaching." At the same time, an action plan for carrying out "pollination" is sent to the factory robot. The user checks the notification, and the robot carries out pollination at the appropriate time. The results are then reported to the server as feedback.

[1060] Example prompts for generative AI models

[1061] Enter your plant's ultrasound, microRNA, and volatile data, and we'll estimate your next steps and suggest an appropriate course of action.

[1062] Example data:

[1063] Ultrasonic: 10kHz

[1064] MicroRNA: [RNA_sequence_A, RNA_sequence_B]

[1065] Volatile substances: [Compound A, Compound B]

[1066] In this way, the entire system works together to achieve optimal plant growth.

[1067] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1068] Step 1:

[1069] Data collection

[1070] The server collects real-time plant data from sensors such as ultrasonic sensors, microRNA analyzers, and volatile substance detectors. In this step, raw data from each sensor is input and output in its raw format.

[1071] Step 2:

[1072] Data Preprocessing

[1073] The server detects missing or outliers in the collected data and imputes or removes them as necessary. Here, the raw data collected in the previous step is used as input, and reliable data is output with outliers replaced with default values ​​such as "default_value." Specific operations include checking the consistency of the data and filtering outliers.

[1074] Step 3:

[1075] Generative AI analysis

[1076] The server inputs the preprocessed data and uses generative AI to estimate the plant's condition. The output is an analysis result that clarifies the plant's current condition and necessary growing conditions. Input data to the generative AI model includes values ​​of ultrasound, microRNA, and volatile substances.

[1077] Step 4:

[1078] Information transmission

[1079] Based on the analysis results of the generated AI, the server sends information about plant growth to the user's device. It also sends an action plan to the factory robot. The analysis results are used as input, and the output is instructions and action plans received by the device and robot. The specific operation involves data communication via a network.

[1080] Step 5:

[1081] Information reception (terminal)

[1082] The terminal receives analysis results and action plans sent from the server in real time. The input is the analysis results and action plans from the server, and the output is display information converted into a format that can be viewed by the user. Specific operations include analyzing the received data and displaying it on the user interface.

[1083] Step 6:

[1084] Action plan execution (factory robot)

[1085] Factory robots automatically or semi-automatically perform appropriate cultivation tasks based on an action plan sent from a server. The action plan from the server is given as input, and the output is the actual physical work, such as watering, fertilizing, and pest control. Specific operations include robot motion control and the use of sensors.

[1086] Step 7:

[1087] Feedback provided by users

[1088] The user inputs feedback about the tasks performed and the changes observed into the terminal, which then sends it to the server. The input is feedback data from the user, and the output is data used to update the generative AI. Specific operations include data input via the user interface and feedback transmission via the network.

[1089] Step 8:

[1090] Feedback reception (server)

[1091] The server receives feedback from users and uses it to improve the accuracy of the generative AI. The input is the user feedback data, and the output is an updated, more accurate generative AI model. Specific operations include analyzing the feedback data and training the generative AI model.

[1092] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1093] The present invention is a system that analyzes signals emitted by plants, such as ultrasound, microRNA, and volatile substances, to estimate the plant's condition and notify the user of appropriate actions based on the results. This system incorporates an emotion engine that recognizes the user's emotions, and also has the function of adjusting the content and timing of notifications based on the user's emotions and making suggestions to reduce the user's stress level. Specific processing of the program is described below.

[1094] server

[1095] 1. Data Collection:

[1096] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances from plants, which reflect the physiological state and stress status of plants.

[1097] 2. Data Preprocessing:

[1098] The server preprocesses the collected data, specifically detecting missing and outlier values ​​and imputing or removing them as necessary, which is an important step to ensure the reliability and accuracy of the data.

[1099] 3. Data Analysis:

[1100] The pre-processed data is fed into generative AI to estimate the plant's condition and needs, including predicting flowering time and risk of pest and disease damage.

[1101] 4. Emotion analysis:

[1102] The server uses an emotion engine to analyze the user's emotional data sent from the device, thereby understanding the user's current emotional state and stress level.

[1103] 5. Information Transmission:

[1104] Based on the analysis results, the server creates a specific action plan, adjusting the content and timing of notifications according to the user's emotional state to notify the user in the most optimal way.

[1105] 6. Feedback received:

[1106] The server receives feedback from users to improve the accuracy of the generative AI. The feedback information is the results of work and observations, and is useful for future analysis.

[1107] Terminal (user's device)

[1108] 1. Receiving information:

[1109] The terminal receives the prediction results and action plans sent from the server in real time.

[1110] 2. Emotional Data Collection:

[1111] The device uses a camera, microphone, and other sensors to collect information such as the user's facial expressions, voice, and heart rate, and transmits this information to a server as emotional data.

[1112] 3. Information display:

[1113] The device then displays the received information in an easy-to-understand format for the user. For example, if the flowering period is approaching, it will display specific pollination timing and procedures.

[1114] 4. Operation Guide:

[1115] If necessary, provide the user with additional information and instructions to assist with specific operations.

[1116] User

[1117] 1. Information verification:

[1118] The user checks the information provided by the terminal and understands specific actions to take regarding plant growth.

[1119] 2. Take action:

[1120] The user then takes appropriate action based on instructions from the device, such as manually pollinating the plants based on the notified pollination timing.

[1121] 3. Record your emotions:

[1122] Users record their emotions and stress levels on their devices and send them to a server, where the emotional data is used for analysis and action optimization.

[1123] 4. Providing Feedback:

[1124] The user inputs the results of their actions and any changes they observe into their device and sends them to the server. This feedback information is used to improve the analytical accuracy of the generative AI.

[1125] Specific examples

[1126] For example, if the server receives a signal from a sensor that "flowering is approaching" and analyzes it, it notifies the user that "the time for pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification will be sent as is, but if it detects that the user is in a high stress state, the message will be adjusted to include a warning such as "The important time for pollination is approaching, but please proceed with the work without overdoing it."

[1127] Furthermore, if the server detects signals of damage caused by pests, it sends an alert to the device saying, "Insect damage has occurred. Immediate action is required." If the user feels high stress in response to this alert, the emotion engine sends a message emphasizing how to deal with the situation, such as, "Insect damage has occurred. Please calmly take action by following the procedures." Based on this, the user can quickly take action, such as spraying pesticides. The results are then sent to the server as feedback.

[1128] In this way, the server, terminals, and users can work together to efficiently and effectively manage plant cultivation. In addition, the combination of the emotion engine reduces the psychological burden on users and improves work efficiency.

[1129] The processing flow will be explained below.

[1130] Server Processing Steps

[1131] Step 1: Data collection

[1132] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances, to detect the physiological and stress states of plants.

[1133] Step 2: Data Preprocessing

[1134] The server detects whether there are any missing or outliers in the collected data and performs imputation or removal as necessary, thereby improving the reliability of the data and the accuracy of the analysis.

[1135] Step 3: Data analysis

[1136] The preprocessed data is input into a generative AI that estimates the plant's condition and needs, and the generative AI predicts flowering time and pest and disease risk based on past data and environmental conditions.

[1137] Step 4: Sentiment Analysis

[1138] The server uses an emotion engine to analyze the user's emotional data received from the device, thereby understanding the user's current emotional state and stress level.

[1139] Step 5: Send information

[1140] Based on the analysis results, the server creates a specific action plan, adjusting the content and timing of notifications according to the user's emotional state to notify the user in the most optimal way.

[1141] Step 6: Receiving feedback

[1142] The server receives feedback from users, including the results of tasks performed and observations, and uses this feedback to improve the analytical accuracy of the generative AI.

[1143] Terminal processing steps

[1144] Step 1: Receiving information

[1145] The terminal receives the analysis results and action plans sent from the server in real time.

[1146] Step 2: Collecting Emotional Data

[1147] The device uses a camera, microphone, and other sensors to collect emotional data such as the user's facial expressions, voice, and heart rate, and sends this data to a server.

[1148] Step 3: Display information

[1149] The device then displays the received information in an easy-to-understand format for the user. For example, if the flowering period is approaching, it will display specific pollination timing and procedures.

[1150] User processing steps

[1151] Step 1: Verify the information

[1152] The user checks the information provided by the terminal and understands specific actions to take regarding plant growth.

[1153] Step 2: Take action

[1154] The user can then take appropriate action based on instructions from the device, such as manually pollinating the plants based on the notified pollination timing.

[1155] Step 3: Record your emotions

[1156] Users record their emotions and stress levels on their devices and send them to a server, where the emotional data is used for analysis and action optimization.

[1157] Step 4: Provide feedback

[1158] The user inputs the results of their execution and any observed changes into their device and sends them to the server. The feedback information is used to improve the analysis accuracy of the generative AI.

[1159] Specific examples

[1160] For example, if the server receives a signal from a sensor that "flowering is approaching" and analyzes it, it notifies the user that "the time for pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification will be sent as is, but if it detects that the user is in a high stress state, the message will be adjusted to include a warning such as "The important time for pollination is approaching, but please proceed with the work without overdoing it."

[1161] When the server detects a signal of pest damage, it sends an alert to the device saying, "Insect damage has occurred. Immediate action is required." If the user feels high stress in response to this alert, the emotion engine sends a message emphasizing how to deal with the situation, such as, "Insect damage has occurred, but please calmly follow the procedures and take action." Based on this, the user can quickly take action, such as spraying pesticides, and send the results of that action to the server as feedback.

[1162] In this way, the server, terminals, and users can work together to efficiently and effectively manage plant cultivation. In addition, the combination of the emotion engine reduces the psychological burden on users and improves work efficiency.

[1163] Example 2

[1164] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1165] Conventional agricultural systems have had difficulty accurately understanding the physiological state and stress levels of plants and taking appropriate measures based on that information. Furthermore, they do not provide information that takes into account the user's emotional state, making it impossible to reduce the user's psychological burden. Furthermore, continuous system optimization based on feedback is insufficient.

[1166] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting ultrasound, microRNA, and volatile substances from plants, means for preprocessing the collected data, means for analyzing the preprocessed data and using a generation AI to estimate the plant's condition, means for analyzing user emotion data, means for sending information about plant growth to the user based on the analysis results and the emotion data, and means for receiving feedback from the user and improving the accuracy of the generation AI. This makes it possible to accurately grasp the plant's condition and take appropriate measures based on that understanding, taking the user's emotional state into consideration. Furthermore, the system can be continuously optimized by reflecting user feedback.

[1167] "Ultrasound" is a type of sound wave that usually has a frequency of 20 kHz or higher and is inaudible to the human ear.

[1168] "MicroRNA" is a small RNA molecule that regulates gene expression and has the function of suppressing the production of specific proteins.

[1169] "Volatile substances" are substances that easily evaporate into gas at room temperature and serve as indicators of the physiological state and stress of plants.

[1170] "Preprocessing" refers to a series of processes that prepare collected data in a form suitable for analysis, and includes operations such as filling in and removing missing or outlier values ​​in the data.

[1171] "Generative AI" is an artificial intelligence technology used to analyze collected data and predict specific situations or conditions.

[1172] "Emotion data" is data including the user's facial expression, voice, heart rate, etc., and indicates the user's current emotional state and stress level.

[1173] "Feedback" is information that records the results of operations performed by the user and observed changes and returns them to the system.

[1174] "Information regarding plant cultivation" includes information regarding the flowering period of plants, timing of pollination, fruit thinning, harvesting time, proper use of pesticides, and avoidance of insect damage.

[1175] "Missing data" refers to values ​​that are not recorded because measurements or collection were not performed correctly.

[1176] An "outlier" is a data value that deviates from the normal range and may affect the analysis results.

[1177] The "emotional state of the user" indicates the state of emotions or moods that the user is feeling, and specifically includes joy, sadness, surprise, anger, fear, and the like.

[1178] The "stress level" indicates the degree of mental tension or pressure that the user feels.

[1179] The "emotion engine" is software that analyzes a user's emotional data and evaluates their current emotional state and stress level.

[1180] MODE FOR CARRYING OUT THE INVENTION

[1181] This invention is a system that analyzes signals emitted by plants, such as ultrasonic waves, microRNA, and volatile substances, to estimate the plant's condition and notify the user of appropriate actions based on the results. This system incorporates an emotion engine that recognizes the user's emotions, and also has the function of adjusting the content and timing of notifications based on the user's emotions and making suggestions to reduce the user's stress level.

[1182] Hardware and software used

[1183] server:

[1184] Hardware: High-performance server (e.g., server with Intel Xeon processor)

[1185] Software: Python, TensorFlow, Pandas, Numpy, IBM Watson Sentiment Analysis API, Microsoft Azure

[1186] Terminal (user's device):

[1187] Hardware: Smartphone (Android or iOS), wearable device (smartwatch with heart rate monitor)

[1188] Software: Dedicated application (Android / iOS app), Bluetooth module

[1189] Data collection

[1190] The server collects real-time signal data of ultrasonic waves, microRNAs, and volatile substances from multiple sensors (e.g., ultrasonic sensors, RNA sequencers, and gas chromatographs) in plants. These signal data reflect the physiological state and stress status of plants.

[1191] Data Preprocessing

[1192] The server preprocesses the collected data using the Python Pandas library to detect missing and outliers, and impute or remove them as needed. Data smoothing and normalization are also performed at this stage.

[1193] Data analysis

[1194] The preprocessed data is then fed into a generative AI model, a deep learning model built using TensorFlow, which analyzes the plant's condition, predicting flowering time and assessing pest and disease risk, and creating a specific action plan.

[1195] Emotion analysis

[1196] The server analyzes the user's emotional data sent from the device using IBM Watson's emotion analysis API, which allows it to understand the user's current emotional state and stress level. Emotional data is collected from the device's camera, microphone, and wearable device.

[1197] Information transmission

[1198] Based on the analysis results and emotional data, the server notifies the user of the appropriate action: for users in a low-stress state, it sends a simple notification such as "The time for pollination is approaching," and for users in a high-stress state, it sends a notification with a warning such as "Please proceed with the pollination work without overdoing it."

[1199] Receive feedback

[1200] The server receives feedback from users, who send the results of their work and observed changes as feedback to the server through a dedicated application. This feedback information is recorded in a database and used to improve the accuracy of the generative AI model.

[1201] Specific examples

[1202] For example, if a plant is about to flower, the server analyzes the data from the sensor and notifies the user that "pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification is sent as is, but if the user is in a high stress state, an adjusted message is sent saying, "The important pollination time is approaching, but please proceed with your work without overdoing it."

[1203] When signals of pest damage are detected, the server sends an alert to the device saying, "Insect damage has occurred. Immediate action is required." If the user feels high stress in response to this alert, the emotion engine sends a message emphasizing countermeasures, such as, "Insect damage has occurred, but please calmly follow the procedures and take action." Based on this, the user can quickly take measures such as spraying pesticides.

[1204] In this way, the server, terminals, and users can work together to efficiently and effectively manage plant cultivation. Furthermore, by utilizing the emotion engine, the psychological burden on users can be reduced and work efficiency can be improved.

[1205] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1206] Program processing flow

[1207] Step 1: Data collection

[1208] How it works: The server collects data in real time from multiple sensors (ultrasonic sensors, RNA sequencers, and gas chromatographs) attached to plants, including the sounds of plant growth (ultrasonic waves), microRNAs (gene expression status), and volatile substances (vaporized chemical components).

[1209] Input: Signal data from the sensor.

[1210] Output: Raw, unprocessed data.

[1211] Step 2: Data Preprocessing

[1212] Specific operation: The server preprocesses the collected raw data. At this stage, it creates a data frame using Python's Pandas library, detects and imputes missing values ​​and outliers, and normalizes the data. Specifically, it performs line imputation, outlier removal, and smoothing.

[1213] Input: Raw, unprocessed data.

[1214] Output: A formatted dataset.

[1215] Step 3: Data analysis

[1216] How it works: The server inputs the preprocessed data into a generative AI model (e.g., a deep learning model using TensorFlow) to analyze the plant's condition and needs. This analysis includes predicting flowering time, assessing pest and disease risk, and generating a specific action plan.

[1217] Input: The formatted dataset.

[1218] Output: Plant status, risk assessment and action plan generated.

[1219] Step 4: Collect emotional data

[1220] Specific operation: The device uses sensors such as a camera, microphone, and smartwatch with a heart rate monitor to collect the user's facial expressions, voice, and heart rate, and transmits them to the server, thereby obtaining the user's emotional data.

[1221] Input: User biometric information (facial expression, voice, heart rate).

[1222] Output: User emotion data.

[1223] Step 5: Sentiment Analysis

[1224] Specific operation: The server analyzes the emotion data received from the device using an emotion engine (e.g., IBM Watson's emotion analysis API), thereby evaluating the user's emotional state and stress level.

[1225] Input: User emotion data.

[1226] Output: Evaluation of emotional state and stress level.

[1227] Step 6: Send information

[1228] Specific operation: Based on the analysis results and emotional data, the server notifies the user of appropriate actions. Notifications are sent via email or push notification, and the content is adjusted according to the user's emotional state. Users in a low-stress state will receive a simple notification such as "The time for pollination is approaching," while users in a high-stress state will receive a notification with a warning such as "Please proceed with the pollination work without overdoing it."

[1229] Input: Analysis results, emotional state and stress level.

[1230] Output: The adjusted notification content.

[1231] Step 7: Receive and display information

[1232] Specific operation: The device receives the notification sent from the server and displays it in a dedicated application. The displayed content includes specific response procedures and timing.

[1233] Input: Notification from the server.

[1234] Output: The action plan that is displayed to the user.

[1235] Step 8: Take Action

[1236] Specific operation: The user checks the information provided by the device and performs actual farm work, such as manually pollinating crops based on the notified pollination timing.

[1237] Input: Information provided by the device.

[1238] Output: The action that was performed.

[1239] Step 9: Provide feedback

[1240] How it works: The user inputs the results of their actions and observed changes into the device and sends them to the server. This feedback is stored in a database and used to improve the accuracy of the generative AI model.

[1241] Input: Execution results, observed changes.

[1242] Output: Feedback data.

[1243] Through this series of processing steps, the system can accurately grasp the plant's growth status, notify the user of appropriate actions, and reduce the user's psychological burden by taking into account the user's emotional state. Furthermore, the system can continuously optimize the entire system by utilizing feedback.

[1244] (Application example 2)

[1245] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1246] Currently, there are systems that closely monitor the condition of plants and provide advice on their cultivation, but there are no notification systems that take into account the user's emotions and stress level. This increases the psychological burden on the user, resulting in inefficient plant cultivation management. The present invention aims to solve these problems.

[1247] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting ultrasound, microRNA, and volatile substances from plants; means for preprocessing the collected data; means for analyzing the preprocessed data and using a generation AI to estimate the plant's condition; means for sending information about plant growth to the user based on the analysis results; means including an emotion engine that recognizes the user's emotions and adjusts the content and timing of notifications; and means for receiving feedback from the user and improving the accuracy of the generation AI. This enables efficient plant growth management while reducing the user's psychological burden by suggesting appropriate actions according to the plant's condition and providing notifications that take the user's emotional state into consideration.

[1248] "Ultrasound" is a sound wave with a vibration frequency of 20,000 Hz or higher, which is inaudible to the human ear but is used to detect the internal structure of materials.

[1249] "MicroRNA" is a short non-coding RNA molecule approximately 22 bases in length that functions to regulate gene expression.

[1250] "Volatiles" are substances that evaporate readily at room temperature and may be released by plants to indicate stress or other physiological conditions.

[1251] "Preprocessing" refers to the process of removing missing values ​​and outliers from collected data and preparing it for analysis.

[1252] "Generative AI" is a type of artificial intelligence technology that has the ability to generate new information and predictions through data analysis.

[1253] The "emotion engine" is a system that recognizes the user's emotions and stress level, and adjusts the content and timing of notifications based on that information.

[1254] "Notification content" is information that notifies the user of the analysis results and specific action plans based on those results.

[1255] "Feedback" is information provided by the user about the execution results and their evaluation, which helps improve the accuracy of the system.

[1256] "Plant condition" refers to comprehensive information about the plant's health, growth process, stress status, etc.

[1257] The system embodying this invention collects ultrasonic waves, microRNAs, and volatile substances from plants, preprocesses and analyzes them, and notifies the user of appropriate actions based on the results. It also incorporates an emotion engine that recognizes the user's emotions and adjusts the content and timing of notifications.

[1258] System hardware configuration

[1259] The system hardware is as follows:

[1260] Plant signal sensors: using ultrasonic sensors, microRNA sensors, and volatile substance sensors.

[1261] User emotion recognition devices: Use cameras, microphones, heart rate sensors, etc.

[1262] Devices: Smartphones, tablets, head-mounted displays, etc.

[1263] System software configuration

[1264] The main software used is as follows:

[1265] Generative AI models: perform data analysis and predictions.

[1266] Emotion recognition engine: Analyzes user emotional data and adjusts notification content.

[1267] Data preprocessing module: Imputes and removes missing and outlier values.

[1268] How the system is implemented

[1269] 1. Data Collection:

[1270] The server collects real-time data from various sensors attached to the plants, including ultrasonic waves, microRNAs, and volatile compounds, which reflect the physiological and stress states of the plants.

[1271] 2. Data Preprocessing:

[1272] The server detects missing or outliers in the collected data and performs imputation or removal as necessary, thereby ensuring the reliability and accuracy of the data.

[1273] 3. Data Analysis:

[1274] The pre-processed data is fed into a generative AI model to estimate the plant's condition and needs, such as predicting flowering time and risk of pest and disease damage.

[1275] 4. Emotion recognition:

[1276] The device uses a camera, microphone, and heart rate sensor to collect user emotional data and transmit it to a server, which then uses an emotion engine to analyze the data and determine the user's current emotional state and stress level.

[1277] 5. Information Transmission:

[1278] The server creates a specific action plan based on the analysis results and adjusts the content and timing of notifications according to the user's emotional state. For example, it may notify the user that pollination is approaching, or send a message containing a warning if the user is under high stress.

[1279] 6. Feedback received:

[1280] The server receives feedback from users and uses it to improve the accuracy of the generative AI. The feedback information is the results of the tasks performed and observations.

[1281] Specific examples

[1282] For example, if the server receives a signal from a sensor that "flowering is approaching" and analyzes it, it notifies the user that "the time for pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification will be sent as is, but if it detects that the user is in a high stress state, the message will be adjusted to include a warning that "the important time for pollination is approaching, but please proceed with the work without overdoing it."

[1283] Prompt Sentence Examples

[1284] "Sensor URL: http: / / api.plant_sensors.com / data"

[1285] "User data: including user emotional data"

[1286] This system allows for efficient plant growth management while reducing the psychological burden on users by suggesting appropriate actions based on the condition of the plant and providing notifications that take into account the user's emotional state.

[1287] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1288] Step 1:

[1289] Data collection

[1290] The server collects ultrasonic, microRNA, and volatile data in real time from sensors attached to plants. The input is the signal data from the sensors, and the output is the collected raw data. This data reflects the physiological state and stress status of the plants.

[1291] Step 2:

[1292] Data Preprocessing

[1293] The server detects missing values ​​and outliers in the collected data and imputes or removes them as necessary. The input is the collected raw data, and after passing through the data cleaning and imputation process, it outputs more accurate and reliable clean data.

[1294] Step 3:

[1295] Data analysis

[1296] The server inputs the preprocessed data into a generative AI model to estimate the plant's condition and needs. The input is preprocessed clean data, and the output is the plant's condition (e.g., flowering time, risk of pests, etc.). Specifically, the AI ​​model analyzes the data and makes predictions and diagnoses.

[1297] Step 4:

[1298] Emotion recognition data collection

[1299] The device uses a camera, microphone, and heart rate sensor to collect emotional data such as the user's facial expressions, voice, and heart rate. The input is sensory data from the camera and microphone, and the output is the user's emotional data (e.g., stress level, emotional state).

[1300] Step 5:

[1301] Emotion analysis

[1302] The server analyzes the user's emotional data sent from the device to understand the user's current emotional state and stress level. The input is the collected user's emotional data, and the output is the analyzed emotional state. The emotion engine runs and analyzes this.

[1303] Step 6:

[1304] Information transmission and notification coordination

[1305] The server creates a specific action plan based on the analysis results and adjusts the content and timing of notifications according to the user's emotional state. The input is the plant's condition and the user's emotional state, and the output is an adjusted notification message. Notifications are given taking into account the user's psychological state, such as warnings about pollination timing or insect damage.

[1306] Step 7:

[1307] Receive feedback

[1308] The server receives feedback provided by users (e.g., the results of actual work performed or observation data) and improves the accuracy of the generative AI model. The input is the feedback information from the user, and the output is an improved AI model. The feedback is analyzed and the system performance is improved.

[1309] This step not only suggests appropriate actions based on the condition of the plant, but also reduces the user's psychological burden by providing notifications that take into account the user's emotional state, thereby enabling efficient plant growth management.

[1310] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1311] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1312] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1313] [Fourth embodiment]

[1314] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1315] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1316] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1317] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1318] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1319] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1320] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1321] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1322] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1323] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1324] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1325] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1326] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1327] The system of the present invention analyzes signals emitted by plants, such as ultrasonic waves, microRNA, and volatile substances, estimates the plant's condition, and based on that estimates, suggests appropriate actions to the user. Specific processing of the program for implementing this system is explained below in natural language.

[1328] server

[1329] 1. Data Collection:

[1330] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances from plants. These signals are important data that indicate the timing of plant flowering and pollination, as well as damage caused by pests and diseases.

[1331] 2. Data Preprocessing:

[1332] The server detects missing or outliers in the collected data and performs imputation or removal as necessary. Preprocessing improves the reliability of the data and the accuracy of analysis.

[1333] 3. Data Analysis:

[1334] The server inputs the preprocessed data into the generative AI, which estimates the plant's condition and needs. The generative AI takes into account past data and environmental conditions to predict the optimal growing conditions for the plant.

[1335] 4. Information Transmission:

[1336] Based on the analysis results, the server notifies the user of specific information such as flowering time, pollination timing, and fruit thinning timing, allowing the user to carry out tasks at the appropriate time.

[1337] 5. Feedback received:

[1338] The server receives feedback from users to improve the accuracy of the generative AI, including the results of their work and observations.

[1339] Terminal (user's device)

[1340] 1. Receiving information:

[1341] The device receives analysis results and action plans sent from the server in real time, allowing users to understand the condition and needs of their plants at any given time.

[1342] 2. User Interface:

[1343] The device then displays the received information in an easy-to-understand format. For example, if the flowering period is approaching, it will display the specific pollination timing and procedure, and encourage the user to take appropriate action.

[1344] User

[1345] 1. Information verification:

[1346] Based on the information provided by the device, users can check specific actions for plant growth, allowing them to take necessary action before it is too late.

[1347] 2. Take action:

[1348] Based on the received information, the user can take appropriate action on-site, such as manually pollinating the crops based on the notified pollination timing.

[1349] 3. Providing Feedback:

[1350] The user inputs the results of their execution and any changes they observe into their device and sends them to the server. This feedback information is used to improve the analysis accuracy of the generative AI.

[1351] Specific examples

[1352] For example, the server receives a signal from a sensor indicating that "flowering is approaching," analyzes the signal, and notifies the user that "the time for pollination is approaching." The user checks the notification and pollinates at the appropriate time. The results of this work are reported to the server as feedback, contributing to improving the accuracy of the next prediction.

[1353] Furthermore, if the server detects signals of damage caused by pests, it will send an alert to the device saying, "Insect damage has occurred. Urgent action is required." Based on this, the user can quickly take measures such as spraying pesticides. The results are then sent to the server as feedback.

[1354] The processing flow will be explained below.

[1355] Server Processing Steps

[1356] Step 1: Data collection

[1357] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances from plants, which reflect the physiological state and stress status of plants.

[1358] Step 2: Data Preprocessing

[1359] The server preprocesses the collected data, specifically detecting missing and outlier values ​​and imputing or removing them as necessary, a crucial step to ensure the reliability and accuracy of the data.

[1360] Step 3: Data analysis

[1361] The pre-processed data is fed into generative AI to estimate the plant's condition and needs, including predicting flowering time and risk of pest and disease damage.

[1362] Step 4: Send information

[1363] Based on the analysis results, the server creates a specific action plan and notifies the user, including specific instructions and recommendations for plant growth.

[1364] Step 5: Receiving feedback

[1365] The server receives feedback from users, which includes work results and observations, and is important for improving the analysis accuracy of the generative AI.

[1366] Terminal (user device) processing steps

[1367] Step 1: Receiving information

[1368] The terminal receives the prediction results and action plans sent from the server in real time.

[1369] Step 2: Display information

[1370] The device then displays the received information in an easy-to-understand format for the user. For example, if the flowering period is approaching, it will display specific pollination timing and procedures.

[1371] Step 3: Operation Guide

[1372] If necessary, provide the user with additional information and instructions to assist with specific operations.

[1373] User processing steps

[1374] Step 1: Verify the information

[1375] The user checks the information provided by the terminal and understands specific actions to take regarding plant growth.

[1376] Step 2: Take action

[1377] The user then takes appropriate action based on instructions from the device, such as manually pollinating the plants based on the notified pollination timing.

[1378] Step 3: Provide feedback

[1379] The user inputs the results of their execution and any changes they observe into their device and sends them to the server. This feedback information is used to improve the analytical accuracy of the generative AI.

[1380] Through the above steps, the server, terminal, and user can work together to efficiently and effectively manage plant growth.

[1381] Example 1

[1382] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1383] Accurately understanding the condition of plants and taking appropriate action in a timely manner is extremely important for efficient plant cultivation and pest prevention. However, current technology does not yet fully establish a system for monitoring plant conditions in real time and taking appropriate measures. In particular, there is room for improvement in the accuracy of data collection and analysis, the timing of user notifications, and how feedback is utilized.

[1384] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1385] In this invention, the server includes means for collecting ultrasound, microRNA, and volatile substances from plants, means for preprocessing the collected data, means for inputting the preprocessed data into a generative AI model to estimate the plant's condition, means for transmitting information about plant growth to a user's device based on the analysis results, means for receiving user feedback and improving the accuracy of the generative AI model, means for visually displaying information through a user interface and notifying the user in real time, and means for retraining the generative AI model based on the feedback. This makes it possible to monitor and analyze the plant's condition in real time and provide the user with necessary action information at the appropriate time.

[1386] "Means for collecting ultrasound, microRNA, and volatile substances from plants" is a general term for devices and sensors that detect ultrasound, microRNA, and volatile substances emitted by plants and obtain this information as data.

[1387] "Means for preprocessing collected data" refers to processes and algorithms for detecting, complementing, or removing missing or outlier values ​​in collected signal data from plants, thereby improving the reliability of the data and the accuracy of the analysis.

[1388] "Means of inputting preprocessed data into a generative AI model to estimate plant condition" refers to a system or algorithm that inputs data that has undergone preprocessing into a machine learning model and uses that data to estimate the current condition and future needs of a plant.

[1389] "Means for sending information about plant growth to the user's device based on the analysis results" refers to a notification system or communication protocol that notifies the user of the results of the generative AI model's analysis, allowing the user to take the necessary action at the appropriate time.

[1390] "Means for receiving user feedback and improving the accuracy of generative AI models" refers to systems and processes for collecting work results and observational information from users and using it as training data for generative AI models to improve the accuracy of future predictions and analyses.

[1391] "Means for visually displaying information through a user interface and providing notifications in real time" refers to an interface for displaying analysis results and action plans in a format that is easily understandable to users, and a system or software for notifying users of necessary information in real time.

[1392] "Means for retraining a generative AI model based on feedback" refers to a process or algorithm that uses feedback information obtained from users to retrain a generative AI model and improve the model's accuracy and reliability.

[1393] The system of the present invention includes a series of means for analyzing signals emitted by plants in real time and providing specific cultivation actions to the user based on the analysis results. The hardware and software used in this system are as follows.

[1394] Hardware

[1395] Ultrasonic sensor: A device that detects ultrasonic waves emitted by plants and collects that data.

[1396] MicroRNA detection device: A device that detects microRNAs in plants and collects the data.

[1397] Volatile substance detector: A sensor that detects volatile organic compounds emitted by plants.

[1398] User's smart device: The device that receives notifications and displays the user interface (e.g., smartphone, tablet).

[1399] software

[1400] Generative AI model: A machine learning model for predicting and analyzing plant conditions. Built using frameworks such as TensorFlow.

[1401] Data preprocessing program: A script using a programming language such as Python to clean the collected data and impute missing values.

[1402] Notification system: A system that uses services such as Firebase to send notifications to users' devices in real time.

[1403] Server Operation

[1404] The server uses various sensors and specialized equipment to collect ultrasonic, microRNA, and volatile chemical signals from plants in real time. The collected data is then stored in a database, and missing and outliers are detected and imputed or removed. The preprocessed data is then fed into a generative AI model, which takes into account past data and current environmental conditions to predict the plant's condition and necessary actions.

[1405] Device behavior

[1406] The device receives the analysis results sent from the server in real time and displays them visually through the user interface. For example, if the flowering period is approaching, the device will display the specific pollination timing and procedure, prompting the user to take appropriate action. The notification system uses Firebase and other services to quickly deliver the analysis results to the user's device.

[1407] User Actions

[1408] Based on the information provided by the device, the user plans and executes specific actions related to plant growth. For example, if the user receives a notification that "pollination time is approaching," the user prepares for pollination and carries it out at the appropriate time. The results of the actual work and any observed changes in the plant are entered into the device, and the feedback is sent to the server. This feedback is used to retrain the generative AI model, contributing to improving the accuracy of analysis in future generations.

[1409] Specific examples

[1410] For example, the server receives a signal from a sensor that "flowering is approaching," analyzes the signal, and notifies the user that "the time for pollination is approaching." The user checks this notification and performs pollination at the appropriate time. The results of this work are reported to the server as feedback, contributing to improved prediction accuracy for the next time. Furthermore, if the server detects a signal of damage caused by pests, it will send an alert to the device saying, "Insect damage has occurred. Urgent action is required." Based on this, the user can quickly take measures such as spraying pesticides, and send the results of this to the server as feedback.

[1411] Prompt Sentence Examples

[1412] You are designing a system that analyzes signals emitted by plants and provides information about their status to the user. These signals include ultrasound, microRNA, and volatile substances. Please specify the hardware and software used in this system and explain the specific data processing and data calculation methods.

[1413] These processes enable real-time monitoring and analysis of plant conditions, providing users with timely information on the actions they need to take.

[1414] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1415] Server Processing Steps

[1416] Step 1: Data collection

[1417] Input: Signal data from ultrasonic sensors, microRNA detectors, and volatile substance detectors.

[1418] How it works: The server receives real-time signal data about the plant's condition from these various sensors. The sensors record data every second and send it to the server. The server stores the received data in a database.

[1419] Output: The raw data collected.

[1420] Step 2: Data Preprocessing

[1421] Input: Raw data collected.

[1422] Specific operation: The server detects missing values ​​and outliers in the received data and processes them (complements or removes them). For complementation, it uses the most recent valid data, and for outliers, it corrects or removes values ​​that are out of range.

[1423] Output: Preprocessed clean data.

[1424] Step 3: Data analysis

[1425] Input: Preprocessed clean data.

[1426] Specific operation: The server inputs the preprocessed data into the generative AI model, which then predicts the plant's condition and growth based on past data and current environmental conditions.

[1427] Output: Analysis results (e.g., flowering time, pollination timing, risk of pests and diseases, etc.).

[1428] Step 4: Send information

[1429] Input: Analysis results.

[1430] Specific operation: Based on the analysis results of the generation AI, the server notifies the user of necessary action information (e.g., flowering time, pollination timing). This information is sent to the user's device via a notification service such as Firebase.

[1431] Output: Notifications sent to the user's device.

[1432] Step 5: Receiving feedback

[1433] Input: Feedback data from users.

[1434] Specific operation: Receives work results and observation data entered by the user into the device, stores this feedback data in a database, and creates a new learning set to retrain the generative AI model.

[1435] Output: Feedback data, improved generative AI model.

[1436] Terminal processing steps

[1437] Step 1: Receiving information

[1438] Input: Analysis results and action plans sent from the server.

[1439] What it does: The device maintains a connection with the server and periodically checks for new notifications. If any notifications are available, they are immediately downloaded and displayed to the user.

[1440] Output: Notification data waiting to be displayed.

[1441] Step 2: User Interface Display

[1442] Input: Notification data.

[1443] Specific operation: The device analyzes the received notification and displays it on the user interface. It provides instructions in visually appealing graphs and text, making it easy for the user to understand.

[1444] Output: A screen display for the user to see.

[1445] User processing steps

[1446] Step 1: Verify the information

[1447] Input: Notifications from your device.

[1448] Specific behavior: The user taps the notification on their device to view more information and plan appropriate actions based on the information displayed.

[1449] Output: A confirmed action plan.

[1450] Step 2: Take action

[1451] Input: Confirmed action plan.

[1452] Specific actions: The user performs the necessary work on site based on the notification. For example, if pollination is required, prepare pollination tools and perform pollination at the appropriate time.

[1453] Output: The training work carried out.

[1454] Step 3: Provide feedback

[1455] Input: Development activities performed and observation data.

[1456] Specific operation: The user inputs the results of their work and observed changes into the terminal and sends them to the server, which improves the analysis accuracy of the entire system.

[1457] Output: Feedback data sent to the server.

[1458] (Application example 1)

[1459] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1460] Conventional plant growth management systems are required to not only grasp the state of plants, but also to formulate and execute appropriate action plans, but there are many challenges to achieving this. Specifically, they require accurate data collection and analysis to accurately analyze signals from plants and provide optimal growth conditions, as well as the autonomy to take appropriate actions based on that data. Conventional methods lack sufficient analytical accuracy and automation of actions, making effective plant growth management difficult.

[1461] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1462] In this invention, the server includes means for collecting ultrasound, microRNA, and volatile substances from plants, means for preprocessing the collected data, means for analyzing the preprocessed data and using a generating AI to estimate the state of the plant, means for sending information about plant growth to a user based on the analysis results, means for receiving feedback from the user and improving the accuracy of the generating AI, and means for controlling a factory robot that executes an action plan required for plant growth based on the analysis results. This realizes a system that can perform everything from data analysis to action execution in an integrated manner, enabling optimal plant growth management.

[1463] "Ultrasound" refers to sound waves with a frequency of 20 kilohertz or higher, which contain unique signals emitted by plants.

[1464] "MicroRNAs" are small RNA molecules that regulate gene expression in plants and are important biomarkers that reflect the health and stress status of plants.

[1465] "Volatiles" are volatile compounds emitted by plants that indicate the plant's condition or environmental stress.

[1466] "Generative AI" is an artificial intelligence model that estimates the condition of plants based on collected data and suggests optimal growth conditions and action plans.

[1467] An "action plan" is a plan that includes specific actions and procedures required for the growth and protection of plants, based on the analysis results of the generative AI.

[1468] A "factory robot" is a mechanical device that operates autonomously or semi-autonomously within a factory and executes action plans related to plant growth management.

[1469] "Data preprocessing" is the process of detecting missing or outliers in collected plant data and imputing or removing them appropriately.

[1470] "Feedback" is information provided by the user about execution results and observed changes, which is used to improve the accuracy of the generative AI.

[1471] This invention is a system for understanding the state of plants in real time and implementing appropriate cultivation actions based on that information. The main components of the system are a server, terminals, and factory robots.

[1472] server

[1473] The server includes the following means:

[1474] 1. Data collection: The server collects data about the plants in real time from various sensors, such as ultrasonic sensors, microRNA analyzers, and volatile substance detectors.

[1475] 2. Data preprocessing: Detect missing or outliers in the collected data, and impute or remove them as necessary to improve the reliability of the data and the accuracy of the analysis.

[1476] 3. Generative AI analysis: The preprocessed data is input into the generative AI, which takes into account past data and environmental conditions to estimate the plant's condition and necessary growing conditions.

[1477] 4. Information transmission: Based on the analysis results, specific information such as flowering time, pollination timing, and fruit thinning timing is notified to the user. In addition, an action plan generated based on the analysis results is transmitted to the factory robot.

[1478] 5. Feedback reception: Receive feedback from the user on the actions taken to improve the analytical accuracy of the generative AI.

[1479] Terminal (user's device)

[1480] The terminal includes the following means:

[1481] 1. Information reception: Receive analysis results and action plans sent from the server in real time.

[1482] 2. User interface: The received information is displayed to the user in an easy-to-understand format, showing the notified pollination timing and procedures.

[1483] Factory robots

[1484] The factory robot includes the following means:

[1485] 1. Action plan execution: According to the action plan sent from the server, appropriate cultivation tasks are carried out automatically or semi-automatically, such as watering, fertilizing, and pest control.

[1486] Overall processing flow

[1487] The server first collects data about the plants from various sensors. The collected data is preprocessed, and missing or outliers are filled in or removed. The preprocessed data is analyzed by the generative AI, which estimates the plant's condition and the conditions necessary for its growth. The analysis results are notified to the user's device, and an appropriate action plan is sent to the factory robot. The factory robot carries out the growth work based on this action plan. The user provides feedback on actual work results and observations, which are sent to the server and used to improve the accuracy of the generative AI.

[1488] Specific examples

[1489] For example, the server collects ultrasonic data, microRNA data, and volatile substance data from a plant and analyzes it to determine that the plant is nearing flowering. As a result, the user's device is notified that "the time for pollination is approaching." At the same time, an action plan for carrying out "pollination" is sent to the factory robot. The user checks the notification, and the robot carries out pollination at the appropriate time. The results are then reported to the server as feedback.

[1490] Example prompts for generative AI models

[1491] Enter your plant's ultrasound, microRNA, and volatile data, and we'll estimate your next steps and suggest an appropriate course of action.

[1492] Example data:

[1493] Ultrasonic: 10kHz

[1494] MicroRNA: [RNA_sequence_A, RNA_sequence_B]

[1495] Volatile substances: [Compound A, Compound B]

[1496] In this way, the entire system works together to achieve optimal plant growth.

[1497] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1498] Step 1:

[1499] Data collection

[1500] The server collects real-time plant data from sensors such as ultrasonic sensors, microRNA analyzers, and volatile substance detectors. In this step, raw data from each sensor is input and output in its raw format.

[1501] Step 2:

[1502] Data Preprocessing

[1503] The server detects missing or outliers in the collected data and imputes or removes them as necessary. Here, the raw data collected in the previous step is used as input, and reliable data is output with outliers replaced with default values ​​such as "default_value." Specific operations include checking the consistency of the data and filtering outliers.

[1504] Step 3:

[1505] Generative AI analysis

[1506] The server inputs the preprocessed data and uses generative AI to estimate the plant's condition. The output is an analysis result that clarifies the plant's current condition and necessary growing conditions. Input data to the generative AI model includes values ​​of ultrasound, microRNA, and volatile substances.

[1507] Step 4:

[1508] Information transmission

[1509] Based on the analysis results of the generated AI, the server sends information about plant growth to the user's device. It also sends an action plan to the factory robot. The analysis results are used as input, and the output is instructions and action plans received by the device and robot. The specific operation involves data communication via a network.

[1510] Step 5:

[1511] Information reception (terminal)

[1512] The terminal receives analysis results and action plans sent from the server in real time. The input is the analysis results and action plans from the server, and the output is display information converted into a format that can be viewed by the user. Specific operations include analyzing the received data and displaying it on the user interface.

[1513] Step 6:

[1514] Action plan execution (factory robot)

[1515] Factory robots automatically or semi-automatically perform appropriate cultivation tasks based on an action plan sent from a server. The action plan from the server is given as input, and the output is the actual physical work, such as watering, fertilizing, and pest control. Specific operations include robot motion control and the use of sensors.

[1516] Step 7:

[1517] Feedback provided by users

[1518] The user inputs feedback about the tasks performed and the changes observed into the terminal, which then sends it to the server. The input is feedback data from the user, and the output is data used to update the generative AI. Specific operations include data input via the user interface and feedback transmission via the network.

[1519] Step 8:

[1520] Feedback reception (server)

[1521] The server receives feedback from users and uses it to improve the accuracy of the generative AI. The input is the user feedback data, and the output is an updated, more accurate generative AI model. Specific operations include analyzing the feedback data and training the generative AI model.

[1522] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1523] The present invention is a system that analyzes signals emitted by plants, such as ultrasound, microRNA, and volatile substances, to estimate the plant's condition and notify the user of appropriate actions based on the results. This system incorporates an emotion engine that recognizes the user's emotions, and also has the function of adjusting the content and timing of notifications based on the user's emotions and making suggestions to reduce the user's stress level. Specific processing of the program is described below.

[1524] server

[1525] 1. Data Collection:

[1526] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances from plants, which reflect the physiological state and stress status of plants.

[1527] 2. Data Preprocessing:

[1528] The server preprocesses the collected data, specifically detecting missing and outlier values ​​and imputing or removing them as necessary, which is an important step to ensure the reliability and accuracy of the data.

[1529] 3. Data Analysis:

[1530] The pre-processed data is fed into generative AI to estimate the plant's condition and needs, including predicting flowering time and risk of pest and disease damage.

[1531] 4. Emotion analysis:

[1532] The server uses an emotion engine to analyze the user's emotional data sent from the device, thereby understanding the user's current emotional state and stress level.

[1533] 5. Information Transmission:

[1534] Based on the analysis results, the server creates a specific action plan, adjusting the content and timing of notifications according to the user's emotional state to notify the user in the most optimal way.

[1535] 6. Feedback received:

[1536] The server receives feedback from users to improve the accuracy of the generative AI. The feedback information is the results of work and observations, and is useful for future analysis.

[1537] Terminal (user's device)

[1538] 1. Receiving information:

[1539] The terminal receives the prediction results and action plans sent from the server in real time.

[1540] 2. Emotional Data Collection:

[1541] The device uses a camera, microphone, and other sensors to collect information such as the user's facial expressions, voice, and heart rate, and transmits this information to a server as emotional data.

[1542] 3. Information display:

[1543] The device then displays the received information in an easy-to-understand format for the user. For example, if the flowering period is approaching, it will display specific pollination timing and procedures.

[1544] 4. Operation Guide:

[1545] If necessary, provide the user with additional information and instructions to assist with specific operations.

[1546] User

[1547] 1. Information verification:

[1548] The user checks the information provided by the terminal and understands specific actions to take regarding plant growth.

[1549] 2. Take action:

[1550] The user then takes appropriate action based on instructions from the device, such as manually pollinating the plants based on the notified pollination timing.

[1551] 3. Record your emotions:

[1552] Users record their emotions and stress levels on their devices and send them to a server, where the emotional data is used for analysis and action optimization.

[1553] 4. Providing Feedback:

[1554] The user inputs the results of their actions and any changes they observe into their device and sends them to the server. This feedback information is used to improve the analytical accuracy of the generative AI.

[1555] Specific examples

[1556] For example, if the server receives a signal from a sensor that "flowering is approaching" and analyzes it, it notifies the user that "the time for pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification will be sent as is, but if it detects that the user is in a high stress state, the message will be adjusted to include a warning such as "The important time for pollination is approaching, but please proceed with the work without overdoing it."

[1557] Furthermore, if the server detects signals of damage caused by pests, it sends an alert to the device saying, "Insect damage has occurred. Immediate action is required." If the user feels high stress in response to this alert, the emotion engine sends a message emphasizing how to deal with the situation, such as, "Insect damage has occurred. Please calmly take action by following the procedures." Based on this, the user can quickly take action, such as spraying pesticides. The results are then sent to the server as feedback.

[1558] In this way, the server, terminals, and users can work together to efficiently and effectively manage plant cultivation. In addition, the combination of the emotion engine reduces the psychological burden on users and improves work efficiency.

[1559] The processing flow will be explained below.

[1560] Server Processing Steps

[1561] Step 1: Data collection

[1562] The server collects real-time signal data from various sensors and specialized devices, including ultrasonic waves, microRNAs, and volatile substances, to detect the physiological and stress states of plants.

[1563] Step 2: Data Preprocessing

[1564] The server detects whether there are any missing or outliers in the collected data and performs imputation or removal as necessary, thereby improving the reliability of the data and the accuracy of the analysis.

[1565] Step 3: Data analysis

[1566] The preprocessed data is input into a generative AI that estimates the plant's condition and needs, and the generative AI predicts flowering time and pest and disease risk based on past data and environmental conditions.

[1567] Step 4: Sentiment Analysis

[1568] The server uses an emotion engine to analyze the user's emotional data received from the device, thereby understanding the user's current emotional state and stress level.

[1569] Step 5: Send information

[1570] Based on the analysis results, the server creates a specific action plan, adjusting the content and timing of notifications according to the user's emotional state to notify the user in the most optimal way.

[1571] Step 6: Receiving feedback

[1572] The server receives feedback from users, including the results of tasks performed and observations, and uses this feedback to improve the analytical accuracy of the generative AI.

[1573] Terminal processing steps

[1574] Step 1: Receiving information

[1575] The terminal receives the analysis results and action plans sent from the server in real time.

[1576] Step 2: Collecting Emotional Data

[1577] The device uses a camera, microphone, and other sensors to collect emotional data such as the user's facial expressions, voice, and heart rate, and sends this data to a server.

[1578] Step 3: Display information

[1579] The device then displays the received information in an easy-to-understand format for the user. For example, if the flowering period is approaching, it will display specific pollination timing and procedures.

[1580] User processing steps

[1581] Step 1: Verify the information

[1582] The user checks the information provided by the terminal and understands specific actions to take regarding plant growth.

[1583] Step 2: Take action

[1584] The user can then take appropriate action based on instructions from the device, such as manually pollinating the plants based on the notified pollination timing.

[1585] Step 3: Record your emotions

[1586] Users record their emotions and stress levels on their devices and send them to a server, where the emotional data is used for analysis and action optimization.

[1587] Step 4: Provide feedback

[1588] The user inputs the results of their execution and any observed changes into their device and sends them to the server. The feedback information is used to improve the analysis accuracy of the generative AI.

[1589] Specific examples

[1590] For example, if the server receives a signal from a sensor that "flowering is approaching" and analyzes it, it notifies the user that "the time for pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification will be sent as is, but if it detects that the user is in a high stress state, the message will be adjusted to include a warning such as "The important time for pollination is approaching, but please proceed with the work without overdoing it."

[1591] When the server detects a signal of pest damage, it sends an alert to the device saying, "Insect damage has occurred. Immediate action is required." If the user feels high stress in response to this alert, the emotion engine sends a message emphasizing how to deal with the situation, such as, "Insect damage has occurred, but please calmly follow the procedures and take action." Based on this, the user can quickly take action, such as spraying pesticides, and send the results of that action to the server as feedback.

[1592] In this way, the server, terminals, and users can work together to efficiently and effectively manage plant cultivation. In addition, the combination of the emotion engine reduces the psychological burden on users and improves work efficiency.

[1593] Example 2

[1594] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1595] Conventional agricultural systems have had difficulty accurately understanding the physiological state and stress levels of plants and taking appropriate measures based on that information. Furthermore, they do not provide information that takes into account the user's emotional state, making it impossible to reduce the user's psychological burden. Furthermore, continuous system optimization based on feedback is insufficient.

[1596] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting ultrasound, microRNA, and volatile substances from plants, means for preprocessing the collected data, means for analyzing the preprocessed data and using a generation AI to estimate the plant's condition, means for analyzing user emotion data, means for sending information about plant growth to the user based on the analysis results and the emotion data, and means for receiving feedback from the user and improving the accuracy of the generation AI. This makes it possible to accurately grasp the plant's condition and take appropriate measures based on that understanding, taking the user's emotional state into consideration. Furthermore, the system can be continuously optimized by reflecting user feedback.

[1597] "Ultrasound" is a type of sound wave that usually has a frequency of 20 kHz or higher and is inaudible to the human ear.

[1598] "MicroRNA" is a small RNA molecule that regulates gene expression and has the function of suppressing the production of specific proteins.

[1599] "Volatile substances" are substances that easily evaporate into gas at room temperature and serve as indicators of the physiological state and stress of plants.

[1600] "Preprocessing" refers to a series of processes that prepare collected data in a form suitable for analysis, and includes operations such as filling in and removing missing or outlier values ​​in the data.

[1601] "Generative AI" is an artificial intelligence technology used to analyze collected data and predict specific situations or conditions.

[1602] "Emotion data" is data including the user's facial expression, voice, heart rate, etc., and indicates the user's current emotional state and stress level.

[1603] "Feedback" is information that records the results of operations performed by the user and observed changes and returns them to the system.

[1604] "Information regarding plant cultivation" includes information regarding the flowering period of plants, timing of pollination, fruit thinning, harvesting time, proper use of pesticides, and avoidance of insect damage.

[1605] "Missing data" refers to values ​​that are not recorded because measurements or collection were not performed correctly.

[1606] An "outlier" is a data value that deviates from the normal range and may affect the analysis results.

[1607] The "emotional state of the user" indicates the state of emotions or moods that the user is feeling, and specifically includes joy, sadness, surprise, anger, fear, and the like.

[1608] The "stress level" indicates the degree of mental tension or pressure that the user feels.

[1609] The "emotion engine" is software that analyzes a user's emotional data and evaluates their current emotional state and stress level.

[1610] MODE FOR CARRYING OUT THE INVENTION

[1611] This invention is a system that analyzes signals emitted by plants, such as ultrasonic waves, microRNA, and volatile substances, to estimate the plant's condition and notify the user of appropriate actions based on the results. This system incorporates an emotion engine that recognizes the user's emotions, and also has the function of adjusting the content and timing of notifications based on the user's emotions and making suggestions to reduce the user's stress level.

[1612] Hardware and software used

[1613] server:

[1614] Hardware: High-performance server (e.g., server with Intel Xeon processor)

[1615] Software: Python, TensorFlow, Pandas, Numpy, IBM Watson Sentiment Analysis API, Microsoft Azure

[1616] Terminal (user's device):

[1617] Hardware: Smartphone (Android or iOS), wearable device (smartwatch with heart rate monitor)

[1618] Software: Dedicated application (Android / iOS app), Bluetooth module

[1619] Data collection

[1620] The server collects real-time signal data of ultrasonic waves, microRNAs, and volatile substances from multiple sensors (e.g., ultrasonic sensors, RNA sequencers, and gas chromatographs) in plants. These signal data reflect the physiological state and stress status of plants.

[1621] Data Preprocessing

[1622] The server preprocesses the collected data using the Python Pandas library to detect missing and outliers, and impute or remove them as needed. Data smoothing and normalization are also performed at this stage.

[1623] Data analysis

[1624] The preprocessed data is then fed into a generative AI model, a deep learning model built using TensorFlow, which analyzes the plant's condition, predicting flowering time and assessing pest and disease risk, and creating a specific action plan.

[1625] Emotion analysis

[1626] The server analyzes the user's emotional data sent from the device using IBM Watson's emotion analysis API, which allows it to understand the user's current emotional state and stress level. Emotional data is collected from the device's camera, microphone, and wearable device.

[1627] Information transmission

[1628] Based on the analysis results and emotional data, the server notifies the user of the appropriate action: for users in a low-stress state, it sends a simple notification such as "The time for pollination is approaching," and for users in a high-stress state, it sends a notification with a warning such as "Please proceed with the pollination work without overdoing it."

[1629] Receive feedback

[1630] The server receives feedback from users, who send the results of their work and observed changes as feedback to the server through a dedicated application. This feedback information is recorded in a database and used to improve the accuracy of the generative AI model.

[1631] Specific examples

[1632] For example, if a plant is about to flower, the server analyzes the data from the sensor and notifies the user that "pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification is sent as is, but if the user is in a high stress state, an adjusted message is sent saying, "The important pollination time is approaching, but please proceed with your work without overdoing it."

[1633] When signals of pest damage are detected, the server sends an alert to the device saying, "Insect damage has occurred. Immediate action is required." If the user feels high stress in response to this alert, the emotion engine sends a message emphasizing countermeasures, such as, "Insect damage has occurred, but please calmly follow the procedures and take action." Based on this, the user can quickly take measures such as spraying pesticides.

[1634] In this way, the server, terminals, and users can work together to efficiently and effectively manage plant cultivation. Furthermore, by utilizing the emotion engine, the psychological burden on users can be reduced and work efficiency can be improved.

[1635] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1636] Program processing flow

[1637] Step 1: Data collection

[1638] How it works: The server collects data in real time from multiple sensors (ultrasonic sensors, RNA sequencers, and gas chromatographs) attached to plants, including the sounds of plant growth (ultrasonic waves), microRNAs (gene expression status), and volatile substances (vaporized chemical components).

[1639] Input: Signal data from the sensor.

[1640] Output: Raw, unprocessed data.

[1641] Step 2: Data Preprocessing

[1642] Specific operation: The server preprocesses the collected raw data. At this stage, it creates a data frame using Python's Pandas library, detects and imputes missing values ​​and outliers, and normalizes the data. Specifically, it performs line imputation, outlier removal, and smoothing.

[1643] Input: Raw, unprocessed data.

[1644] Output: A formatted dataset.

[1645] Step 3: Data analysis

[1646] How it works: The server inputs the preprocessed data into a generative AI model (e.g., a deep learning model using TensorFlow) to analyze the plant's condition and needs. This analysis includes predicting flowering time, assessing pest and disease risk, and generating a specific action plan.

[1647] Input: The formatted dataset.

[1648] Output: Plant status, risk assessment and action plan generated.

[1649] Step 4: Collect emotional data

[1650] Specific operation: The device uses sensors such as a camera, microphone, and smartwatch with a heart rate monitor to collect the user's facial expressions, voice, and heart rate, and transmits them to the server, thereby obtaining the user's emotional data.

[1651] Input: User biometric information (facial expression, voice, heart rate).

[1652] Output: User emotion data.

[1653] Step 5: Sentiment Analysis

[1654] Specific operation: The server analyzes the emotion data received from the device using an emotion engine (e.g., IBM Watson's emotion analysis API), thereby evaluating the user's emotional state and stress level.

[1655] Input: User emotion data.

[1656] Output: Evaluation of emotional state and stress level.

[1657] Step 6: Send information

[1658] Specific operation: Based on the analysis results and emotional data, the server notifies the user of appropriate actions. Notifications are sent via email or push notification, and the content is adjusted according to the user's emotional state. Users in a low-stress state will receive a simple notification such as "The time for pollination is approaching," while users in a high-stress state will receive a notification with a warning such as "Please proceed with the pollination work without overdoing it."

[1659] Input: Analysis results, emotional state and stress level.

[1660] Output: The adjusted notification content.

[1661] Step 7: Receive and display information

[1662] Specific operation: The device receives the notification sent from the server and displays it in a dedicated application. The displayed content includes specific response procedures and timing.

[1663] Input: Notification from the server.

[1664] Output: The action plan that is displayed to the user.

[1665] Step 8: Take Action

[1666] Specific operation: The user checks the information provided by the device and performs actual farm work, such as manually pollinating crops based on the notified pollination timing.

[1667] Input: Information provided by the device.

[1668] Output: The action that was performed.

[1669] Step 9: Provide feedback

[1670] How it works: The user inputs the results of their actions and observed changes into the device and sends them to the server. This feedback is stored in a database and used to improve the accuracy of the generative AI model.

[1671] Input: Execution results, observed changes.

[1672] Output: Feedback data.

[1673] Through this series of processing steps, the system can accurately grasp the plant's growth status, notify the user of appropriate actions, and reduce the user's psychological burden by taking into account the user's emotional state. Furthermore, the system can continuously optimize the entire system by utilizing feedback.

[1674] (Application example 2)

[1675] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1676] Currently, there are systems that closely monitor the condition of plants and provide advice on their cultivation, but there are no notification systems that take into account the user's emotions and stress level. This increases the psychological burden on the user, resulting in inefficient plant cultivation management. The present invention aims to solve these problems.

[1677] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting ultrasound, microRNA, and volatile substances from plants; means for preprocessing the collected data; means for analyzing the preprocessed data and using a generation AI to estimate the plant's condition; means for sending information about plant growth to the user based on the analysis results; means including an emotion engine that recognizes the user's emotions and adjusts the content and timing of notifications; and means for receiving feedback from the user and improving the accuracy of the generation AI. This enables efficient plant growth management while reducing the user's psychological burden by suggesting appropriate actions according to the plant's condition and providing notifications that take the user's emotional state into consideration.

[1678] "Ultrasound" is a sound wave with a vibration frequency of 20,000 Hz or higher, which is inaudible to the human ear but is used to detect the internal structure of materials.

[1679] "MicroRNA" is a short non-coding RNA molecule approximately 22 bases in length that functions to regulate gene expression.

[1680] "Volatiles" are substances that evaporate readily at room temperature and may be released by plants to indicate stress or other physiological conditions.

[1681] "Preprocessing" refers to the process of removing missing values ​​and outliers from collected data and preparing it for analysis.

[1682] "Generative AI" is a type of artificial intelligence technology that has the ability to generate new information and predictions through data analysis.

[1683] The "emotion engine" is a system that recognizes the user's emotions and stress level, and adjusts the content and timing of notifications based on that information.

[1684] "Notification content" is information that notifies the user of the analysis results and specific action plans based on those results.

[1685] "Feedback" is information provided by the user about the execution results and their evaluation, which helps improve the accuracy of the system.

[1686] "Plant condition" refers to comprehensive information about the plant's health, growth process, stress status, etc.

[1687] The system embodying this invention collects ultrasonic waves, microRNAs, and volatile substances from plants, preprocesses and analyzes them, and notifies the user of appropriate actions based on the results. It also incorporates an emotion engine that recognizes the user's emotions and adjusts the content and timing of notifications.

[1688] System hardware configuration

[1689] The system hardware is as follows:

[1690] Plant signal sensors: using ultrasonic sensors, microRNA sensors, and volatile substance sensors.

[1691] User emotion recognition devices: Use cameras, microphones, heart rate sensors, etc.

[1692] Devices: Smartphones, tablets, head-mounted displays, etc.

[1693] System software configuration

[1694] The main software used is as follows:

[1695] Generative AI models: perform data analysis and predictions.

[1696] Emotion recognition engine: Analyzes user emotional data and adjusts notification content.

[1697] Data preprocessing module: Imputes and removes missing and outlier values.

[1698] How the system is implemented

[1699] 1. Data Collection:

[1700] The server collects real-time data from various sensors attached to the plants, including ultrasonic waves, microRNAs, and volatile compounds, which reflect the physiological and stress states of the plants.

[1701] 2. Data Preprocessing:

[1702] The server detects missing or outliers in the collected data and performs imputation or removal as necessary, thereby ensuring the reliability and accuracy of the data.

[1703] 3. Data Analysis:

[1704] The pre-processed data is fed into a generative AI model to estimate the plant's condition and needs, such as predicting flowering time and risk of pest and disease damage.

[1705] 4. Emotion recognition:

[1706] The device uses a camera, microphone, and heart rate sensor to collect user emotional data and transmit it to a server, which then uses an emotion engine to analyze the data and determine the user's current emotional state and stress level.

[1707] 5. Information Transmission:

[1708] The server creates a specific action plan based on the analysis results and adjusts the content and timing of notifications according to the user's emotional state. For example, it may notify the user that pollination is approaching, or send a message containing a warning if the user is under high stress.

[1709] 6. Feedback received:

[1710] The server receives feedback from users and uses it to improve the accuracy of the generative AI. The feedback information is the results of the tasks performed and observations.

[1711] Specific examples

[1712] For example, if the server receives a signal from a sensor that "flowering is approaching" and analyzes it, it notifies the user that "the time for pollination is approaching." If the emotion engine detects that the user is in a low stress state, the notification will be sent as is, but if it detects that the user is in a high stress state, the message will be adjusted to include a warning that "the important time for pollination is approaching, but please proceed with the work without overdoing it."

[1713] Prompt Sentence Examples

[1714] "Sensor URL: http: / / api.plant_sensors.com / data"

[1715] "User data: including user emotional data"

[1716] This system allows for efficient plant growth management while reducing the psychological burden on users by suggesting appropriate actions based on the condition of the plant and providing notifications that take into account the user's emotional state.

[1717] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1718] Step 1:

[1719] Data collection

[1720] The server collects ultrasonic, microRNA, and volatile data in real time from sensors attached to plants. The input is the signal data from the sensors, and the output is the collected raw data. This data reflects the physiological state and stress status of the plants.

[1721] Step 2:

[1722] Data Preprocessing

[1723] The server detects missing values ​​and outliers in the collected data and imputes or removes them as necessary. The input is the collected raw data, and after passing through the data cleaning and imputation process, it outputs more accurate and reliable clean data.

[1724] Step 3:

[1725] Data analysis

[1726] The server inputs the preprocessed data into a generative AI model to estimate the plant's condition and needs. The input is preprocessed clean data, and the output is the plant's condition (e.g., flowering time, risk of pests, etc.). Specifically, the AI ​​model analyzes the data and makes predictions and diagnoses.

[1727] Step 4:

[1728] Emotion recognition data collection

[1729] The device uses a camera, microphone, and heart rate sensor to collect emotional data such as the user's facial expressions, voice, and heart rate. The input is sensory data from the camera and microphone, and the output is the user's emotional data (e.g., stress level, emotional state).

[1730] Step 5:

[1731] Emotion analysis

[1732] The server analyzes the user's emotional data sent from the device to understand the user's current emotional state and stress level. The input is the collected user's emotional data, and the output is the analyzed emotional state. The emotion engine runs and analyzes this.

[1733] Step 6:

[1734] Information transmission and notification coordination

[1735] The server creates a specific action plan based on the analysis results and adjusts the content and timing of notifications according to the user's emotional state. The input is the plant's condition and the user's emotional state, and the output is an adjusted notification message. Notifications are given taking into account the user's psychological state, such as warnings about pollination timing or insect damage.

[1736] Step 7:

[1737] Receive feedback

[1738] The server receives feedback provided by users (e.g., the results of actual work performed or observation data) and improves the accuracy of the generative AI model. The input is the feedback information from the user, and the output is an improved AI model. The feedback is analyzed and the system performance is improved.

[1739] This step not only suggests appropriate actions based on the condition of the plant, but also reduces the user's psychological burden by providing notifications that take into account the user's emotional state, thereby enabling efficient plant growth management.

[1740] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1741] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1742] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1743] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1744] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1745] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1746] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1747] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1748] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1749] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1750] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1751] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1752] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1753] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1754] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1755] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1756] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1757] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1758] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1759] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1760] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1761] The following is further disclosed regarding the above embodiment.

[1762] (Claim 1)

[1763] A means for collecting ultrasound, microRNA, and volatiles from plants;

[1764] means for pre-processing the collected data;

[1765] a means for using generative AI to analyze the pre-processed data and estimate the plant condition;

[1766] means for transmitting information about plant growth to a user based on the analysis results;

[1767] The system includes a means for receiving user feedback to improve the accuracy of the generative AI.

[1768] (Claim 2)

[1769] 2. The system according to claim 1, wherein the information relating to plant cultivation includes information relating to flowering time, timing of pollination, fruit thinning, harvest time, proper use of pesticides, and avoidance of insect damage.

[1770] (Claim 3)

[1771] 2. The system of claim 1, further comprising means for detecting, imputing, or removing missing or outlier values ​​in preprocessing of data collected from plants.

[1772] "Example 1"

[1773] (Claim 1)

[1774] A means for collecting ultrasound, microRNA, and volatiles from plants;

[1775] means for pre-processing the collected data;

[1776] A means for inputting the preprocessed data into a generative AI model to estimate the plant condition;

[1777] means for transmitting information about the growth of the plant to a user's terminal based on the analysis result;

[1778] a means for receiving user feedback and improving the accuracy of the generative AI model; and

[1779] a means for visually displaying information and providing real-time notifications through a user interface;

[1780] A means to retrain generative AI models based on feedback; and

[1781] A system including:

[1782] (Claim 2)

[1783] The system of claim 1, wherein the information regarding plant cultivation includes information regarding flowering time, timing of pollination, fruit thinning, harvest time, proper use of pesticides, and avoidance of insect damage, and the user can take appropriate action based on the information.

[1784] (Claim 3)

[1785] The system according to claim 1, further comprising means for detecting, complementing or removing missing values ​​and outliers in the preprocessing of data collected from plants, thereby eliminating the influence of defective data and improving the accuracy of analysis.

[1786] "Application Example 1"

[1787] (Claim 1)

[1788] A means for collecting ultrasound, microRNA, and volatiles from plants;

[1789] means for pre-processing the collected data;

[1790] a means for using generative AI to analyze the pre-processed data and estimate the plant condition;

[1791] means for transmitting information about plant growth to a user based on the analysis results;

[1792] A means of receiving user feedback and improving the accuracy of the generative AI;

[1793] a means for controlling a factory robot that executes an action plan required for plant growth based on the analysis results;

[1794] A system including:

[1795] (Claim 2)

[1796] 2. The system according to claim 1, wherein the information relating to plant cultivation includes information relating to flowering time, timing of pollination, fruit thinning, harvest time, proper use of pesticides, and avoidance of insect damage.

[1797] (Claim 3)

[1798] 2. The system of claim 1, further comprising means for detecting, imputing, or removing missing or outlier values ​​in preprocessing of data collected from plants.

[1799] "Example 2: Combining Emotion Engines"

[1800] (Claim 1)

[1801] A means for collecting ultrasound, microRNA, and volatiles from plants;

[1802] means for pre-processing the collected data;

[1803] a means for using generative AI to analyze the pre-processed data and estimate the plant condition;

[1804] means for analyzing user emotion data;

[1805] means for transmitting information about plant growth to a user based on the analysis results and emotion data;

[1806] A means of receiving user feedback and improving the accuracy of the generative AI;

[1807] A system including:

[1808] (Claim 2)

[1809] 2. The system according to claim 1, wherein the information relating to plant cultivation includes information relating to flowering time, timing of pollination, fruit thinning, harvest time, proper use of pesticides, and avoidance of insect damage.

[1810] (Claim 3)

[1811] 2. The system of claim 1, further comprising means for detecting, imputing, or removing missing or outlier values ​​in preprocessing of data collected from plants.

[1812] "Application example 2 when combining emotion engines"

[1813] (Claim 1)

[1814] A means for collecting ultrasound, microRNA, and volatiles from plants;

[1815] means for pre-processing the collected data;

[1816] a means for using generative AI to analyze the pre-processed data and estimate the plant condition;

[1817] means for transmitting information about plant growth to a user based on the analysis results;

[1818] A means including an emotion engine that recognizes the emotion of the user and adjusts the content and timing of notifications;

[1819] The system includes a means for receiving user feedback to improve the accuracy of the generative AI.

[1820] (Claim 2)

[1821] The system of claim 1, wherein the information regarding plant cultivation is information regarding flowering time, pollination timing, fruit thinning, harvesting time, proper use of pesticides, and avoidance of insect damage, and further includes adjusting the content of notifications based on the user's emotional state.

[1822] (Claim 3)

[1823] 2. The system of claim 1, further comprising means for detecting, imputing, or removing missing or outlier values ​​in preprocessing of data collected from plants. [Explanation of symbols]

[1824] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for collecting ultrasound, microRNA, and volatiles from plants; means for pre-processing the collected data; a means for using generative AI to analyze the pre-processed data and estimate the plant condition; means for transmitting information about plant growth to a user based on the analysis results; The system includes a means for receiving user feedback to improve the accuracy of the generative AI.

2. 2. The system according to claim 1, wherein the information relating to plant growth includes information relating to flowering time, timing of pollination, fruit thinning, harvest time, proper use of pesticides, and avoidance of insect damage.

3. The system according to claim 1, further comprising means for detecting, imputing or removing missing values ​​and outliers in preprocessing of data collected from plants.

Images