System

Abstract

A system is provided.SOLUTION: The system includes a means for collecting climate information, a means for collecting soil analysis information, a means for collecting crop growth information, a means for analyzing the information using a multi-modal AI, a means for proposing an optimal agricultural technique based on the analysis result, and a means for controlling a robot that automates agricultural work in a farm.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Modern agriculture is susceptible to the effects of climate change, with small-scale farming and urban agriculture being particularly at risk. Furthermore, a lack of efficient and environmentally friendly agricultural methods makes it difficult to achieve sustainable agriculture. Supporting sustainable agriculture requires a system that can effectively collect and analyze climate data, soil analysis data, and crop growth status data, and then propose appropriate agricultural methods based on that data. This invention aims to solve these problems and improve agricultural efficiency and productivity while reducing environmental impact. [Means for solving the problem]

[0005] The present invention is a system that collects weather data, soil analysis data, and crop growth status data and analyzes this data using multimodal AI. The system includes a means for proposing optimal agricultural methods based on the analysis results and a means for controlling a robot that automates agricultural work on the farm. It also includes a means for transmitting the collected data to a server, analyzing it there, and sending the generated advice to the robot. These means enable environmentally friendly, sustainable agriculture. Specifically, the system performs analysis taking into account soil moisture, pH value, and the stage of crop growth, and can propose agricultural methods, including optimal irrigation and drainage measures.

[0006] "Climate Data" means information about weather conditions in a particular area, such as temperature, humidity, precipitation, and wind speed.

[0007] "Soil analysis data" refers to information that indicates the physical and chemical properties of soil, including, specifically, soil moisture, pH value, and nutrient content.

[0008] "Crop growth status data" is information indicating the growth stage of the crop, and includes, for example, height, number of leaves, color, and whether or not there is disease.

[0009] "Multimodal AI" is an artificial intelligence technology that can process and analyze multiple different types of data simultaneously.

[0010] "Analysis" is the process of identifying hidden patterns and relationships based on collected data.

[0011] "Best agricultural practices" are agricultural methods and procedures that optimize environmental conditions and crop health.

[0012] A "robot" is a mechanical device that automatically performs agricultural tasks.

[0013] A "server" is a computer system that collects, stores, and analyzes data.

[0014] "Advice" refers to guidelines for action or recommended farming methods provided based on the analysis results.

[0015] "Irrigation" is the method used to provide water to crops.

[0016] "Drainage measures" are methods for removing excess water. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The agricultural support system of the present invention collects weather data, soil analysis data, and crop growth status data, analyzes this data using multimodal AI, proposes efficient and environmentally friendly agricultural methods, and executes these methods using automated robots. The system consists of the following main components:

[0039] 1. Data Collection Module

[0040] User: The user first inputs the type of crop they are growing and the location of their farm into the system, which sets the baseline for collecting and analyzing climate data.

[0041] Terminal (robot): The terminal uses soil analysis sensors and crop growth monitoring cameras to periodically measure soil moisture, pH, and crop growth status. This acquired data is sent to the server in real time.

[0042] 2. Data transmission and analysis module

[0043] Server: The server obtains the necessary climate data from an external weather data API. It also receives soil data and crop growth data sent from the device, integrates them, and analyzes them using multimodal AI. This analysis derives optimal agricultural methods that take into account soil moisture, pH value, predicted precipitation, crop growth stage, and other factors.

[0044] 3. Advice Generation Module

[0045] Server: Based on the analysis results, it generates advice to propose specific agricultural methods, for example, recommending irrigation if soil moisture is low, or drainage measures if rainfall is forecast.

[0046] 4. Robot Control Module

[0047] Terminal (robot): Receives advice sent from the server and performs actual agricultural work based on the instructions. For example, if an irrigation command is issued, it will automatically water the area, and if drainage measures are required, it will activate the drainage system.

[0048] Specific examples

[0049] Let's take the example of an urban farmer growing tomatoes on a rooftop farm in Tokyo.

[0050] 1. User: A user logs into the system and inputs that the crop being grown is "tomatoes" and that the farm's location is "Tokyo."

[0051] 2. Terminal (Robot): The robot measures the soil moisture and pH value with sensors and records the tomato growth stage with a camera. The collected data is sent to the server in real time.

[0052] 3. Server: The server retrieves the latest weather information for Tokyo from an external weather data API, integrates it with soil humidity, pH value, and tomato growth status, and analyzes it. As a result of the analysis, it is determined that the soil humidity is low and that irrigation is necessary.

[0053] 4. Server: Generates irrigation advice and sends it to the robot.

[0054] 5. Terminal (Robot): Receives advice from the server, automatically starts irrigation, and provides the tomatoes with the appropriate amount of water.

[0055] This system allows farmers to receive advice on optimal farming methods in real time, enabling them to practice sustainable farming without hassle, thereby supporting the realization of efficient and environmentally friendly agriculture.

[0056] The processing flow will be explained below.

[0057] Step 1:

[0058] User: The user logs into the system and inputs the type of crop they are growing (e.g., tomatoes) and the location of their farm (e.g., Tokyo). This information serves as the basis for subsequent data collection and analysis.

[0059] Step 2:

[0060] Terminal (Robot): The robot uses soil analysis sensors and growth monitoring cameras to measure the soil moisture, pH value, and crop growth status in the farm. The collected data is sent to the server in real time.

[0061] Step 3:

[0062] Server: The server obtains the latest climate data for Tokyo (temperature, humidity, precipitation, wind speed, etc.) from an external weather data API. This climate data is integrated as information required for analysis.

[0063] Step 4:

[0064] Server: The server integrates the soil data, crop growth data, and weather data sent from the terminal (robot), inputs it into a multimodal AI system, and performs analysis. This analysis derives optimal agricultural methods that take into account soil moisture, pH value, and the stage of crop growth.

[0065] Step 5:

[0066] Server: Generates specific agricultural advice based on the results of multimodal AI analysis, such as recommending irrigation if soil moisture is low or drainage measures if rainfall is forecast.

[0067] Step 6:

[0068] Server: Sends the generated agricultural advice to the terminal (robot). This advice includes specific operational instructions (e.g., start irrigation, activate drainage system, etc.).

[0069] Step 7:

[0070] Terminal (Robot): The robot automatically performs agricultural tasks based on advice received from the server. For example, if an instruction is given to irrigate, it starts supplying water and supplies the appropriate amount. If an instruction is given to take drainage measures, it activates the drainage system to remove excess water.

[0071] This series of steps enables users to practice efficient and sustainable agriculture, improving productivity while reducing environmental impact.

[0072] Example 1

[0073] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0074] In traditional agriculture, it is difficult to determine the optimal timing of work based on climate change and soil conditions, making it difficult to efficiently manage crop growth. Furthermore, manual management is labor-intensive and time-consuming, requiring highly accurate data analysis to achieve sustainable agriculture. While there is a growing need for automation using robots, current systems lack seamless integration from data collection to work execution.

[0075] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0076] In this invention, the server includes: a means for a user to input information about the type of crop and the location of the farm; a means for measuring the soil moisture and pH value and the growth status of the crop using a soil analysis sensor and a crop growth monitoring camera; a means for transmitting the above data to the server in real time; a means for acquiring climate data from an external weather data API; a means for analyzing the collected data using multimodal AI; a means for proposing optimal agricultural techniques based on the analysis results; and a means for controlling a robot that automatically performs agricultural work on the farm based on the advice transmitted from the server. This enables the automation and optimization of agricultural work.

[0077] A "user" is an entity that inputs information into the system and provides data, such as a farm manager or agricultural worker.

[0078] "Crop type" refers to the particular type or variety of agricultural crop being grown.

[0079] "Farm location information" refers to information about the geographic location of the farm, and specifically includes data such as coordinates and address.

[0080] A "soil analysis sensor" is a device for measuring chemical and physical properties of soil, such as moisture and pH value.

[0081] A "crop growth monitoring camera" is a camera device for visually recording the growth status of cultivated crops.

[0082] "Real-time" refers to data being processed and transmitted immediately, with almost no time lag.

[0083] A "server" is a central computer system that processes, analyzes, stores, and communicates with other system components.

[0084] "External Weather Data API" means an external application programming interface that provides weather data over the Internet.

[0085] "Climate Data" means information about current and forecast weather conditions in a particular geographic area, such as precipitation, temperature, and humidity.

[0086] "Multimodal AI" is an artificial intelligence technology that integrates and analyzes multiple different types of data (e.g., climate data, soil data, video data).

[0087] "Analysis Results" refers to information or conclusions obtained as a result of analysis conducted based on collected data.

[0088] "Agricultural methods" refer to the specific work and technical means used in growing crops.

[0089] A "robot" is a mechanical device that automatically performs agricultural tasks within a farm.

[0090] "Control" refers to a system issuing instructions to a robot or other device and managing and operating its operations.

[0091] "Advice" refers to optimal farming techniques and specific work instructions provided based on the analysis results.

[0092] The agricultural support system of the present invention collects weather data, soil analysis data, and crop growth status data and analyzes them using multimodal AI to automate and optimize agricultural work. Based on the analysis results, it proposes optimal agricultural methods and executes those methods using automated robots. The main components of this system are described below.

[0093] Data Entry and Collection

[0094] First, users input the type of crop they are growing and the location of their farm. This information is used to set the criteria for collecting and analyzing climate data. For example, entering "tomatoes" and "Tokyo" will generate an analysis based on Tokyo's climate conditions.

[0095] Next, the terminal (robot) uses a soil analysis sensor and a crop growth monitoring camera to periodically measure the soil humidity, pH value, and crop growth status. Specifically, the soil sensor measures humidity, and the camera records the growth status. This data is sent to the server in real time.

[0096] Data Transmission and Integration

[0097] The server receives soil data and crop growth data sent from the device and obtains necessary climate data from an external weather data API. For example, it integrates information such as predicted rainfall, temperature, and humidity and analyzes it together with the soil and crop growth data.

[0098] Data analysis

[0099] The server inputs the collected weather, soil, and crop growth data into the multimodal AI for analysis, which then derives optimal agricultural methods that take into account soil moisture, pH, predicted precipitation, and the stage of crop growth. Specifically, the AI ​​determines that soil moisture is low and that irrigation is necessary.

[0100] Advice Generation

[0101] Based on the analysis results, the server generates agricultural advice, including the timing and amount of irrigation and how to drain the fields after rainfall. For example, specific instructions such as "apply 100 liters of water immediately" are generated and sent to the device.

[0102] Robot Control and Execution

[0103] The terminal (robot) receives the advice sent from the server and performs the actual agricultural work based on the instructions. Specifically, the robot activates the irrigation system and sprays the set amount of water. In this way, agricultural work is carried out automatically and efficiently.

[0104] Specific example explanation

[0105] Let's take the example of an urban farmer growing tomatoes on a rooftop farm in Tokyo. A user logs into the system and enters that the crop being grown is "tomatoes" and that the farm is located in "Tokyo." The robot uses sensors to measure the moisture and pH value of the soil and a camera to record the tomato growth stage. The collected data is sent to a server in real time.

[0106] The server obtains the latest weather information for Tokyo from an external weather data API, and performs analysis by integrating it with soil humidity, pH value, and tomato growth status. The analysis results show that the soil humidity is low and determines that irrigation is necessary. The server generates advice recommending irrigation and sends it to the robot. The robot receives the advice from the server, automatically starts irrigation, and supplies the tomatoes with the appropriate amount of water.

[0107] Prompt Sentence Examples

[0108] "What are the main features of this system?"

[0109] How should users enter data?

[0110] "Please explain in detail how the data collection module works."

[0111] "Please tell me how this system suggests agricultural methods."

[0112] Please explain the flow of this system using a specific example.

[0113] This system allows farmers to receive advice on optimal farming methods in real time, enabling them to practice sustainable farming without hassle, thereby supporting the realization of efficient and environmentally friendly agriculture.

[0114] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0115] Step 1:

[0116] A user logs into the system and enters the type of crop they are growing (e.g., tomatoes) and the location of their farm (e.g., Tokyo).

[0117] Input: type of crop, farm location

[0118] Output: Reference climate data acquisition conditions

[0119] Specific operation: The user enters "tomato" and "Tokyo" through the system interface, which is then sent to the server.

[0120] Step 2:

[0121] The terminal (robot) uses a soil analysis sensor to measure the moisture and pH value of the soil, and a crop growth monitoring camera to capture images of the crop growth. This data is sent to a server in real time.

[0122] Input: Measurements from sensors and cameras

[0123] Output: Measurement data (humidity, pH value, growth status images)

[0124] How it works: The robot's soil sensors measure humidity at 30% and pH at 6.5, and its camera captures images of the tomato's growth stages, all of which are sent to a server.

[0125] Step 3:

[0126] The server accesses an external weather data API to obtain the latest weather data (e.g., predicted rainfall, temperature, and humidity) based on the farm's location, and also receives soil and crop growth data sent from the device.

[0127] Input: Farm location, soil data, growth data

[0128] Output: Integrated climate and farm data

[0129] Specific operation: The server sends an API request to obtain weather data for "Tokyo" (e.g., rainfall 50mm, temperature 25°C, humidity 60%) and integrates it with data from the device.

[0130] Step 4:

[0131] The data collected by the server is input into a multimodal AI system for analysis, which then determines optimal agricultural methods taking into account factors such as soil moisture, pH, predicted precipitation, and the stage of crop development.

[0132] Input: Integrated climate data, soil data, growth data

[0133] Output: Analysis results (e.g., irrigation required)

[0134] Specific operation: Multimodal AI analyzes the data and determines that irrigation is necessary because the humidity is low at 30%.

[0135] Step 5:

[0136] Based on the analysis, the server generates specific agricultural advice, including the timing and amount of irrigation and drainage measures after rainfall.

[0137] Input: Analysis results

[0138] Output: Farming advice (e.g., water 100 liters now)

[0139] Specific operation: The server generates the instruction "Please sprinkle 100 liters of water now" in text format and sends it to the terminal.

[0140] Step 6:

[0141] The terminal (robot) receives the advice sent from the server and performs the actual agricultural work based on the instructions, for example, activating the irrigation system and spraying a set amount of water.

[0142] Input: Advice from the server

[0143] Output: Agricultural work performed (e.g. irrigation)

[0144] Specific action: The robot starts the irrigation system and executes the command "spray 100 liters of water."

[0145] Through these steps, a system will be built that allows agricultural work to be carried out efficiently and automatically, supporting sustainable agriculture.

[0146] (Application example 1)

[0147] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0148] Current agricultural systems lack a system that consistently collects and analyzes environmental and crop data, and then proposes and implements optimal agricultural methods based on the results. Furthermore, users have limited means of accessing real-time information on their own devices, making it difficult to manage their farms quickly and effectively. This hinders agricultural efficiency and sustainability.

[0149] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0150] In this invention, the server includes means for collecting climate data, means for collecting soil analysis data, means for collecting crop growth status data, means for analyzing the above data using multimodal artificial intelligence, means for proposing optimal agricultural methods based on the analysis results, means for notifying users of the analysis results and agricultural method advice in real time, and means for controlling machinery that automates agricultural work on the farm based on the analysis results. This allows users to receive analysis results and advice in real time, enabling fast and effective agricultural management. Furthermore, the automation of agricultural work is expected to advance, leading to the realization of efficient and sustainable agriculture.

[0151] "Climate data" refers to data on weather conditions and environmental factors that affect crop growth, including temperature, humidity, precipitation, wind speed, etc.

[0152] "Soil analysis data" refers to data on the physical and chemical properties of agricultural soil, including pH, humidity, and nutrient concentration.

[0153] "Crop growth status data" refers to data that indicates the growth stage of a crop, including height, number of leaves, color, signs of disease, and the like.

[0154] "Multimodal AI" is an AI technology that integrates and analyzes multiple data modalities (e.g., climate data, soil data, image data, etc.).

[0155] "Analysis results" are the results of analysis generated by multimodal artificial intelligence and are information used to derive specific agricultural methods.

[0156] "Agricultural practices" are specific procedures and techniques used to optimize crop growth and increase yields.

[0157] "Users" are farmers and managers who use the system to manage and optimize agricultural operations.

[0158] "Real-time" refers to a situation in which data is collected and analyzed immediately, and the results are provided to users instantly.

[0159] "Advice" refers to recommendations for agricultural techniques and work provided to users based on analysis results from multimodal artificial intelligence.

[0160] A "personal digital assistant" is a portable electronic device, such as a smartphone or tablet, that allows a user to view information in real time.

[0161] "Machines that automate agricultural work" are devices that perform agricultural work automatically, such as tractors, irrigation systems, and harvesting robots.

[0162] A "central processing unit" is a computer system or server that analyzes collected data and generates optimal agricultural practices.

[0163] The agricultural support system of the present invention collects and analyzes weather data, soil analysis data, and data on the growth status of crops, proposes optimal agricultural methods, and executes those methods using automated machinery. Specific means for realizing this system will be described below.

[0164] 1. Data Collection Module

[0165] User: The user first inputs the type of crop they are growing and the location of their farm, which sets the basis for collecting and analyzing climate data.

[0166] Terminal: The terminal uses soil analysis sensors and crop growth monitoring cameras to periodically measure soil moisture, pH, and crop growth status. This acquired data is sent to the server in real time.

[0167] 2. Data transmission and analysis module

[0168] Server: The server obtains the necessary weather data from an external weather data API. It also receives soil data and crop growth data sent from the device, integrates them, and analyzes them using multimodal AI. This analysis derives optimal agricultural methods that take into account soil moisture, pH value, predicted precipitation, crop growth stage, and other factors.

[0169] 3. Advice Generation Module

[0170] Server: Based on the analysis results, the server generates advice to propose specific agricultural methods. For example, it recommends irrigation if soil moisture is low, or drainage measures if rainfall is forecast. The generated advice is sent to the user's mobile information terminal in real time.

[0171] 4. Robot Control Module

[0172] Terminal: Receives advice sent from the server and carries out the actual agricultural work based on that instruction. For example, if an instruction is given to irrigate, it will automatically water the area, and if drainage measures are required, it will activate the drainage system. This is done using automated machinery on the farm.

[0173] Specific examples

[0174] Let's take the example of an urban farmer growing tomatoes on a rooftop farm in a city.

[0175] 1. User: A user logs into the system and inputs that the crop being grown is "tomatoes" and that the farm is located "inside the city."

[0176] 2. Terminal: The terminal uses sensors to measure the moisture and pH of the soil, and a camera to record the tomato's growth stage. The collected data is sent to the server in real time.

[0177] 3. Server: The server retrieves the latest weather information for the city from an external weather data API, integrates it with soil humidity, acidity, and tomato growth status, and analyzes it. As a result of the analysis, it is determined that the soil humidity is low and that irrigation is necessary.

[0178] 4. Server: Generates irrigation advice and sends it to the user's mobile device in real time.

[0179] 5. Terminal: Receives advice from the server and automatically starts irrigation, providing the right amount of water to the tomatoes.

[0180] Prompt Sentence Examples

[0181] "Build an AI model that recommends irrigation when soil moisture is below 30% and no rain is forecast."

[0182] "Build an application that monitors the growth of tomatoes on an urban farm and suggests optimal farming practices."

[0183] This will enable users to receive analysis results and advice in real time, enabling quick and effective agricultural management. It is also expected that the automation of agricultural work will progress, leading to efficient and sustainable agriculture.

[0184] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0185] Step 1:

[0186] Users log in to the system and input the type of crop they are growing and the location of their farm, which then sets the criteria for collecting and analyzing climate data. The input is the type of crop and location, and the output is the set criteria.

[0187] Step 2:

[0188] The device uses soil analysis sensors and crop growth monitoring cameras to periodically measure soil moisture, pH, and crop growth. The measurement data is sent to the server in real time. The input is the measurement value of each sensor, and the output is data for analysis sent to the server.

[0189] Step 3:

[0190] The server acquires external weather data through the weather data API and integrates it with soil data and crop growth data sent from the terminal. The inputs are weather data, soil data, and crop growth data, and the output is the integrated dataset.

[0191] Step 4:

[0192] The server analyzes the integrated data using multimodal artificial intelligence to determine optimal agricultural practices, taking into account soil moisture, pH, predicted precipitation, crop growth stage, etc. The input is the integrated dataset, and the output is the analysis result (e.g., whether irrigation is required).

[0193] Step 5:

[0194] Based on the analysis results, the server generates advice to propose specific agricultural methods. For example, if the soil moisture is low, it recommends irrigation. The generated advice is sent to the user's mobile information terminal in real time. The input is the analysis results, and the output is the advice sent to the user.

[0195] Step 6:

[0196] The user's mobile information device receives the notification from the server and displays the analysis results and advice to the user. The user can then take the necessary action based on the displayed advice. The input is the notification from the server, and the output is the analysis results and advice presented to the user.

[0197] Step 7:

[0198] The terminal receives advice sent from the server and controls the automated machinery on the farm based on the instructions to carry out agricultural work. For example, if an irrigation command is issued, it will automatically water the farm, and if drainage measures are required, it will activate the drainage system. The input is the advice from the server, and the output is the agricultural work that has been carried out.

[0199] In this way, users can receive real-time advice on optimal farming techniques, leading to efficient and sustainable farming.

[0200] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0201] The agricultural support system of the present invention collects weather data, soil analysis data, and crop growth status data, analyzes this data using multimodal AI, proposes efficient and environmentally friendly agricultural methods, and executes these methods using automated robots. The present invention also incorporates an emotion engine that recognizes the user's emotions, and by utilizing the user's emotion data in the analysis, it is possible to provide more personalized advice. The configuration and operation of the system of the present invention are described in detail below.

[0202] System configuration

[0203] 1. Data Collection Module

[0204] Users log in to the system and enter the type of crop they are growing and the location of their farm, which determines the scope of data to be collected.

[0205] Terminal (robot): Using a soil analysis sensor and a crop growth monitoring camera, it collects data measuring soil humidity, pH value, and crop growth status. It also collects user emotion data through voice recognition and facial expression analysis.

[0206] 2. Data transmission and analysis module

[0207] Server: Receives soil data, crop growth data, and user emotion data sent from the device, and integrates it with weather data obtained from an external weather data API. Using multimodal AI, this data is analyzed to comprehensively assess soil condition, crop status, and weather conditions.

[0208] 3. Advice Generation Module

[0209] Server: Based on the results of multimodal AI analysis, the server generates advice proposing optimal farming methods. It also takes into account the user's emotional data and includes stress reduction measures and warnings as needed.

[0210] 4. Robot Control Module

[0211] Terminal (robot): Receives advice sent from the server and performs agricultural tasks based on the instructions. For example, if an instruction is given to irrigate, it will automatically water the land, and if an instruction is given to take drainage measures, it will activate the drainage system.

[0212] Specific examples

[0213] A specific example of using the system of the present invention will be described below.

[0214] Consider an urban farmer growing tomatoes on a rooftop farm in Tokyo.

[0215] 1. User: The user logs in to the system and inputs that the crop they are growing is "tomatoes" and that the farm is located in "Tokyo." The system also recognizes that the collected emotional data includes the user's voice and facial expressions while working.

[0216] 2. Terminal (Robot): The robot measures the soil moisture and pH value with sensors, records the tomato growth stage with a camera, and simultaneously collects emotional data from the user's voice and facial expressions.

[0217] 3. Server: The server retrieves the latest weather information for Tokyo from the weather data API, and then analyzes it together with soil data, crop growth data, and the user's emotional data. The analysis results indicate that the soil moisture is particularly low and that irrigation is necessary. The emotion engine also detects that the user is feeling stressed.

[0218] 4. Server: Based on the analysis results, generate irrigation advice along with relaxation activity recommendations, for example, "Perform irrigation and take a short break in between."

[0219] 5. Terminal (Robot): The robot starts irrigation and provides the appropriate amount of water. At the same time, a notification is displayed encouraging the user to take a break.

[0220] In this way, this system supports efficient and sustainable agriculture by integrating and analyzing environmental data and user emotional data to propose personalized agricultural methods.

[0221] The processing flow will be explained below.

[0222] Step 1:

[0223] User: Logs into the system and inputs the type of crop being grown (e.g., tomatoes) and the location of the farm (e.g., Tokyo). This sets up the basic information that allows the system to collect and analyze the appropriate data.

[0224] Step 2:

[0225] Terminal (robot): Using a soil analysis sensor, it measures the soil moisture and pH value in the farm. At the same time, it uses a crop growth monitoring camera to record the tomato growth status (e.g., height, number of leaves, color). In addition, it uses voice recognition and facial expression analysis sensors to collect emotional data from the user's voice and facial expressions. The collected data is sent to the server in real time.

[0226] Step 3:

[0227] Server: Obtains the latest weather data for Tokyo (temperature, humidity, precipitation, wind speed, etc.) from an external weather data API. This data is also analyzed along with other collected data.

[0228] Step 4:

[0229] Server: Integrates soil data, crop growth data, and user emotional data sent from the device. This data is input into the multimodal AI for analysis. Specifically, it determines the need for irrigation if the soil moisture is low, the stage of crop growth, and the user's stress level.

[0230] Step 5:

[0231] Server: Based on the analysis results, the server generates specific advice on agricultural techniques. For example, if the soil moisture is low, the server generates the advice "Please irrigate." If the user's emotional data indicates a high stress level, the server adds the recommendation "Please take a break to relax while irrigating."

[0232] Step 6:

[0233] Server: Sends advice to the robot, including specific work instructions (e.g., start irrigation, recommend breaks).

[0234] Step 7:

[0235] Terminal (Robot): Automatically performs agricultural tasks based on advice received from the server. For example, if an irrigation instruction is given, it will automatically start watering. It also displays a notification to the user to take a break to relax.

[0236] This series of steps allows users to easily practice efficient and sustainable farming while also optimally managing their own emotional state. The system aims to achieve both optimal farming practices and user emotional management by collecting and analyzing data in real time.

[0237] Example 2

[0238] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0239] While conventional agricultural support systems collect and analyze weather data, soil analysis data, and crop growth status data, they lack the ability to provide personalized advice or propose agricultural methods that take into account the user's emotional data. Furthermore, there are limited means for implementing automated agricultural work using the collected data, making it difficult to achieve efficient and sustainable agriculture.

[0240] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0241] In this invention, the server includes means for collecting and analyzing weather data, soil analysis data, crop growth status data, and user emotion data, means for generating optimal agricultural techniques and advice for the user on stress reduction measures and cautions based on the analysis results, and means for transmitting the generated advice to the agricultural machine and having the machine perform work based on the instructions. This enables comprehensive data analysis that incorporates user emotion data, making it possible to propose and implement efficient and personalized agricultural techniques.

[0242] "Climate data" refers to information about weather, temperature, humidity, precipitation, wind speed, and other meteorological conditions.

[0243] "Soil analysis data" refers to information about the physical and chemical properties of soil, such as soil moisture, pH, and nutrient content.

[0244] "Crop growth status data" refers to information about the state of growth of crops, such as the stage of growth of the crops, leaf color, and the presence or absence of pests or diseases.

[0245] "User emotion data" refers to information about the user's emotional state analyzed from their voice and facial expressions.

[0246] "Multimodal AI" refers to artificial intelligence that performs integrated analysis of multiple different types of data (e.g., text data, image data, audio data, etc.).

[0247] "Analysis results" refers to information obtained as a result of analysis conducted by AI based on collected data.

[0248] "Agricultural techniques" refer to methods and techniques for carrying out agricultural work efficiently.

[0249] "Agricultural machinery" means an automated machine used to perform agricultural work.

[0250] "Stress reduction measures" are suggestions and measures to reduce the user's stress.

[0251] A "warning" is a notification or warning that prompts the user to take some kind of action.

[0252] The agricultural support system according to the present invention is composed of multiple modules that work in conjunction with each other to provide efficient and personalized agricultural support. This system is made up of a data collection module, a data transmission and analysis module, an advice generation module, and a robot control module. Details and specific operations of each module are explained below.

[0253] Data Collection Module

[0254] User:

[0255] Users first log into the system and input the type of crop they are growing and the location of their farm, which sets the scope of data to be collected.

[0256] Terminal (Robot):

[0257] The device is equipped with a soil analysis sensor and a crop growth monitoring camera. The soil analysis sensor measures the soil's moisture and pH value, and the crop growth monitoring camera records the crop's growth status. It also collects user emotion data through voice recognition and facial expression analysis. All of this data is collected in real time.

[0258] Data Transmission and Analysis Module

[0259] Terminal (Robot):

[0260] The collected soil data, crop growth data, and user emotion data are sent to a server via wireless communication (e.g., Wi-Fi).

[0261] server:

[0262] The server acquires weather data from an external weather data API and integrates it with data sent from the device. Specifically, weather data, soil data, crop growth data, and user emotion data are all stored in a database and analyzed using multimodal AI.

[0263] Advice Generation Module

[0264] server:

[0265] Multimodal AI is used to comprehensively analyze data. The analysis integrates weather data, soil data, growth data, and emotional data to generate results. Based on the analysis results, advice is generated suggesting optimal farming methods. If the user is feeling stressed, advice on stress reduction measures and caution is also included.

[0266] For example, enter the following prompt into the server:

[0267] "Analyze the transmitted sensor and camera data and suggest optimal agricultural methods. If the soil moisture is below 60%, advise irrigation. Also, if the user is feeling stressed, provide advice on how to deal with it."

[0268] Robot Control Module

[0269] Terminal (Robot):

[0270] The system receives advice generated by the server and performs agricultural tasks based on the instructions. For example, it automatically waters the field when irrigation is requested, or activates the drainage system when drainage measures are requested. It also displays notifications encouraging the user to take a break.

[0271] A concrete example of the overall flow

[0272] Suppose an urban farmer is growing tomatoes on a rooftop farm in Tokyo. The user logs into the system and inputs that the crop is tomatoes and that the farm is in Tokyo. The terminal (robot) measures the humidity and pH value of the soil and records the tomato's growth stage with a camera. It also collects emotional data from voice and facial expressions. The server combines this data with weather data and produces an analysis result. Based on the analysis result, it determines that irrigation is necessary and suggests appropriate farming methods to the user. It also instructs the user to take a break if they are feeling stressed.

[0273] In this way, the system of the present invention comprehensively analyzes environmental data and user emotional data, and provides personalized agricultural methods, thereby supporting efficient and sustainable agriculture.

[0274] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0275] Step 1: Data entry

[0276] Users log in to the system and input the type of crop they are growing and the location of their farm. Based on this input data, the system sets the range of data to be collected and the required data items. Specifically, users access a dedicated application from their smartphone or PC and input the type of crop (e.g., tomatoes) and the location of their farm (e.g., Tokyo). The input data is sent to the server and saved in a configuration database.

[0277] Input: type of crop, farm location

[0278] Output: Configuration database updated

[0279] Step 2: Collect environmental data

[0280] The terminal (robot) uses soil analysis sensors and crop growth monitoring cameras to measure soil moisture, pH, and crop growth status. This data is collected periodically. The robot patrols the farm, measuring soil moisture and pH using the soil sensors and saving the results in its internal memory. At the same time, it records the crop growth stages using the monitoring cameras.

[0281] Input: None (data collected periodically)

[0282] Output: Measurement data (humidity, pH value, growth status)

[0283] Step 3: Collecting emotion data

[0284] The terminal (robot) analyzes the user's voice and facial expressions to collect emotional data. Specifically, when the robot is near the user, it activates a voice recognition system and facial expression analysis system. The words spoken and facial expressions of the user while working are analyzed in real time to generate emotional data.

[0285] Input: User's voice, facial expression video

[0286] Output: Emotion data

[0287] Step 4: Send data

[0288] The terminal (robot) transmits the collected soil data, crop growth data, and emotion data to the server via wireless communication (Wi-Fi). When the robot finishes collecting data, it automatically transfers the data to the server via the wireless communication module.

[0289] Input: Measurement data, emotion data

[0290] Output: Send data to the server

[0291] Step 5: Obtaining Weather Data

[0292] The server obtains the latest weather data from an external weather data API. Specifically, the server periodically sends requests to the API to obtain real-time weather data (temperature, humidity, precipitation, etc.). The obtained data is stored in an internal database.

[0293] Input: None (data acquired periodically)

[0294] Output: Weather data

[0295] Step 6: Data integration and analysis

[0296] The server integrates soil data, crop growth data, emotion data, and weather data sent from the device and analyzes them using multimodal AI.The server also extracts various data stored in the database and performs multidimensional analysis using deep learning models.

[0297] Input: soil data, crop growth data, emotion data, weather data

[0298] Output: Analysis results

[0299] Step 7: Advice Generation

[0300] The server generates optimal agricultural methods and advice for users based on the results of multimodal AI analysis. Based on the data derived from the analysis, it creates specific advice on irrigation and fertilizer application timing, pest control, and work efficiency. It also includes stress reduction measures and warnings based on the user's emotional data.

[0301] Specific prompt examples:

[0302] "Analyze the transmitted sensor and camera data and suggest optimal agricultural methods. If the soil moisture is below 60%, advise irrigation. Also, if the user is feeling stressed, provide advice on how to deal with it."

[0303] Input: Analysis results

[0304] Output: Advice content

[0305] Step 8: Submitting Advice

[0306] The server sends the generated advice to the terminal (robot), formats the advice into an appropriate format, and transmits it to the robot via wireless communication.

[0307] Input: Advice content

[0308] Output: Send advice to the robot

[0309] Step 9: Robot execution

[0310] The terminal (robot) performs agricultural work based on instructions from the server. The robot analyzes the advice it receives and begins the necessary agricultural work. For example, if the analysis results indicate that irrigation is necessary, the robot will activate the irrigation system and spray the appropriate amount of water. It will also display a notification encouraging the user to take a break.

[0311] Input: Advice content

[0312] Output: Execution of agricultural work, notification to user

[0313] (Application example 2)

[0314] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0315] In conventional production sites, environmental data and worker emotional data were not fully utilized, resulting in insufficient production efficiency and worker stress management. This made it difficult to find the optimal production method, which could have a negative impact on productivity and the work environment. The present invention aims to solve these problems by providing a system that comprehensively analyzes environmental data and worker emotional data and proposes the optimal production method.

[0316] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting weather data, means for collecting environmental data, means for collecting worker emotion data, means for analyzing the above data using multimodal AI, means for proposing an optimal production method based on the analysis results, and means for controlling equipment that automates the production process. This makes it possible to comprehensively analyze the collected data, improve production efficiency, and manage worker stress.

[0317] "Climate data" is information about weather conditions such as temperature, humidity, air pressure, precipitation, and wind speed.

[0318] "Environmental data" refers to information on various physical environments such as temperature, humidity, vibration, and noise within a factory or production site.

[0319] "Worker's emotional data" is information about the emotional state obtained by analyzing the worker's physiological responses such as voice and facial expressions.

[0320] "Multimodal AI" is an artificial intelligence technology that integrates and analyzes different types of data (e.g., voice, images, text, etc.).

[0321] "Production methods" refer to the procedures and methods of specific production processes and work processes.

[0322] "Equipment that automates production processes" refers to devices and systems that automate production activities within factories or production sites.

[0323] A "server" is a central computer in a system that analyzes and processes data, and stores and provides information.

[0324] The present invention is a system that collects weather data, environmental data, and worker emotion data at a production site, analyzes this data using multimodal AI, proposes optimal production methods, and automates the production process. Specific embodiments of the present invention are described in detail below.

[0325] System configuration

[0326] 1. Data Collection Module

[0327] The server is equipped with sensors and devices to collect weather and environmental data, as well as worker emotional data, including temperature, humidity, and vibration sensors, and an emotion recognition system that analyzes the voices and facial expressions of workers.

[0328] 2. Data transmission and analysis module

[0329] The collected data is sent to a server, which then integrates it and analyzes it using multimodal AI. This analysis uses various sensor APIs, emotion recognition APIs, and an HTTP request library.

[0330] 3. Advice Generation Module

[0331] Based on the analysis results, the server generates advice proposing optimal production methods. Specifically, if the temperature inside the factory is high, instructions are generated to activate the cooling system or encourage workers to take breaks.

[0332] 4. Robot Control Module

[0333] Upon receiving instructions from the server, the terminal (robot) performs automated tasks in the production process, such as operating the cooling system, adjusting humidity, and automatically transporting goods.

[0334] Hardware and software used

[0335] Hardware:

[0336] Temperature Sensor

[0337] Humidity Sensor

[0338] Vibration Sensor

[0339] Emotion Recognition System

[0340] Robots that carry out production processes

[0341] software:

[0342] Sensor API for collecting information

[0343] Emotion recognition API

[0344] HTTP request library

[0345] Multimodal AI analysis API

[0346] Specific examples

[0347] If environmental data on a factory production line indicates high temperature and humidity levels and that workers are feeling stressed, the system will activate the cooling system and send a notification to the worker's smartphone telling them to take a break. In this way, it is possible to maintain the health of workers without reducing production efficiency.

[0348] Prompt Sentence Examples

[0349] An optimization system using robots on factory production lines. Data is collected from temperature, humidity, and vibration sensors, as well as worker emotional data. Multimodal AI is used to analyze the data and generate optimal production methods and robot operation instructions. Instructions include activating the cooling system when temperatures are high, and notifications to encourage workers to take breaks.

[0350] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0351] Step 1:

[0352] Users log in to the system using a smartphone or head-mounted display and input information about the production line and the equipment to be managed. This allows the system to identify the target for monitoring and data collection. The input information includes the type of equipment, its installation location, and worker information. Based on this input data, the system starts operating the necessary sensors and devices.

[0353] Step 2:

[0354] The terminals (sensors) collect environmental data (temperature, humidity, vibration) within the factory and emotional data from workers' voices and facial expressions. Specifically, the temperature sensor detects the temperature on-site, the humidity sensor measures humidity, and the vibration sensor records the vibration state of the equipment. In addition, the emotion recognition system analyzes the workers' faces and voices to generate emotional data. The collected data is formatted and sent to a server in real time.

[0355] Step 3:

[0356] The server receives the environmental data and emotional data sent from the device, integrates them, and stores them in a database. Specifically, the server processes HTTP requests, retrieves data via various data APIs, and stores it. This allows the system to grasp the overall environmental situation and the worker's emotional state.

[0357] Step 4:

[0358] The server analyzes the data stored in the database using a multimodal AI model. Specifically, it receives temperature, humidity, vibration, and emotion data as input and analyzes the correlation between each piece of data. As a result of the analysis, it evaluates how specific environmental conditions and worker status affect production efficiency. The output is generated as optimal production methods and advice.

[0359] Step 5:

[0360] The generated production methods and advice are sent from the server to the terminal (robot). Specific instructions are transmitted to the robot using APIs and communication protocols. Examples include instructions to activate the cooling system if the temperature is high, or notifications to encourage workers to take a break if they are feeling stressed.

[0361] Step 6:

[0362] The terminals (robots) carry out production processes based on instructions received from the server. Specific operations include operating the cooling system, adjusting humidity, and transporting parts. This allows for efficient operation of the production line and optimization of the working environment.

[0363] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0364] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0365] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0366] [Second embodiment]

[0367] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0368] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0369] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0370] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0371] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0372] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0373] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0374] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0375] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0376] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0377] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0378] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0379] The agricultural support system of the present invention collects weather data, soil analysis data, and crop growth status data, analyzes this data using multimodal AI, proposes efficient and environmentally friendly agricultural methods, and executes these methods using automated robots. The system consists of the following main components:

[0380] 1. Data Collection Module

[0381] User: The user first inputs the type of crop they are growing and the location of their farm into the system, which sets the baseline for collecting and analyzing climate data.

[0382] Terminal (robot): The terminal uses soil analysis sensors and crop growth monitoring cameras to periodically measure soil moisture, pH, and crop growth status. This acquired data is sent to the server in real time.

[0383] 2. Data transmission and analysis module

[0384] Server: The server obtains the necessary climate data from an external weather data API. It also receives soil data and crop growth data sent from the device, integrates them, and analyzes them using multimodal AI. This analysis derives optimal agricultural methods that take into account soil moisture, pH value, predicted precipitation, crop growth stage, and other factors.

[0385] 3. Advice Generation Module

[0386] Server: Based on the analysis results, it generates advice to propose specific agricultural methods, for example, recommending irrigation if soil moisture is low, or drainage measures if rainfall is forecast.

[0387] 4. Robot Control Module

[0388] Terminal (robot): Receives advice sent from the server and performs actual agricultural work based on the instructions. For example, if an irrigation command is issued, it will automatically water the area, and if drainage measures are required, it will activate the drainage system.

[0389] Specific examples

[0390] Let's take the example of an urban farmer growing tomatoes on a rooftop farm in Tokyo.

[0391] 1. User: A user logs into the system and inputs that the crop being grown is "tomatoes" and that the farm's location is "Tokyo."

[0392] 2. Terminal (Robot): The robot measures the soil moisture and pH value with sensors and records the tomato growth stage with a camera. The collected data is sent to the server in real time.

[0393] 3. Server: The server retrieves the latest weather information for Tokyo from an external weather data API, integrates it with soil humidity, pH value, and tomato growth status, and analyzes it. As a result of the analysis, it is determined that the soil humidity is low and that irrigation is necessary.

[0394] 4. Server: Generates irrigation advice and sends it to the robot.

[0395] 5. Terminal (Robot): Receives advice from the server, automatically starts irrigation, and provides the tomatoes with the appropriate amount of water.

[0396] This system allows farmers to receive advice on optimal farming methods in real time, enabling them to practice sustainable farming without hassle, thereby supporting the realization of efficient and environmentally friendly agriculture.

[0397] The processing flow will be explained below.

[0398] Step 1:

[0399] User: The user logs into the system and inputs the type of crop they are growing (e.g., tomatoes) and the location of their farm (e.g., Tokyo). This information serves as the basis for subsequent data collection and analysis.

[0400] Step 2:

[0401] Terminal (Robot): The robot uses soil analysis sensors and growth monitoring cameras to measure the soil moisture, pH value, and crop growth status in the farm. The collected data is sent to the server in real time.

[0402] Step 3:

[0403] Server: The server obtains the latest climate data for Tokyo (temperature, humidity, precipitation, wind speed, etc.) from an external weather data API. This climate data is integrated as information required for analysis.

[0404] Step 4:

[0405] Server: The server integrates the soil data, crop growth data, and weather data sent from the terminal (robot), inputs it into a multimodal AI system, and performs analysis. This analysis derives optimal agricultural methods that take into account soil moisture, pH value, and the stage of crop growth.

[0406] Step 5:

[0407] Server: Generates specific agricultural advice based on the results of multimodal AI analysis, such as recommending irrigation if soil moisture is low or drainage measures if rainfall is forecast.

[0408] Step 6:

[0409] Server: Sends the generated agricultural advice to the terminal (robot). This advice includes specific operational instructions (e.g., start irrigation, activate drainage system, etc.).

[0410] Step 7:

[0411] Terminal (Robot): The robot automatically performs agricultural tasks based on advice received from the server. For example, if an instruction is given to irrigate, it starts supplying water and supplies the appropriate amount. If an instruction is given to take drainage measures, it activates the drainage system to remove excess water.

[0412] This series of steps enables users to practice efficient and sustainable agriculture, improving productivity while reducing environmental impact.

[0413] Example 1

[0414] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0415] In traditional agriculture, it is difficult to determine the optimal timing of work based on climate change and soil conditions, making it difficult to efficiently manage crop growth. Furthermore, manual management is labor-intensive and time-consuming, requiring highly accurate data analysis to achieve sustainable agriculture. While there is a growing need for automation using robots, current systems lack seamless integration from data collection to work execution.

[0416] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0417] In this invention, the server includes: a means for a user to input information about the type of crop and the location of the farm; a means for measuring the soil moisture and pH value and the growth status of the crop using a soil analysis sensor and a crop growth monitoring camera; a means for transmitting the above data to the server in real time; a means for acquiring climate data from an external weather data API; a means for analyzing the collected data using multimodal AI; a means for proposing optimal agricultural techniques based on the analysis results; and a means for controlling a robot that automatically performs agricultural work on the farm based on the advice transmitted from the server. This enables the automation and optimization of agricultural work.

[0418] A "user" is an entity that inputs information into the system and provides data, such as a farm manager or agricultural worker.

[0419] "Crop type" refers to the particular type or variety of agricultural crop being grown.

[0420] "Farm location information" refers to information about the geographic location of the farm, and specifically includes data such as coordinates and address.

[0421] A "soil analysis sensor" is a device for measuring chemical and physical properties of soil, such as moisture and pH value.

[0422] A "crop growth monitoring camera" is a camera device for visually recording the growth status of cultivated crops.

[0423] "Real-time" refers to data being processed and transmitted immediately, with almost no time lag.

[0424] A "server" is a central computer system that processes, analyzes, stores, and communicates with other system components.

[0425] "External Weather Data API" means an external application programming interface that provides weather data over the Internet.

[0426] "Climate Data" means information about current and forecast weather conditions in a particular geographic area, such as precipitation, temperature, and humidity.

[0427] "Multimodal AI" is an artificial intelligence technology that integrates and analyzes multiple different types of data (e.g., climate data, soil data, video data).

[0428] "Analysis Results" refers to information or conclusions obtained as a result of analysis conducted based on collected data.

[0429] "Agricultural methods" refer to the specific work and technical means used in growing crops.

[0430] A "robot" is a mechanical device that automatically performs agricultural tasks within a farm.

[0431] "Control" refers to a system issuing instructions to a robot or other device and managing and operating its operations.

[0432] "Advice" refers to optimal farming techniques and specific work instructions provided based on the analysis results.

[0433] The agricultural support system of the present invention collects weather data, soil analysis data, and crop growth status data and analyzes them using multimodal AI to automate and optimize agricultural work. Based on the analysis results, it proposes optimal agricultural methods and executes those methods using automated robots. The main components of this system are described below.

[0434] Data Entry and Collection

[0435] First, users input the type of crop they are growing and the location of their farm. This information is used to set the criteria for collecting and analyzing climate data. For example, entering "tomatoes" and "Tokyo" will generate an analysis based on Tokyo's climate conditions.

[0436] Next, the terminal (robot) uses a soil analysis sensor and a crop growth monitoring camera to periodically measure the soil humidity, pH value, and crop growth status. Specifically, the soil sensor measures humidity, and the camera records the growth status. This data is sent to the server in real time.

[0437] Data Transmission and Integration

[0438] The server receives soil data and crop growth data sent from the device and obtains necessary climate data from an external weather data API. For example, it integrates information such as predicted rainfall, temperature, and humidity and analyzes it together with the soil and crop growth data.

[0439] Data analysis

[0440] The server inputs the collected weather, soil, and crop growth data into the multimodal AI for analysis, which then derives optimal agricultural methods that take into account soil moisture, pH, predicted precipitation, and the stage of crop growth. Specifically, the AI ​​determines that soil moisture is low and that irrigation is necessary.

[0441] Advice Generation

[0442] Based on the analysis results, the server generates agricultural advice, including the timing and amount of irrigation and how to drain the fields after rainfall. For example, specific instructions such as "apply 100 liters of water immediately" are generated and sent to the device.

[0443] Robot Control and Execution

[0444] The terminal (robot) receives the advice sent from the server and performs the actual agricultural work based on the instructions. Specifically, the robot activates the irrigation system and sprays the set amount of water. In this way, agricultural work is carried out automatically and efficiently.

[0445] Specific example explanation

[0446] Let's take the example of an urban farmer growing tomatoes on a rooftop farm in Tokyo. A user logs into the system and enters that the crop being grown is "tomatoes" and that the farm is located in "Tokyo." The robot uses sensors to measure the moisture and pH value of the soil and a camera to record the tomato growth stage. The collected data is sent to a server in real time.

[0447] The server obtains the latest weather information for Tokyo from an external weather data API, and performs analysis by integrating it with soil humidity, pH value, and tomato growth status. The analysis results show that the soil humidity is low and determines that irrigation is necessary. The server generates advice recommending irrigation and sends it to the robot. The robot receives the advice from the server, automatically starts irrigation, and supplies the tomatoes with the appropriate amount of water.

[0448] Prompt Sentence Examples

[0449] "What are the main features of this system?"

[0450] How should users enter data?

[0451] "Please explain in detail how the data collection module works."

[0452] "Please tell me how this system suggests agricultural methods."

[0453] Please explain the flow of this system using a specific example.

[0454] This system allows farmers to receive advice on optimal farming methods in real time, enabling them to practice sustainable farming without hassle, thereby supporting the realization of efficient and environmentally friendly agriculture.

[0455] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0456] Step 1:

[0457] A user logs into the system and enters the type of crop they are growing (e.g., tomatoes) and the location of their farm (e.g., Tokyo).

[0458] Input: type of crop, farm location

[0459] Output: Reference climate data acquisition conditions

[0460] Specific operation: The user enters "tomato" and "Tokyo" through the system interface, which is then sent to the server.

[0461] Step 2:

[0462] The terminal (robot) uses a soil analysis sensor to measure the moisture and pH value of the soil, and a crop growth monitoring camera to capture images of the crop growth. This data is sent to a server in real time.

[0463] Input: Measurements from sensors and cameras

[0464] Output: Measurement data (humidity, pH value, growth status images)

[0465] How it works: The robot's soil sensors measure humidity at 30% and pH at 6.5, and its camera captures images of the tomato's growth stages, all of which are sent to a server.

[0466] Step 3:

[0467] The server accesses an external weather data API to obtain the latest weather data (e.g., predicted rainfall, temperature, and humidity) based on the farm's location, and also receives soil and crop growth data sent from the device.

[0468] Input: Farm location, soil data, growth data

[0469] Output: Integrated climate and farm data

[0470] Specific operation: The server sends an API request to obtain weather data for "Tokyo" (e.g., rainfall 50mm, temperature 25°C, humidity 60%) and integrates it with data from the device.

[0471] Step 4:

[0472] The data collected by the server is input into a multimodal AI system for analysis, which then determines optimal agricultural methods taking into account factors such as soil moisture, pH, predicted precipitation, and the stage of crop development.

[0473] Input: Integrated climate data, soil data, growth data

[0474] Output: Analysis results (e.g., irrigation required)

[0475] Specific operation: Multimodal AI analyzes the data and determines that irrigation is necessary because the humidity is low at 30%.

[0476] Step 5:

[0477] Based on the analysis, the server generates specific agricultural advice, including the timing and amount of irrigation and drainage measures after rainfall.

[0478] Input: Analysis results

[0479] Output: Farming advice (e.g., water 100 liters now)

[0480] Specific operation: The server generates the instruction "Please sprinkle 100 liters of water now" in text format and sends it to the terminal.

[0481] Step 6:

[0482] The terminal (robot) receives the advice sent from the server and performs the actual agricultural work based on the instructions, for example, activating the irrigation system and spraying a set amount of water.

[0483] Input: Advice from the server

[0484] Output: Agricultural work performed (e.g. irrigation)

[0485] Specific action: The robot starts the irrigation system and executes the command "spray 100 liters of water."

[0486] Through these steps, a system will be built that allows agricultural work to be carried out efficiently and automatically, supporting sustainable agriculture.

[0487] (Application example 1)

[0488] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0489] Current agricultural systems lack a system that consistently collects and analyzes environmental and crop data, and then proposes and implements optimal agricultural methods based on the results. Furthermore, users have limited means of accessing real-time information on their own devices, making it difficult to manage their farms quickly and effectively. This hinders agricultural efficiency and sustainability.

[0490] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0491] In this invention, the server includes means for collecting climate data, means for collecting soil analysis data, means for collecting crop growth status data, means for analyzing the above data using multimodal artificial intelligence, means for proposing optimal agricultural methods based on the analysis results, means for notifying users of the analysis results and agricultural method advice in real time, and means for controlling machinery that automates agricultural work on the farm based on the analysis results. This allows users to receive analysis results and advice in real time, enabling fast and effective agricultural management. Furthermore, the automation of agricultural work is expected to advance, leading to the realization of efficient and sustainable agriculture.

[0492] "Climate data" refers to data on weather conditions and environmental factors that affect crop growth, including temperature, humidity, precipitation, wind speed, etc.

[0493] "Soil analysis data" refers to data on the physical and chemical properties of agricultural soil, including pH, humidity, and nutrient concentration.

[0494] "Crop growth status data" refers to data that indicates the growth stage of a crop, including height, number of leaves, color, signs of disease, and the like.

[0495] "Multimodal AI" is an AI technology that integrates and analyzes multiple data modalities (e.g., climate data, soil data, image data, etc.).

[0496] "Analysis results" are the results of analysis generated by multimodal artificial intelligence and are information used to derive specific agricultural methods.

[0497] "Agricultural practices" are specific procedures and techniques used to optimize crop growth and increase yields.

[0498] "Users" are farmers and managers who use the system to manage and optimize agricultural operations.

[0499] "Real-time" refers to a situation in which data is collected and analyzed immediately, and the results are provided to users instantly.

[0500] "Advice" refers to recommendations for agricultural techniques and work provided to users based on analysis results from multimodal artificial intelligence.

[0501] A "personal digital assistant" is a portable electronic device, such as a smartphone or tablet, that allows a user to view information in real time.

[0502] "Machines that automate agricultural work" are devices that perform agricultural work automatically, such as tractors, irrigation systems, and harvesting robots.

[0503] A "central processing unit" is a computer system or server that analyzes collected data and generates optimal agricultural practices.

[0504] The agricultural support system of the present invention collects and analyzes weather data, soil analysis data, and data on the growth status of crops, proposes optimal agricultural methods, and executes those methods using automated machinery. Specific means for realizing this system will be described below.

[0505] 1. Data Collection Module

[0506] User: The user first inputs the type of crop they are growing and the location of their farm, which sets the basis for collecting and analyzing climate data.

[0507] Terminal: The terminal uses soil analysis sensors and crop growth monitoring cameras to periodically measure soil moisture, pH, and crop growth status. This acquired data is sent to the server in real time.

[0508] 2. Data transmission and analysis module

[0509] Server: The server obtains the necessary weather data from an external weather data API. It also receives soil data and crop growth data sent from the device, integrates them, and analyzes them using multimodal AI. This analysis derives optimal agricultural methods that take into account soil moisture, pH value, predicted precipitation, crop growth stage, and other factors.

[0510] 3. Advice Generation Module

[0511] Server: Based on the analysis results, the server generates advice to propose specific agricultural methods. For example, it recommends irrigation if soil moisture is low, or drainage measures if rainfall is forecast. The generated advice is sent to the user's mobile information terminal in real time.

[0512] 4. Robot Control Module

[0513] Terminal: Receives advice sent from the server and carries out the actual agricultural work based on that instruction. For example, if an instruction is given to irrigate, it will automatically water the area, and if drainage measures are required, it will activate the drainage system. This is done using automated machinery on the farm.

[0514] Specific examples

[0515] Let's take the example of an urban farmer growing tomatoes on a rooftop farm in a city.

[0516] 1. User: A user logs into the system and inputs that the crop being grown is "tomatoes" and that the farm is located "inside the city."

[0517] 2. Terminal: The terminal uses sensors to measure the moisture and pH of the soil, and a camera to record the tomato's growth stage. The collected data is sent to the server in real time.

[0518] 3. Server: The server retrieves the latest weather information for the city from an external weather data API, integrates it with soil humidity, acidity, and tomato growth status, and analyzes it. As a result of the analysis, it is determined that the soil humidity is low and that irrigation is necessary.

[0519] 4. Server: Generates irrigation advice and sends it to the user's mobile device in real time.

[0520] 5. Terminal: Receives advice from the server and automatically starts irrigation, providing the right amount of water to the tomatoes.

[0521] Prompt Sentence Examples

[0522] "Build an AI model that recommends irrigation when soil moisture is below 30% and no rain is forecast."

[0523] "Build an application that monitors the growth of tomatoes on an urban farm and suggests optimal farming practices."

[0524] This will enable users to receive analysis results and advice in real time, enabling quick and effective agricultural management. It is also expected that the automation of agricultural work will progress, leading to efficient and sustainable agriculture.

[0525] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0526] Step 1:

[0527] Users log in to the system and input the type of crop they are growing and the location of their farm, which then sets the criteria for collecting and analyzing climate data. The input is the type of crop and location, and the output is the set criteria.

[0528] Step 2:

[0529] The device uses soil analysis sensors and crop growth monitoring cameras to periodically measure soil moisture, pH, and crop growth. The measurement data is sent to the server in real time. The input is the measurement value of each sensor, and the output is data for analysis sent to the server.

[0530] Step 3:

[0531] The server acquires external weather data through the weather data API and integrates it with soil data and crop growth data sent from the terminal. The inputs are weather data, soil data, and crop growth data, and the output is the integrated dataset.

[0532] Step 4:

[0533] The server analyzes the integrated data using multimodal artificial intelligence to determine optimal agricultural practices, taking into account soil moisture, pH, predicted precipitation, crop growth stage, etc. The input is the integrated dataset, and the output is the analysis result (e.g., whether irrigation is required).

[0534] Step 5:

[0535] Based on the analysis results, the server generates advice to propose specific agricultural methods. For example, if the soil moisture is low, it recommends irrigation. The generated advice is sent to the user's mobile information terminal in real time. The input is the analysis results, and the output is the advice sent to the user.

[0536] Step 6:

[0537] The user's mobile information device receives the notification from the server and displays the analysis results and advice to the user. The user can then take the necessary action based on the displayed advice. The input is the notification from the server, and the output is the analysis results and advice presented to the user.

[0538] Step 7:

[0539] The terminal receives advice sent from the server and controls the automated machinery on the farm based on the instructions to carry out agricultural work. For example, if an irrigation command is issued, it will automatically water the farm, and if drainage measures are required, it will activate the drainage system. The input is the advice from the server, and the output is the agricultural work that has been carried out.

[0540] In this way, users can receive real-time advice on optimal farming techniques, leading to efficient and sustainable farming.

[0541] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0542] The agricultural support system of the present invention collects weather data, soil analysis data, and crop growth status data, analyzes this data using multimodal AI, proposes efficient and environmentally friendly agricultural methods, and executes these methods using automated robots. The present invention also incorporates an emotion engine that recognizes the user's emotions, and by utilizing the user's emotion data in the analysis, it is possible to provide more personalized advice. The configuration and operation of the system of the present invention are described in detail below.

[0543] System configuration

[0544] 1. Data Collection Module

[0545] Users log in to the system and enter the type of crop they are growing and the location of their farm, which determines the scope of data to be collected.

[0546] Terminal (robot): Using a soil analysis sensor and a crop growth monitoring camera, it collects data measuring soil humidity, pH value, and crop growth status. It also collects user emotion data through voice recognition and facial expression analysis.

[0547] 2. Data transmission and analysis module

[0548] Server: Receives soil data, crop growth data, and user emotion data sent from the device, and integrates it with weather data obtained from an external weather data API. Using multimodal AI, this data is analyzed to comprehensively assess soil condition, crop status, and weather conditions.

[0549] 3. Advice Generation Module

[0550] Server: Based on the results of multimodal AI analysis, the server generates advice proposing optimal farming methods. It also takes into account the user's emotional data and includes stress reduction measures and warnings as needed.

[0551] 4. Robot Control Module

[0552] Terminal (robot): Receives advice sent from the server and performs agricultural tasks based on the instructions. For example, if an instruction is given to irrigate, it will automatically water the land, and if an instruction is given to take drainage measures, it will activate the drainage system.

[0553] Specific examples

[0554] A specific example of using the system of the present invention will be described below.

[0555] Consider an urban farmer growing tomatoes on a rooftop farm in Tokyo.

[0556] 1. User: The user logs in to the system and inputs that the crop they are growing is "tomatoes" and that the farm is located in "Tokyo." The system also recognizes that the collected emotional data includes the user's voice and facial expressions while working.

[0557] 2. Terminal (Robot): The robot measures the soil moisture and pH value with sensors, records the tomato growth stage with a camera, and simultaneously collects emotional data from the user's voice and facial expressions.

[0558] 3. Server: The server retrieves the latest weather information for Tokyo from the weather data API, and then analyzes it together with soil data, crop growth data, and the user's emotional data. The analysis results indicate that the soil moisture is particularly low and that irrigation is necessary. The emotion engine also detects that the user is feeling stressed.

[0559] 4. Server: Based on the analysis results, generate irrigation advice along with relaxation activity recommendations, for example, "Perform irrigation and take a short break in between."

[0560] 5. Terminal (Robot): The robot starts irrigation and provides the appropriate amount of water. At the same time, a notification is displayed encouraging the user to take a break.

[0561] In this way, this system supports efficient and sustainable agriculture by integrating and analyzing environmental data and user emotional data to propose personalized agricultural methods.

[0562] The processing flow will be explained below.

[0563] Step 1:

[0564] User: Logs into the system and inputs the type of crop being grown (e.g., tomatoes) and the location of the farm (e.g., Tokyo). This sets up the basic information that allows the system to collect and analyze the appropriate data.

[0565] Step 2:

[0566] Terminal (robot): Using a soil analysis sensor, it measures the soil moisture and pH value in the farm. At the same time, it uses a crop growth monitoring camera to record the tomato growth status (e.g., height, number of leaves, color). In addition, it uses voice recognition and facial expression analysis sensors to collect emotional data from the user's voice and facial expressions. The collected data is sent to the server in real time.

[0567] Step 3:

[0568] Server: Obtains the latest weather data for Tokyo (temperature, humidity, precipitation, wind speed, etc.) from an external weather data API. This data is also analyzed along with other collected data.

[0569] Step 4:

[0570] Server: Integrates soil data, crop growth data, and user emotional data sent from the device. This data is input into the multimodal AI for analysis. Specifically, it determines the need for irrigation if the soil moisture is low, the stage of crop growth, and the user's stress level.

[0571] Step 5:

[0572] Server: Based on the analysis results, the server generates specific advice on agricultural techniques. For example, if the soil moisture is low, the server generates the advice "Please irrigate." If the user's emotional data indicates a high stress level, the server adds the recommendation "Please take a break to relax while irrigating."

[0573] Step 6:

[0574] Server: Sends advice to the robot, including specific work instructions (e.g., start irrigation, recommend breaks).

[0575] Step 7:

[0576] Terminal (Robot): Automatically performs agricultural tasks based on advice received from the server. For example, if an irrigation instruction is given, it will automatically start watering. It also displays a notification to the user to take a break to relax.

[0577] This series of steps allows users to easily practice efficient and sustainable farming while also optimally managing their own emotional state. The system aims to achieve both optimal farming practices and user emotional management by collecting and analyzing data in real time.

[0578] Example 2

[0579] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0580] While conventional agricultural support systems collect and analyze weather data, soil analysis data, and crop growth status data, they lack the ability to provide personalized advice or propose agricultural methods that take into account the user's emotional data. Furthermore, there are limited means for implementing automated agricultural work using the collected data, making it difficult to achieve efficient and sustainable agriculture.

[0581] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0582] In this invention, the server includes means for collecting and analyzing weather data, soil analysis data, crop growth status data, and user emotion data, means for generating optimal agricultural techniques and advice for the user on stress reduction measures and cautions based on the analysis results, and means for transmitting the generated advice to the agricultural machine and having the machine perform work based on the instructions. This enables comprehensive data analysis that incorporates user emotion data, making it possible to propose and implement efficient and personalized agricultural techniques.

[0583] "Climate data" refers to information about weather, temperature, humidity, precipitation, wind speed, and other meteorological conditions.

[0584] "Soil analysis data" refers to information about the physical and chemical properties of soil, such as soil moisture, pH, and nutrient content.

[0585] "Crop growth status data" refers to information about the state of growth of crops, such as the stage of growth of the crops, leaf color, and the presence or absence of pests or diseases.

[0586] "User emotion data" refers to information about the user's emotional state analyzed from their voice and facial expressions.

[0587] "Multimodal AI" refers to artificial intelligence that performs integrated analysis of multiple different types of data (e.g., text data, image data, audio data, etc.).

[0588] "Analysis results" refers to information obtained as a result of analysis conducted by AI based on collected data.

[0589] "Agricultural techniques" refer to methods and techniques for carrying out agricultural work efficiently.

[0590] "Agricultural machinery" means an automated machine used to perform agricultural work.

[0591] "Stress reduction measures" are suggestions and measures to reduce the user's stress.

[0592] A "warning" is a notification or warning that prompts the user to take some kind of action.

[0593] The agricultural support system according to the present invention is composed of multiple modules that work in conjunction with each other to provide efficient and personalized agricultural support. This system is made up of a data collection module, a data transmission and analysis module, an advice generation module, and a robot control module. Details and specific operations of each module are explained below.

[0594] Data Collection Module

[0595] User:

[0596] Users first log into the system and input the type of crop they are growing and the location of their farm, which sets the scope of data to be collected.

[0597] Terminal (Robot):

[0598] The device is equipped with a soil analysis sensor and a crop growth monitoring camera. The soil analysis sensor measures the soil's moisture and pH value, and the crop growth monitoring camera records the crop's growth status. It also collects user emotion data through voice recognition and facial expression analysis. All of this data is collected in real time.

[0599] Data Transmission and Analysis Module

[0600] Terminal (Robot):

[0601] The collected soil data, crop growth data, and user emotion data are sent to a server via wireless communication (e.g., Wi-Fi).

[0602] server:

[0603] The server acquires weather data from an external weather data API and integrates it with data sent from the device. Specifically, weather data, soil data, crop growth data, and user emotion data are all stored in a database and analyzed using multimodal AI.

[0604] Advice Generation Module

[0605] server:

[0606] Multimodal AI is used to comprehensively analyze data. The analysis integrates weather data, soil data, growth data, and emotional data to generate results. Based on the analysis results, advice is generated suggesting optimal farming methods. If the user is feeling stressed, advice on stress reduction measures and caution is also included.

[0607] For example, enter the following prompt into the server:

[0608] "Analyze the transmitted sensor and camera data and suggest optimal agricultural methods. If the soil moisture is below 60%, advise irrigation. Also, if the user is feeling stressed, provide advice on how to deal with it."

[0609] Robot Control Module

[0610] Terminal (Robot):

[0611] The system receives advice generated by the server and performs agricultural tasks based on the instructions. For example, it automatically waters the field when irrigation is requested, or activates the drainage system when drainage measures are requested. It also displays notifications encouraging the user to take a break.

[0612] A concrete example of the overall flow

[0613] Suppose an urban farmer is growing tomatoes on a rooftop farm in Tokyo. The user logs into the system and inputs that the crop is tomatoes and that the farm is in Tokyo. The terminal (robot) measures the humidity and pH value of the soil and records the tomato's growth stage with a camera. It also collects emotional data from voice and facial expressions. The server combines this data with weather data and produces an analysis result. Based on the analysis result, it determines that irrigation is necessary and suggests appropriate farming methods to the user. It also instructs the user to take a break if they are feeling stressed.

[0614] In this way, the system of the present invention comprehensively analyzes environmental data and user emotional data, and provides personalized agricultural methods, thereby supporting efficient and sustainable agriculture.

[0615] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0616] Step 1: Data entry

[0617] Users log in to the system and input the type of crop they are growing and the location of their farm. Based on this input data, the system sets the range of data to be collected and the required data items. Specifically, users access a dedicated application from their smartphone or PC and input the type of crop (e.g., tomatoes) and the location of their farm (e.g., Tokyo). The input data is sent to the server and saved in a configuration database.

[0618] Input: type of crop, farm location

[0619] Output: Configuration database updated

[0620] Step 2: Collect environmental data

[0621] The terminal (robot) uses soil analysis sensors and crop growth monitoring cameras to measure soil moisture, pH, and crop growth status. This data is collected periodically. The robot patrols the farm, measuring soil moisture and pH using the soil sensors and saving the results in its internal memory. At the same time, it records the crop growth stages using the monitoring cameras.

[0622] Input: None (data collected periodically)

[0623] Output: Measurement data (humidity, pH value, growth status)

[0624] Step 3: Collecting emotion data

[0625] The terminal (robot) analyzes the user's voice and facial expressions to collect emotional data. Specifically, when the robot is near the user, it activates a voice recognition system and facial expression analysis system. The words spoken and facial expressions of the user while working are analyzed in real time to generate emotional data.

[0626] Input: User's voice, facial expression video

[0627] Output: Emotion data

[0628] Step 4: Send data

[0629] The terminal (robot) transmits the collected soil data, crop growth data, and emotion data to the server via wireless communication (Wi-Fi). When the robot finishes collecting data, it automatically transfers the data to the server via the wireless communication module.

[0630] Input: Measurement data, emotion data

[0631] Output: Send data to the server

[0632] Step 5: Obtaining Weather Data

[0633] The server obtains the latest weather data from an external weather data API. Specifically, the server periodically sends requests to the API to obtain real-time weather data (temperature, humidity, precipitation, etc.). The obtained data is stored in an internal database.

[0634] Input: None (data acquired periodically)

[0635] Output: Weather data

[0636] Step 6: Data integration and analysis

[0637] The server integrates soil data, crop growth data, emotion data, and weather data sent from the device and analyzes them using multimodal AI.The server also extracts various data stored in the database and performs multidimensional analysis using deep learning models.

[0638] Input: soil data, crop growth data, emotion data, weather data

[0639] Output: Analysis results

[0640] Step 7: Advice Generation

[0641] The server generates optimal agricultural methods and advice for users based on the results of multimodal AI analysis. Based on the data derived from the analysis, it creates specific advice on irrigation and fertilizer application timing, pest control, and work efficiency. It also includes stress reduction measures and warnings based on the user's emotional data.

[0642] Specific prompt examples:

[0643] "Analyze the transmitted sensor and camera data and suggest optimal agricultural methods. If the soil moisture is below 60%, advise irrigation. Also, if the user is feeling stressed, provide advice on how to deal with it."

[0644] Input: Analysis results

[0645] Output: Advice content

[0646] Step 8: Submitting Advice

[0647] The server sends the generated advice to the terminal (robot), formats the advice into an appropriate format, and transmits it to the robot via wireless communication.

[0648] Input: Advice content

[0649] Output: Send advice to the robot

[0650] Step 9: Robot execution

[0651] The terminal (robot) performs agricultural work based on instructions from the server. The robot analyzes the advice it receives and begins the necessary agricultural work. For example, if the analysis results indicate that irrigation is necessary, the robot will activate the irrigation system and spray the appropriate amount of water. It will also display a notification encouraging the user to take a break.

[0652] Input: Advice content

[0653] Output: Execution of agricultural work, notification to user

[0654] (Application example 2)

[0655] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0656] In conventional production sites, environmental data and worker emotional data were not fully utilized, resulting in insufficient production efficiency and worker stress management. This made it difficult to find the optimal production method, which could have a negative impact on productivity and the work environment. The present invention aims to solve these problems by providing a system that comprehensively analyzes environmental data and worker emotional data and proposes the optimal production method.

[0657] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting weather data, means for collecting environmental data, means for collecting worker emotion data, means for analyzing the above data using multimodal AI, means for proposing an optimal production method based on the analysis results, and means for controlling equipment that automates the production process. This makes it possible to comprehensively analyze the collected data, improve production efficiency, and manage worker stress.

[0658] "Climate data" is information about weather conditions such as temperature, humidity, air pressure, precipitation, and wind speed.

[0659] "Environmental data" refers to information on various physical environments such as temperature, humidity, vibration, and noise within a factory or production site.

[0660] "Worker's emotional data" is information about the emotional state obtained by analyzing the worker's physiological responses such as voice and facial expressions.

[0661] "Multimodal AI" is an artificial intelligence technology that integrates and analyzes different types of data (e.g., voice, images, text, etc.).

[0662] "Production methods" refer to the procedures and methods of specific production processes and work processes.

[0663] "Equipment that automates production processes" refers to devices and systems that automate production activities within factories or production sites.

[0664] A "server" is a central computer in a system that analyzes and processes data, and stores and provides information.

[0665] The present invention is a system that collects weather data, environmental data, and worker emotion data at a production site, analyzes this data using multimodal AI, proposes optimal production methods, and automates the production process. Specific embodiments of the present invention are described in detail below.

[0666] System configuration

[0667] 1. Data Collection Module

[0668] The server is equipped with sensors and devices to collect weather and environmental data, as well as worker emotional data, including temperature, humidity, and vibration sensors, and an emotion recognition system that analyzes the voices and facial expressions of workers.

[0669] 2. Data transmission and analysis module

[0670] The collected data is sent to a server, which then integrates it and analyzes it using multimodal AI. This analysis uses various sensor APIs, emotion recognition APIs, and an HTTP request library.

[0671] 3. Advice Generation Module

[0672] Based on the analysis results, the server generates advice proposing optimal production methods. Specifically, if the temperature inside the factory is high, instructions are generated to activate the cooling system or encourage workers to take breaks.

[0673] 4. Robot Control Module

[0674] Upon receiving instructions from the server, the terminal (robot) performs automated tasks in the production process, such as operating the cooling system, adjusting humidity, and automatically transporting goods.

[0675] Hardware and software used

[0676] Hardware:

[0677] Temperature Sensor

[0678] Humidity Sensor

[0679] Vibration Sensor

[0680] Emotion Recognition System

[0681] Robots that carry out production processes

[0682] software:

[0683] Sensor API for collecting information

[0684] Emotion recognition API

[0685] HTTP request library

[0686] Multimodal AI analysis API

[0687] Specific examples

[0688] If environmental data on a factory production line indicates high temperature and humidity levels and that workers are feeling stressed, the system will activate the cooling system and send a notification to the worker's smartphone telling them to take a break. In this way, it is possible to maintain the health of workers without reducing production efficiency.

[0689] Prompt Sentence Examples

[0690] An optimization system using robots on factory production lines. Data is collected from temperature, humidity, and vibration sensors, as well as worker emotional data. Multimodal AI is used to analyze the data and generate optimal production methods and robot operation instructions. Instructions include activating the cooling system when temperatures are high, and notifications to encourage workers to take breaks.

[0691] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0692] Step 1:

[0693] Users log in to the system using a smartphone or head-mounted display and input information about the production line and the equipment to be managed. This allows the system to identify the target for monitoring and data collection. The input information includes the type of equipment, its installation location, and worker information. Based on this input data, the system starts operating the necessary sensors and devices.

[0694] Step 2:

[0695] The terminals (sensors) collect environmental data (temperature, humidity, vibration) within the factory and emotional data from workers' voices and facial expressions. Specifically, the temperature sensor detects the temperature on-site, the humidity sensor measures humidity, and the vibration sensor records the vibration state of the equipment. In addition, the emotion recognition system analyzes the workers' faces and voices to generate emotional data. The collected data is formatted and sent to a server in real time.

[0696] Step 3:

[0697] The server receives the environmental data and emotional data sent from the device, integrates them, and stores them in a database. Specifically, the server processes HTTP requests, retrieves data via various data APIs, and stores it. This allows the system to grasp the overall environmental situation and the worker's emotional state.

[0698] Step 4:

[0699] The server analyzes the data stored in the database using a multimodal AI model. Specifically, it receives temperature, humidity, vibration, and emotion data as input and analyzes the correlation between each piece of data. As a result of the analysis, it evaluates how specific environmental conditions and worker status affect production efficiency. The output is generated as optimal production methods and advice.

[0700] Step 5:

[0701] The generated production methods and advice are sent from the server to the terminal (robot). Specific instructions are transmitted to the robot using APIs and communication protocols. Examples include instructions to activate the cooling system if the temperature is high, or notifications to encourage workers to take a break if they are feeling stressed.

[0702] Step 6:

[0703] The terminals (robots) carry out production processes based on instructions received from the server. Specific operations include operating the cooling system, adjusting humidity, and transporting parts. This allows for efficient operation of the production line and optimization of the working environment.

[0704] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0705] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0706] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0707] [Third embodiment]

[0708] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0709] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0710] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0711] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0712] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0713] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0714] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0715] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0716] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0717] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0718] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0719] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0720] The agricultural support system of the present invention collects weather data, soil analysis data, and crop growth status data, analyzes this data using multimodal AI, proposes efficient and environmentally friendly agricultural methods, and executes these methods using automated robots. The system consists of the following main components:

[0721] 1. Data Collection Module

[0722] User: The user first inputs the type of crop they are growing and the location of their farm into the system, which sets the baseline for collecting and analyzing climate data.

[0723] Terminal (robot): The terminal uses soil analysis sensors and crop growth monitoring cameras to periodically measure soil moisture, pH, and crop growth status. This acquired data is sent to the server in real time.

[0724] 2. Data transmission and analysis module

[0725] Server: The server obtains the necessary climate data from an external weather data API. It also receives soil data and crop growth data sent from the device, integrates them, and analyzes them using multimodal AI. This analysis derives optimal agricultural methods that take into account soil moisture, pH value, predicted precipitation, crop growth stage, and other factors.

[0726] 3. Advice Generation Module

[0727] Server: Based on the analysis results, it generates advice to propose specific agricultural methods, for example, recommending irrigation if soil moisture is low, or drainage measures if rainfall is forecast.

[0728] 4. Robot Control Module

[0729] Terminal (robot): Receives advice sent from the server and performs actual agricultural work based on the instructions. For example, if an irrigation command is issued, it will automatically water the area, and if drainage measures are required, it will activate the drainage system.

[0730] Specific examples

[0731] Let's take the example of an urban farmer growing tomatoes on a rooftop farm in Tokyo.

[0732] 1. User: A user logs into the system and inputs that the crop being grown is "tomatoes" and that the farm's location is "Tokyo."

[0733] 2. Terminal (Robot): The robot measures the soil moisture and pH value with sensors and records the tomato growth stage with a camera. The collected data is sent to the server in real time.

[0734] 3. Server: The server retrieves the latest weather information for Tokyo from an external weather data API, integrates it with soil humidity, pH value, and tomato growth status, and analyzes it. As a result of the analysis, it is determined that the soil humidity is low and that irrigation is necessary.

[0735] 4. Server: Generates irrigation advice and sends it to the robot.

[0736] 5. Terminal (Robot): Receives advice from the server, automatically starts irrigation, and provides the tomatoes with the appropriate amount of water.

[0737] This system allows farmers to receive advice on optimal farming methods in real time, enabling them to practice sustainable farming without hassle, thereby supporting the realization of efficient and environmentally friendly agriculture.

[0738] The processing flow will be explained below.

[0739] Step 1:

[0740] User: The user logs into the system and inputs the type of crop they are growing (e.g., tomatoes) and the location of their farm (e.g., Tokyo). This information serves as the basis for subsequent data collection and analysis.

[0741] Step 2:

[0742] Terminal (Robot): The robot uses soil analysis sensors and growth monitoring cameras to measure the soil moisture, pH value, and crop growth status in the farm. The collected data is sent to the server in real time.

[0743] Step 3:

[0744] Server: The server obtains the latest climate data for Tokyo (temperature, humidity, precipitation, wind speed, etc.) from an external weather data API. This climate data is integrated as information required for analysis.

[0745] Step 4:

[0746] Server: The server integrates the soil data, crop growth data, and weather data sent from the terminal (robot), inputs it into a multimodal AI system, and performs analysis. This analysis derives optimal agricultural methods that take into account soil moisture, pH value, and the stage of crop growth.

[0747] Step 5:

[0748] Server: Generates specific agricultural advice based on the results of multimodal AI analysis, such as recommending irrigation if soil moisture is low or drainage measures if rainfall is forecast.

[0749] Step 6:

[0750] Server: Sends the generated agricultural advice to the terminal (robot). This advice includes specific operational instructions (e.g., start irrigation, activate drainage system, etc.).

[0751] Step 7:

[0752] Terminal (Robot): The robot automatically performs agricultural tasks based on advice received from the server. For example, if an instruction is given to irrigate, it starts supplying water and supplies the appropriate amount. If an instruction is given to take drainage measures, it activates the drainage system to remove excess water.

[0753] This series of steps enables users to practice efficient and sustainable agriculture, improving productivity while reducing environmental impact.

[0754] Example 1

[0755] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0756] In traditional agriculture, it is difficult to determine the optimal timing of work based on climate change and soil conditions, making it difficult to efficiently manage crop growth. Furthermore, manual management is labor-intensive and time-consuming, requiring highly accurate data analysis to achieve sustainable agriculture. While there is a growing need for automation using robots, current systems lack seamless integration from data collection to work execution.

[0757] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0758] In this invention, the server includes: a means for a user to input information about the type of crop and the location of the farm; a means for measuring the soil moisture and pH value and the growth status of the crop using a soil analysis sensor and a crop growth monitoring camera; a means for transmitting the above data to the server in real time; a means for acquiring climate data from an external weather data API; a means for analyzing the collected data using multimodal AI; a means for proposing optimal agricultural techniques based on the analysis results; and a means for controlling a robot that automatically performs agricultural work on the farm based on the advice transmitted from the server. This enables the automation and optimization of agricultural work.

[0759] A "user" is an entity that inputs information into the system and provides data, such as a farm manager or agricultural worker.

[0760] "Crop type" refers to the particular type or variety of agricultural crop being grown.

[0761] "Farm location information" refers to information about the geographic location of the farm, and specifically includes data such as coordinates and address.

[0762] A "soil analysis sensor" is a device for measuring chemical and physical properties of soil, such as moisture and pH value.

[0763] A "crop growth monitoring camera" is a camera device for visually recording the growth status of cultivated crops.

[0764] "Real-time" refers to data being processed and transmitted immediately, with almost no time lag.

[0765] A "server" is a central computer system that processes, analyzes, stores, and communicates with other system components.

[0766] "External Weather Data API" means an external application programming interface that provides weather data over the Internet.

[0767] "Climate Data" means information about current and forecast weather conditions in a particular geographic area, such as precipitation, temperature, and humidity.

[0768] "Multimodal AI" is an artificial intelligence technology that integrates and analyzes multiple different types of data (e.g., climate data, soil data, video data).

[0769] "Analysis Results" refers to information or conclusions obtained as a result of analysis conducted based on collected data.

[0770] "Agricultural methods" refer to the specific work and technical means used in growing crops.

[0771] A "robot" is a mechanical device that automatically performs agricultural tasks within a farm.

[0772] "Control" refers to a system issuing instructions to a robot or other device and managing and operating its operations.

[0773] "Advice" refers to optimal farming techniques and specific work instructions provided based on the analysis results.

[0774] The agricultural support system of the present invention collects weather data, soil analysis data, and crop growth status data and analyzes them using multimodal AI to automate and optimize agricultural work. Based on the analysis results, it proposes optimal agricultural methods and executes those methods using automated robots. The main components of this system are described below.

[0775] Data Entry and Collection

[0776] First, users input the type of crop they are growing and the location of their farm. This information is used to set the criteria for collecting and analyzing climate data. For example, entering "tomatoes" and "Tokyo" will generate an analysis based on Tokyo's climate conditions.

[0777] Next, the terminal (robot) uses a soil analysis sensor and a crop growth monitoring camera to periodically measure the soil humidity, pH value, and crop growth status. Specifically, the soil sensor measures humidity, and the camera records the growth status. This data is sent to the server in real time.

[0778] Data Transmission and Integration

[0779] The server receives soil data and crop growth data sent from the device and obtains necessary climate data from an external weather data API. For example, it integrates information such as predicted rainfall, temperature, and humidity and analyzes it together with the soil and crop growth data.

[0780] Data analysis

[0781] The server inputs the collected weather, soil, and crop growth data into the multimodal AI for analysis, which then derives optimal agricultural methods that take into account soil moisture, pH, predicted precipitation, and the stage of crop growth. Specifically, the AI ​​determines that soil moisture is low and that irrigation is necessary.

[0782] Advice Generation

[0783] Based on the analysis results, the server generates agricultural advice, including the timing and amount of irrigation and how to drain the fields after rainfall. For example, specific instructions such as "apply 100 liters of water immediately" are generated and sent to the device.

[0784] Robot Control and Execution

[0785] The terminal (robot) receives the advice sent from the server and performs the actual agricultural work based on the instructions. Specifically, the robot activates the irrigation system and sprays the set amount of water. In this way, agricultural work is carried out automatically and efficiently.

[0786] Specific example explanation

[0787] Let's take the example of an urban farmer growing tomatoes on a rooftop farm in Tokyo. A user logs into the system and enters that the crop being grown is "tomatoes" and that the farm is located in "Tokyo." The robot uses sensors to measure the moisture and pH value of the soil and a camera to record the tomato growth stage. The collected data is sent to a server in real time.

[0788] The server obtains the latest weather information for Tokyo from an external weather data API, and performs analysis by integrating it with soil humidity, pH value, and tomato growth status. The analysis results show that the soil humidity is low and determines that irrigation is necessary. The server generates advice recommending irrigation and sends it to the robot. The robot receives the advice from the server, automatically starts irrigation, and supplies the tomatoes with the appropriate amount of water.

[0789] Prompt Sentence Examples

[0790] "What are the main features of this system?"

[0791] How should users enter data?

[0792] "Please explain in detail how the data collection module works."

[0793] "Please tell me how this system suggests agricultural methods."

[0794] Please explain the flow of this system using a specific example.

[0795] This system allows farmers to receive advice on optimal farming methods in real time, enabling them to practice sustainable farming without hassle, thereby supporting the realization of efficient and environmentally friendly agriculture.

[0796] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0797] Step 1:

[0798] A user logs into the system and enters the type of crop they are growing (e.g., tomatoes) and the location of their farm (e.g., Tokyo).

[0799] Input: type of crop, farm location

[0800] Output: Reference climate data acquisition conditions

[0801] Specific operation: The user enters "tomato" and "Tokyo" through the system interface, which is then sent to the server.

[0802] Step 2:

[0803] The terminal (robot) uses a soil analysis sensor to measure the moisture and pH value of the soil, and a crop growth monitoring camera to capture images of the crop growth. This data is sent to a server in real time.

[0804] Input: Measurements from sensors and cameras

[0805] Output: Measurement data (humidity, pH value, growth status images)

[0806] How it works: The robot's soil sensors measure humidity at 30% and pH at 6.5, and its camera captures images of the tomato's growth stages, all of which are sent to a server.

[0807] Step 3:

[0808] The server accesses an external weather data API to obtain the latest weather data (e.g., predicted rainfall, temperature, and humidity) based on the farm's location, and also receives soil and crop growth data sent from the device.

[0809] Input: Farm location, soil data, growth data

[0810] Output: Integrated climate and farm data

[0811] Specific operation: The server sends an API request to obtain weather data for "Tokyo" (e.g., rainfall 50mm, temperature 25°C, humidity 60%) and integrates it with data from the device.

[0812] Step 4:

[0813] The data collected by the server is input into a multimodal AI system for analysis, which then determines optimal agricultural methods taking into account factors such as soil moisture, pH, predicted precipitation, and the stage of crop development.

[0814] Input: Integrated climate data, soil data, growth data

[0815] Output: Analysis results (e.g., irrigation required)

[0816] Specific operation: Multimodal AI analyzes the data and determines that irrigation is necessary because the humidity is low at 30%.

[0817] Step 5:

[0818] Based on the analysis, the server generates specific agricultural advice, including the timing and amount of irrigation and drainage measures after rainfall.

[0819] Input: Analysis results

[0820] Output: Farming advice (e.g., water 100 liters now)

[0821] Specific operation: The server generates the instruction "Please sprinkle 100 liters of water now" in text format and sends it to the terminal.

[0822] Step 6:

[0823] The terminal (robot) receives the advice sent from the server and performs the actual agricultural work based on the instructions, for example, activating the irrigation system and spraying a set amount of water.

[0824] Input: Advice from the server

[0825] Output: Agricultural work performed (e.g. irrigation)

[0826] Specific action: The robot starts the irrigation system and executes the command "spray 100 liters of water."

[0827] Through these steps, a system will be built that allows agricultural work to be carried out efficiently and automatically, supporting sustainable agriculture.

[0828] (Application example 1)

[0829] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0830] Current agricultural systems lack a system that consistently collects and analyzes environmental and crop data, and then proposes and implements optimal agricultural methods based on the results. Furthermore, users have limited means of accessing real-time information on their own devices, making it difficult to manage their farms quickly and effectively. This hinders agricultural efficiency and sustainability.

[0831] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0832] In this invention, the server includes means for collecting climate data, means for collecting soil analysis data, means for collecting crop growth status data, means for analyzing the above data using multimodal artificial intelligence, means for proposing optimal agricultural methods based on the analysis results, means for notifying users of the analysis results and agricultural method advice in real time, and means for controlling machinery that automates agricultural work on the farm based on the analysis results. This allows users to receive analysis results and advice in real time, enabling fast and effective agricultural management. Furthermore, the automation of agricultural work is expected to advance, leading to the realization of efficient and sustainable agriculture.

[0833] "Climate data" refers to data on weather conditions and environmental factors that affect crop growth, including temperature, humidity, precipitation, wind speed, etc.

[0834] "Soil analysis data" refers to data on the physical and chemical properties of agricultural soil, including pH, humidity, and nutrient concentration.

[0835] "Crop growth status data" refers to data that indicates the growth stage of a crop, including height, number of leaves, color, signs of disease, and the like.

[0836] "Multimodal AI" is an AI technology that integrates and analyzes multiple data modalities (e.g., climate data, soil data, image data, etc.).

[0837] "Analysis results" are the results of analysis generated by multimodal artificial intelligence and are information used to derive specific agricultural methods.

[0838] "Agricultural practices" are specific procedures and techniques used to optimize crop growth and increase yields.

[0839] "Users" are farmers and managers who use the system to manage and optimize agricultural operations.

[0840] "Real-time" refers to a situation in which data is collected and analyzed immediately, and the results are provided to users instantly.

[0841] "Advice" refers to recommendations for agricultural techniques and work provided to users based on analysis results from multimodal artificial intelligence.

[0842] A "personal digital assistant" is a portable electronic device, such as a smartphone or tablet, that allows a user to view information in real time.

[0843] "Machines that automate agricultural work" are devices that perform agricultural work automatically, such as tractors, irrigation systems, and harvesting robots.

[0844] A "central processing unit" is a computer system or server that analyzes collected data and generates optimal agricultural practices.

[0845] The agricultural support system of the present invention collects and analyzes weather data, soil analysis data, and data on the growth status of crops, proposes optimal agricultural methods, and executes those methods using automated machinery. Specific means for realizing this system will be described below.

[0846] 1. Data Collection Module

[0847] User: The user first inputs the type of crop they are growing and the location of their farm, which sets the basis for collecting and analyzing climate data.

[0848] Terminal: The terminal uses soil analysis sensors and crop growth monitoring cameras to periodically measure soil moisture, pH, and crop growth status. This acquired data is sent to the server in real time.

[0849] 2. Data transmission and analysis module

[0850] Server: The server obtains the necessary weather data from an external weather data API. It also receives soil data and crop growth data sent from the device, integrates them, and analyzes them using multimodal AI. This analysis derives optimal agricultural methods that take into account soil moisture, pH value, predicted precipitation, crop growth stage, and other factors.

[0851] 3. Advice Generation Module

[0852] Server: Based on the analysis results, the server generates advice to propose specific agricultural methods. For example, it recommends irrigation if soil moisture is low, or drainage measures if rainfall is forecast. The generated advice is sent to the user's mobile information terminal in real time.

[0853] 4. Robot Control Module

[0854] Terminal: Receives advice sent from the server and carries out the actual agricultural work based on that instruction. For example, if an instruction is given to irrigate, it will automatically water the area, and if drainage measures are required, it will activate the drainage system. This is done using automated machinery on the farm.

[0855] Specific examples

[0856] Let's take the example of an urban farmer growing tomatoes on a rooftop farm in a city.

[0857] 1. User: A user logs into the system and inputs that the crop being grown is "tomatoes" and that the farm is located "inside the city."

[0858] 2. Terminal: The terminal uses sensors to measure the moisture and pH of the soil, and a camera to record the tomato's growth stage. The collected data is sent to the server in real time.

[0859] 3. Server: The server retrieves the latest weather information for the city from an external weather data API, integrates it with soil humidity, acidity, and tomato growth status, and analyzes it. As a result of the analysis, it is determined that the soil humidity is low and that irrigation is necessary.

[0860] 4. Server: Generates irrigation advice and sends it to the user's mobile device in real time.

[0861] 5. Terminal: Receives advice from the server and automatically starts irrigation, providing the right amount of water to the tomatoes.

[0862] Prompt Sentence Examples

[0863] "Build an AI model that recommends irrigation when soil moisture is below 30% and no rain is forecast."

[0864] "Build an application that monitors the growth of tomatoes on an urban farm and suggests optimal farming practices."

[0865] This will enable users to receive analysis results and advice in real time, enabling quick and effective agricultural management. It is also expected that the automation of agricultural work will progress, leading to efficient and sustainable agriculture.

[0866] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0867] Step 1:

[0868] Users log in to the system and input the type of crop they are growing and the location of their farm, which then sets the criteria for collecting and analyzing climate data. The input is the type of crop and location, and the output is the set criteria.

[0869] Step 2:

[0870] The device uses soil analysis sensors and crop growth monitoring cameras to periodically measure soil moisture, pH, and crop growth. The measurement data is sent to the server in real time. The input is the measurement value of each sensor, and the output is data for analysis sent to the server.

[0871] Step 3:

[0872] The server acquires external weather data through the weather data API and integrates it with soil data and crop growth data sent from the terminal. The inputs are weather data, soil data, and crop growth data, and the output is the integrated dataset.

[0873] Step 4:

[0874] The server analyzes the integrated data using multimodal artificial intelligence to determine optimal agricultural practices, taking into account soil moisture, pH, predicted precipitation, crop growth stage, etc. The input is the integrated dataset, and the output is the analysis result (e.g., whether irrigation is required).

[0875] Step 5:

[0876] Based on the analysis results, the server generates advice to propose specific agricultural methods. For example, if the soil moisture is low, it recommends irrigation. The generated advice is sent to the user's mobile information terminal in real time. The input is the analysis results, and the output is the advice sent to the user.

[0877] Step 6:

[0878] The user's mobile information device receives the notification from the server and displays the analysis results and advice to the user. The user can then take the necessary action based on the displayed advice. The input is the notification from the server, and the output is the analysis results and advice presented to the user.

[0879] Step 7:

[0880] The terminal receives advice sent from the server and controls the automated machinery on the farm based on the instructions to carry out agricultural work. For example, if an irrigation command is issued, it will automatically water the farm, and if drainage measures are required, it will activate the drainage system. The input is the advice from the server, and the output is the agricultural work that has been carried out.

[0881] In this way, users can receive real-time advice on optimal farming techniques, leading to efficient and sustainable farming.

[0882] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0883] The agricultural support system of the present invention collects weather data, soil analysis data, and crop growth status data, analyzes this data using multimodal AI, proposes efficient and environmentally friendly agricultural methods, and executes these methods using automated robots. The present invention also incorporates an emotion engine that recognizes the user's emotions, and by utilizing the user's emotion data in the analysis, it is possible to provide more personalized advice. The configuration and operation of the system of the present invention are described in detail below.

[0884] System configuration

[0885] 1. Data Collection Module

[0886] Users log in to the system and enter the type of crop they are growing and the location of their farm, which determines the scope of data to be collected.

[0887] Terminal (robot): Using a soil analysis sensor and a crop growth monitoring camera, it collects data measuring soil humidity, pH value, and crop growth status. It also collects user emotion data through voice recognition and facial expression analysis.

[0888] 2. Data transmission and analysis module

[0889] Server: Receives soil data, crop growth data, and user emotion data sent from the device, and integrates it with weather data obtained from an external weather data API. Using multimodal AI, this data is analyzed to comprehensively assess soil condition, crop status, and weather conditions.

[0890] 3. Advice Generation Module

[0891] Server: Based on the results of multimodal AI analysis, the server generates advice proposing optimal farming methods. It also takes into account the user's emotional data and includes stress reduction measures and warnings as needed.

[0892] 4. Robot Control Module

[0893] Terminal (robot): Receives advice sent from the server and performs agricultural tasks based on the instructions. For example, if an instruction is given to irrigate, it will automatically water the land, and if an instruction is given to take drainage measures, it will activate the drainage system.

[0894] Specific examples

[0895] A specific example of using the system of the present invention will be described below.

[0896] Consider an urban farmer growing tomatoes on a rooftop farm in Tokyo.

[0897] 1. User: The user logs in to the system and inputs that the crop they are growing is "tomatoes" and that the farm is located in "Tokyo." The system also recognizes that the collected emotional data includes the user's voice and facial expressions while working.

[0898] 2. Terminal (Robot): The robot measures the soil moisture and pH value with sensors, records the tomato growth stage with a camera, and simultaneously collects emotional data from the user's voice and facial expressions.

[0899] 3. Server: The server retrieves the latest weather information for Tokyo from the weather data API, and then analyzes it together with soil data, crop growth data, and the user's emotional data. The analysis results indicate that the soil moisture is particularly low and that irrigation is necessary. The emotion engine also detects that the user is feeling stressed.

[0900] 4. Server: Based on the analysis results, generate irrigation advice along with relaxation activity recommendations, for example, "Perform irrigation and take a short break in between."

[0901] 5. Terminal (Robot): The robot starts irrigation and provides the appropriate amount of water. At the same time, a notification is displayed encouraging the user to take a break.

[0902] In this way, this system supports efficient and sustainable agriculture by integrating and analyzing environmental data and user emotional data to propose personalized agricultural methods.

[0903] The processing flow will be explained below.

[0904] Step 1:

[0905] User: Logs into the system and inputs the type of crop being grown (e.g., tomatoes) and the location of the farm (e.g., Tokyo). This sets up the basic information that allows the system to collect and analyze the appropriate data.

[0906] Step 2:

[0907] Terminal (robot): Using a soil analysis sensor, it measures the soil moisture and pH value in the farm. At the same time, it uses a crop growth monitoring camera to record the tomato growth status (e.g., height, number of leaves, color). In addition, it uses voice recognition and facial expression analysis sensors to collect emotional data from the user's voice and facial expressions. The collected data is sent to the server in real time.

[0908] Step 3:

[0909] Server: Obtains the latest weather data for Tokyo (temperature, humidity, precipitation, wind speed, etc.) from an external weather data API. This data is also analyzed along with other collected data.

[0910] Step 4:

[0911] Server: Integrates soil data, crop growth data, and user emotional data sent from the device. This data is input into the multimodal AI for analysis. Specifically, it determines the need for irrigation if the soil moisture is low, the stage of crop growth, and the user's stress level.

[0912] Step 5:

[0913] Server: Based on the analysis results, the server generates specific advice on agricultural techniques. For example, if the soil moisture is low, the server generates the advice "Please irrigate." If the user's emotional data indicates a high stress level, the server adds the recommendation "Please take a break to relax while irrigating."

[0914] Step 6:

[0915] Server: Sends advice to the robot, including specific work instructions (e.g., start irrigation, recommend breaks).

[0916] Step 7:

[0917] Terminal (Robot): Automatically performs agricultural tasks based on advice received from the server. For example, if an irrigation instruction is given, it will automatically start watering. It also displays a notification to the user to take a break to relax.

[0918] This series of steps allows users to easily practice efficient and sustainable farming while also optimally managing their own emotional state. The system aims to achieve both optimal farming practices and user emotional management by collecting and analyzing data in real time.

[0919] Example 2

[0920] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0921] While conventional agricultural support systems collect and analyze weather data, soil analysis data, and crop growth status data, they lack the ability to provide personalized advice or propose agricultural methods that take into account the user's emotional data. Furthermore, there are limited means for implementing automated agricultural work using the collected data, making it difficult to achieve efficient and sustainable agriculture.

[0922] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0923] In this invention, the server includes means for collecting and analyzing weather data, soil analysis data, crop growth status data, and user emotion data, means for generating optimal agricultural techniques and advice for the user on stress reduction measures and cautions based on the analysis results, and means for transmitting the generated advice to the agricultural machine and having the machine perform work based on the instructions. This enables comprehensive data analysis that incorporates user emotion data, making it possible to propose and implement efficient and personalized agricultural techniques.

[0924] "Climate data" refers to information about weather, temperature, humidity, precipitation, wind speed, and other meteorological conditions.

[0925] "Soil analysis data" refers to information about the physical and chemical properties of soil, such as soil moisture, pH, and nutrient content.

[0926] "Crop growth status data" refers to information about the state of growth of crops, such as the stage of growth of the crops, leaf color, and the presence or absence of pests or diseases.

[0927] "User emotion data" refers to information about the user's emotional state analyzed from their voice and facial expressions.

[0928] "Multimodal AI" refers to artificial intelligence that performs integrated analysis of multiple different types of data (e.g., text data, image data, audio data, etc.).

[0929] "Analysis results" refers to information obtained as a result of analysis conducted by AI based on collected data.

[0930] "Agricultural techniques" refer to methods and techniques for carrying out agricultural work efficiently.

[0931] "Agricultural machinery" means an automated machine used to perform agricultural work.

[0932] "Stress reduction measures" are suggestions and measures to reduce the user's stress.

[0933] A "warning" is a notification or warning that prompts the user to take some kind of action.

[0934] The agricultural support system according to the present invention is composed of multiple modules that work in conjunction with each other to provide efficient and personalized agricultural support. This system is made up of a data collection module, a data transmission and analysis module, an advice generation module, and a robot control module. Details and specific operations of each module are explained below.

[0935] Data Collection Module

[0936] User:

[0937] Users first log into the system and input the type of crop they are growing and the location of their farm, which sets the scope of data to be collected.

[0938] Terminal (Robot):

[0939] The device is equipped with a soil analysis sensor and a crop growth monitoring camera. The soil analysis sensor measures the soil's moisture and pH value, and the crop growth monitoring camera records the crop's growth status. It also collects user emotion data through voice recognition and facial expression analysis. All of this data is collected in real time.

[0940] Data Transmission and Analysis Module

[0941] Terminal (Robot):

[0942] The collected soil data, crop growth data, and user emotion data are sent to a server via wireless communication (e.g., Wi-Fi).

[0943] server:

[0944] The server acquires weather data from an external weather data API and integrates it with data sent from the device. Specifically, weather data, soil data, crop growth data, and user emotion data are all stored in a database and analyzed using multimodal AI.

[0945] Advice Generation Module

[0946] server:

[0947] Multimodal AI is used to comprehensively analyze data. The analysis integrates weather data, soil data, growth data, and emotional data to generate results. Based on the analysis results, advice is generated suggesting optimal farming methods. If the user is feeling stressed, advice on stress reduction measures and caution is also included.

[0948] For example, enter the following prompt into the server:

[0949] "Analyze the transmitted sensor and camera data and suggest optimal agricultural methods. If the soil moisture is below 60%, advise irrigation. Also, if the user is feeling stressed, provide advice on how to deal with it."

[0950] Robot Control Module

[0951] Terminal (Robot):

[0952] The system receives advice generated by the server and performs agricultural tasks based on the instructions. For example, it automatically waters the field when irrigation is requested, or activates the drainage system when drainage measures are requested. It also displays notifications encouraging the user to take a break.

[0953] A concrete example of the overall flow

[0954] Suppose an urban farmer is growing tomatoes on a rooftop farm in Tokyo. The user logs into the system and inputs that the crop is tomatoes and that the farm is in Tokyo. The terminal (robot) measures the humidity and pH value of the soil and records the tomato's growth stage with a camera. It also collects emotional data from voice and facial expressions. The server combines this data with weather data and produces an analysis result. Based on the analysis result, it determines that irrigation is necessary and suggests appropriate farming methods to the user. It also instructs the user to take a break if they are feeling stressed.

[0955] In this way, the system of the present invention comprehensively analyzes environmental data and user emotional data, and provides personalized agricultural methods, thereby supporting efficient and sustainable agriculture.

[0956] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0957] Step 1: Data entry

[0958] Users log in to the system and input the type of crop they are growing and the location of their farm. Based on this input data, the system sets the range of data to be collected and the required data items. Specifically, users access a dedicated application from their smartphone or PC and input the type of crop (e.g., tomatoes) and the location of their farm (e.g., Tokyo). The input data is sent to the server and saved in a configuration database.

[0959] Input: type of crop, farm location

[0960] Output: Configuration database updated

[0961] Step 2: Collect environmental data

[0962] The terminal (robot) uses soil analysis sensors and crop growth monitoring cameras to measure soil moisture, pH, and crop growth status. This data is collected periodically. The robot patrols the farm, measuring soil moisture and pH using the soil sensors and saving the results in its internal memory. At the same time, it records the crop growth stages using the monitoring cameras.

[0963] Input: None (data collected periodically)

[0964] Output: Measurement data (humidity, pH value, growth status)

[0965] Step 3: Collecting emotion data

[0966] The terminal (robot) analyzes the user's voice and facial expressions to collect emotional data. Specifically, when the robot is near the user, it activates a voice recognition system and facial expression analysis system. The words spoken and facial expressions of the user while working are analyzed in real time to generate emotional data.

[0967] Input: User's voice, facial expression video

[0968] Output: Emotion data

[0969] Step 4: Send data

[0970] The terminal (robot) transmits the collected soil data, crop growth data, and emotion data to the server via wireless communication (Wi-Fi). When the robot finishes collecting data, it automatically transfers the data to the server via the wireless communication module.

[0971] Input: Measurement data, emotion data

[0972] Output: Send data to the server

[0973] Step 5: Obtaining Weather Data

[0974] The server obtains the latest weather data from an external weather data API. Specifically, the server periodically sends requests to the API to obtain real-time weather data (temperature, humidity, precipitation, etc.). The obtained data is stored in an internal database.

[0975] Input: None (data acquired periodically)

[0976] Output: Weather data

[0977] Step 6: Data integration and analysis

[0978] The server integrates soil data, crop growth data, emotion data, and weather data sent from the device and analyzes them using multimodal AI.The server also extracts various data stored in the database and performs multidimensional analysis using deep learning models.

[0979] Input: soil data, crop growth data, emotion data, weather data

[0980] Output: Analysis results

[0981] Step 7: Advice Generation

[0982] The server generates optimal agricultural methods and advice for users based on the results of multimodal AI analysis. Based on the data derived from the analysis, it creates specific advice on irrigation and fertilizer application timing, pest control, and work efficiency. It also includes stress reduction measures and warnings based on the user's emotional data.

[0983] Specific prompt examples:

[0984] "Analyze the transmitted sensor and camera data and suggest optimal agricultural methods. If the soil moisture is below 60%, advise irrigation. Also, if the user is feeling stressed, provide advice on how to deal with it."

[0985] Input: Analysis results

[0986] Output: Advice content

[0987] Step 8: Submitting Advice

[0988] The server sends the generated advice to the terminal (robot), formats the advice into an appropriate format, and transmits it to the robot via wireless communication.

[0989] Input: Advice content

[0990] Output: Send advice to the robot

[0991] Step 9: Robot execution

[0992] The terminal (robot) performs agricultural work based on instructions from the server. The robot analyzes the advice it receives and begins the necessary agricultural work. For example, if the analysis results indicate that irrigation is necessary, the robot will activate the irrigation system and spray the appropriate amount of water. It will also display a notification encouraging the user to take a break.

[0993] Input: Advice content

[0994] Output: Execution of agricultural work, notification to user

[0995] (Application example 2)

[0996] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0997] In conventional production sites, environmental data and worker emotional data were not fully utilized, resulting in insufficient production efficiency and worker stress management. This made it difficult to find the optimal production method, which could have a negative impact on productivity and the work environment. The present invention aims to solve these problems by providing a system that comprehensively analyzes environmental data and worker emotional data and proposes the optimal production method.

[0998] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting weather data, means for collecting environmental data, means for collecting worker emotion data, means for analyzing the above data using multimodal AI, means for proposing an optimal production method based on the analysis results, and means for controlling equipment that automates the production process. This makes it possible to comprehensively analyze the collected data, improve production efficiency, and manage worker stress.

[0999] "Climate data" is information about weather conditions such as temperature, humidity, air pressure, precipitation, and wind speed.

[1000] "Environmental data" refers to information on various physical environments such as temperature, humidity, vibration, and noise within a factory or production site.

[1001] "Worker's emotional data" is information about the emotional state obtained by analyzing the worker's physiological responses such as voice and facial expressions.

[1002] "Multimodal AI" is an artificial intelligence technology that integrates and analyzes different types of data (e.g., voice, images, text, etc.).

[1003] "Production methods" refer to the procedures and methods of specific production processes and work processes.

[1004] "Equipment that automates production processes" refers to devices and systems that automate production activities within factories or production sites.

[1005] A "server" is a central computer in a system that analyzes and processes data, and stores and provides information.

[1006] The present invention is a system that collects weather data, environmental data, and worker emotion data at a production site, analyzes this data using multimodal AI, proposes optimal production methods, and automates the production process. Specific embodiments of the present invention are described in detail below.

[1007] System configuration

[1008] 1. Data Collection Module

[1009] The server is equipped with sensors and devices to collect weather and environmental data, as well as worker emotional data, including temperature, humidity, and vibration sensors, and an emotion recognition system that analyzes the voices and facial expressions of workers.

[1010] 2. Data transmission and analysis module

[1011] The collected data is sent to a server, which then integrates it and analyzes it using multimodal AI. This analysis uses various sensor APIs, emotion recognition APIs, and an HTTP request library.

[1012] 3. Advice Generation Module

[1013] Based on the analysis results, the server generates advice proposing optimal production methods. Specifically, if the temperature inside the factory is high, instructions are generated to activate the cooling system or encourage workers to take breaks.

[1014] 4. Robot Control Module

[1015] Upon receiving instructions from the server, the terminal (robot) performs automated tasks in the production process, such as operating the cooling system, adjusting humidity, and automatically transporting goods.

[1016] Hardware and software used

[1017] Hardware:

[1018] Temperature Sensor

[1019] Humidity Sensor

[1020] Vibration Sensor

[1021] Emotion Recognition System

[1022] Robots that carry out production processes

[1023] software:

[1024] Sensor API for collecting information

[1025] Emotion recognition API

[1026] HTTP request library

[1027] Multimodal AI analysis API

[1028] Specific examples

[1029] If environmental data on a factory production line indicates high temperature and humidity levels and that workers are feeling stressed, the system will activate the cooling system and send a notification to the worker's smartphone telling them to take a break. In this way, it is possible to maintain the health of workers without reducing production efficiency.

[1030] Prompt Sentence Examples

[1031] An optimization system using robots on factory production lines. Data is collected from temperature, humidity, and vibration sensors, as well as worker emotional data. Multimodal AI is used to analyze the data and generate optimal production methods and robot operation instructions. Instructions include activating the cooling system when temperatures are high, and notifications to encourage workers to take breaks.

[1032] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1033] Step 1:

[1034] Users log in to the system using a smartphone or head-mounted display and input information about the production line and the equipment to be managed. This allows the system to identify the target for monitoring and data collection. The input information includes the type of equipment, its installation location, and worker information. Based on this input data, the system starts operating the necessary sensors and devices.

[1035] Step 2:

[1036] The terminals (sensors) collect environmental data (temperature, humidity, vibration) within the factory and emotional data from workers' voices and facial expressions. Specifically, the temperature sensor detects the temperature on-site, the humidity sensor measures humidity, and the vibration sensor records the vibration state of the equipment. In addition, the emotion recognition system analyzes the workers' faces and voices to generate emotional data. The collected data is formatted and sent to a server in real time.

[1037] Step 3:

[1038] The server receives the environmental data and emotional data sent from the device, integrates them, and stores them in a database. Specifically, the server processes HTTP requests, retrieves data via various data APIs, and stores it. This allows the system to grasp the overall environmental situation and the worker's emotional state.

[1039] Step 4:

[1040] The server analyzes the data stored in the database using a multimodal AI model. Specifically, it receives temperature, humidity, vibration, and emotion data as input and analyzes the correlation between each piece of data. As a result of the analysis, it evaluates how specific environmental conditions and worker status affect production efficiency. The output is generated as optimal production methods and advice.

[1041] Step 5:

[1042] The generated production methods and advice are sent from the server to the terminal (robot). Specific instructions are transmitted to the robot using APIs and communication protocols. Examples include instructions to activate the cooling system if the temperature is high, or notifications to encourage workers to take a break if they are feeling stressed.

[1043] Step 6:

[1044] The terminals (robots) carry out production processes based on instructions received from the server. Specific operations include operating the cooling system, adjusting humidity, and transporting parts. This allows for efficient operation of the production line and optimization of the working environment.

[1045] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1046] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1047] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1048] [Fourth embodiment]

[1049] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1050] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1051] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1052] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1053] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1054] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1055] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1056] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1057] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1058] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1059] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1060] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1061] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1062] The agricultural support system of the present invention collects weather data, soil analysis data, and crop growth status data, analyzes this data using multimodal AI, proposes efficient and environmentally friendly agricultural methods, and executes these methods using automated robots. The system consists of the following main components:

[1063] 1. Data Collection Module

[1064] User: The user first inputs the type of crop they are growing and the location of their farm into the system, which sets the baseline for collecting and analyzing climate data.

[1065] Terminal (robot): The terminal uses soil analysis sensors and crop growth monitoring cameras to periodically measure soil moisture, pH, and crop growth status. This acquired data is sent to the server in real time.

[1066] 2. Data transmission and analysis module

[1067] Server: The server obtains the necessary climate data from an external weather data API. It also receives soil data and crop growth data sent from the device, integrates them, and analyzes them using multimodal AI. This analysis derives optimal agricultural methods that take into account soil moisture, pH value, predicted precipitation, crop growth stage, and other factors.

[1068] 3. Advice Generation Module

[1069] Server: Based on the analysis results, it generates advice to propose specific agricultural methods, for example, recommending irrigation if soil moisture is low, or drainage measures if rainfall is forecast.

[1070] 4. Robot Control Module

[1071] Terminal (robot): Receives advice sent from the server and performs actual agricultural work based on the instructions. For example, if an irrigation command is issued, it will automatically water the area, and if drainage measures are required, it will activate the drainage system.

[1072] Specific examples

[1073] Let's take the example of an urban farmer growing tomatoes on a rooftop farm in Tokyo.

[1074] 1. User: A user logs into the system and inputs that the crop being grown is "tomatoes" and that the farm's location is "Tokyo."

[1075] 2. Terminal (Robot): The robot measures the soil moisture and pH value with sensors and records the tomato growth stage with a camera. The collected data is sent to the server in real time.

[1076] 3. Server: The server retrieves the latest weather information for Tokyo from an external weather data API, integrates it with soil humidity, pH value, and tomato growth status, and analyzes it. As a result of the analysis, it is determined that the soil humidity is low and that irrigation is necessary.

[1077] 4. Server: Generates irrigation advice and sends it to the robot.

[1078] 5. Terminal (Robot): Receives advice from the server, automatically starts irrigation, and provides the tomatoes with the appropriate amount of water.

[1079] This system allows farmers to receive advice on optimal farming methods in real time, enabling them to practice sustainable farming without hassle, thereby supporting the realization of efficient and environmentally friendly agriculture.

[1080] The processing flow will be explained below.

[1081] Step 1:

[1082] User: The user logs into the system and inputs the type of crop they are growing (e.g., tomatoes) and the location of their farm (e.g., Tokyo). This information serves as the basis for subsequent data collection and analysis.

[1083] Step 2:

[1084] Terminal (Robot): The robot uses soil analysis sensors and growth monitoring cameras to measure the soil moisture, pH value, and crop growth status in the farm. The collected data is sent to the server in real time.

[1085] Step 3:

[1086] Server: The server obtains the latest climate data for Tokyo (temperature, humidity, precipitation, wind speed, etc.) from an external weather data API. This climate data is integrated as information required for analysis.

[1087] Step 4:

[1088] Server: The server integrates the soil data, crop growth data, and weather data sent from the terminal (robot), inputs it into a multimodal AI system, and performs analysis. This analysis derives optimal agricultural methods that take into account soil moisture, pH value, and the stage of crop growth.

[1089] Step 5:

[1090] Server: Generates specific agricultural advice based on the results of multimodal AI analysis, such as recommending irrigation if soil moisture is low or drainage measures if rainfall is forecast.

[1091] Step 6:

[1092] Server: Sends the generated agricultural advice to the terminal (robot). This advice includes specific operational instructions (e.g., start irrigation, activate drainage system, etc.).

[1093] Step 7:

[1094] Terminal (Robot): The robot automatically performs agricultural tasks based on advice received from the server. For example, if an instruction is given to irrigate, it starts supplying water and supplies the appropriate amount. If an instruction is given to take drainage measures, it activates the drainage system to remove excess water.

[1095] This series of steps enables users to practice efficient and sustainable agriculture, improving productivity while reducing environmental impact.

[1096] Example 1

[1097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1098] In traditional agriculture, it is difficult to determine the optimal timing of work based on climate change and soil conditions, making it difficult to efficiently manage crop growth. Furthermore, manual management is labor-intensive and time-consuming, requiring highly accurate data analysis to achieve sustainable agriculture. While there is a growing need for automation using robots, current systems lack seamless integration from data collection to work execution.

[1099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1100] In this invention, the server includes: a means for a user to input information about the type of crop and the location of the farm; a means for measuring the soil moisture and pH value and the growth status of the crop using a soil analysis sensor and a crop growth monitoring camera; a means for transmitting the above data to the server in real time; a means for acquiring climate data from an external weather data API; a means for analyzing the collected data using multimodal AI; a means for proposing optimal agricultural techniques based on the analysis results; and a means for controlling a robot that automatically performs agricultural work on the farm based on the advice transmitted from the server. This enables the automation and optimization of agricultural work.

[1101] A "user" is an entity that inputs information into the system and provides data, such as a farm manager or agricultural worker.

[1102] "Crop type" refers to the particular type or variety of agricultural crop being grown.

[1103] "Farm location information" refers to information about the geographic location of the farm, and specifically includes data such as coordinates and address.

[1104] A "soil analysis sensor" is a device for measuring chemical and physical properties of soil, such as moisture and pH value.

[1105] A "crop growth monitoring camera" is a camera device for visually recording the growth status of cultivated crops.

[1106] "Real-time" refers to data being processed and transmitted immediately, with almost no time lag.

[1107] A "server" is a central computer system that processes, analyzes, stores, and communicates with other system components.

[1108] "External Weather Data API" means an external application programming interface that provides weather data over the Internet.

[1109] "Climate Data" means information about current and forecast weather conditions in a particular geographic area, such as precipitation, temperature, and humidity.

[1110] "Multimodal AI" is an artificial intelligence technology that integrates and analyzes multiple different types of data (e.g., climate data, soil data, video data).

[1111] "Analysis Results" refers to information or conclusions obtained as a result of analysis conducted based on collected data.

[1112] "Agricultural methods" refer to the specific work and technical means used in growing crops.

[1113] A "robot" is a mechanical device that automatically performs agricultural tasks within a farm.

[1114] "Control" refers to a system issuing instructions to a robot or other device and managing and operating its operations.

[1115] "Advice" refers to optimal farming techniques and specific work instructions provided based on the analysis results.

[1116] The agricultural support system of the present invention collects weather data, soil analysis data, and crop growth status data and analyzes them using multimodal AI to automate and optimize agricultural work. Based on the analysis results, it proposes optimal agricultural methods and executes those methods using automated robots. The main components of this system are described below.

[1117] Data Entry and Collection

[1118] First, users input the type of crop they are growing and the location of their farm. This information is used to set the criteria for collecting and analyzing climate data. For example, entering "tomatoes" and "Tokyo" will generate an analysis based on Tokyo's climate conditions.

[1119] Next, the terminal (robot) uses a soil analysis sensor and a crop growth monitoring camera to periodically measure the soil humidity, pH value, and crop growth status. Specifically, the soil sensor measures humidity, and the camera records the growth status. This data is sent to the server in real time.

[1120] Data Transmission and Integration

[1121] The server receives soil data and crop growth data sent from the device and obtains necessary climate data from an external weather data API. For example, it integrates information such as predicted rainfall, temperature, and humidity and analyzes it together with the soil and crop growth data.

[1122] Data analysis

[1123] The server inputs the collected weather, soil, and crop growth data into the multimodal AI for analysis, which then derives optimal agricultural methods that take into account soil moisture, pH, predicted precipitation, and the stage of crop growth. Specifically, the AI ​​determines that soil moisture is low and that irrigation is necessary.

[1124] Advice Generation

[1125] Based on the analysis results, the server generates agricultural advice, including the timing and amount of irrigation and how to drain the fields after rainfall. For example, specific instructions such as "apply 100 liters of water immediately" are generated and sent to the device.

[1126] Robot Control and Execution

[1127] The terminal (robot) receives the advice sent from the server and performs the actual agricultural work based on the instructions. Specifically, the robot activates the irrigation system and sprays the set amount of water. In this way, agricultural work is carried out automatically and efficiently.

[1128] Specific example explanation

[1129] Let's take the example of an urban farmer growing tomatoes on a rooftop farm in Tokyo. A user logs into the system and enters that the crop being grown is "tomatoes" and that the farm is located in "Tokyo." The robot uses sensors to measure the moisture and pH value of the soil and a camera to record the tomato growth stage. The collected data is sent to a server in real time.

[1130] The server obtains the latest weather information for Tokyo from an external weather data API, and performs analysis by integrating it with soil humidity, pH value, and tomato growth status. The analysis results show that the soil humidity is low and determines that irrigation is necessary. The server generates advice recommending irrigation and sends it to the robot. The robot receives the advice from the server, automatically starts irrigation, and supplies the tomatoes with the appropriate amount of water.

[1131] Prompt Sentence Examples

[1132] "What are the main features of this system?"

[1133] How should users enter data?

[1134] "Please explain in detail how the data collection module works."

[1135] "Please tell me how this system suggests agricultural methods."

[1136] Please explain the flow of this system using a specific example.

[1137] This system allows farmers to receive advice on optimal farming methods in real time, enabling them to practice sustainable farming without hassle, thereby supporting the realization of efficient and environmentally friendly agriculture.

[1138] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1139] Step 1:

[1140] A user logs into the system and enters the type of crop they are growing (e.g., tomatoes) and the location of their farm (e.g., Tokyo).

[1141] Input: type of crop, farm location

[1142] Output: Reference climate data acquisition conditions

[1143] Specific operation: The user enters "tomato" and "Tokyo" through the system interface, which is then sent to the server.

[1144] Step 2:

[1145] The terminal (robot) uses a soil analysis sensor to measure the moisture and pH value of the soil, and a crop growth monitoring camera to capture images of the crop growth. This data is sent to a server in real time.

[1146] Input: Measurements from sensors and cameras

[1147] Output: Measurement data (humidity, pH value, growth status images)

[1148] How it works: The robot's soil sensors measure humidity at 30% and pH at 6.5, and its camera captures images of the tomato's growth stages, all of which are sent to a server.

[1149] Step 3:

[1150] The server accesses an external weather data API to obtain the latest weather data (e.g., predicted rainfall, temperature, and humidity) based on the farm's location, and also receives soil and crop growth data sent from the device.

[1151] Input: Farm location, soil data, growth data

[1152] Output: Integrated climate and farm data

[1153] Specific operation: The server sends an API request to obtain weather data for "Tokyo" (e.g., rainfall 50mm, temperature 25°C, humidity 60%) and integrates it with data from the device.

[1154] Step 4:

[1155] The data collected by the server is input into a multimodal AI system for analysis, which then determines optimal agricultural methods taking into account factors such as soil moisture, pH, predicted precipitation, and the stage of crop development.

[1156] Input: Integrated climate data, soil data, growth data

[1157] Output: Analysis results (e.g., irrigation required)

[1158] Specific operation: Multimodal AI analyzes the data and determines that irrigation is necessary because the humidity is low at 30%.

[1159] Step 5:

[1160] Based on the analysis, the server generates specific agricultural advice, including the timing and amount of irrigation and drainage measures after rainfall.

[1161] Input: Analysis results

[1162] Output: Farming advice (e.g., water 100 liters now)

[1163] Specific operation: The server generates the instruction "Please sprinkle 100 liters of water now" in text format and sends it to the terminal.

[1164] Step 6:

[1165] The terminal (robot) receives the advice sent from the server and performs the actual agricultural work based on the instructions, for example, activating the irrigation system and spraying a set amount of water.

[1166] Input: Advice from the server

[1167] Output: Agricultural work performed (e.g. irrigation)

[1168] Specific action: The robot starts the irrigation system and executes the command "spray 100 liters of water."

[1169] Through these steps, a system will be built that allows agricultural work to be carried out efficiently and automatically, supporting sustainable agriculture.

[1170] (Application example 1)

[1171] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1172] Current agricultural systems lack a system that consistently collects and analyzes environmental and crop data, and then proposes and implements optimal agricultural methods based on the results. Furthermore, users have limited means of accessing real-time information on their own devices, making it difficult to manage their farms quickly and effectively. This hinders agricultural efficiency and sustainability.

[1173] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1174] In this invention, the server includes means for collecting climate data, means for collecting soil analysis data, means for collecting crop growth status data, means for analyzing the above data using multimodal artificial intelligence, means for proposing optimal agricultural methods based on the analysis results, means for notifying users of the analysis results and agricultural method advice in real time, and means for controlling machinery that automates agricultural work on the farm based on the analysis results. This allows users to receive analysis results and advice in real time, enabling fast and effective agricultural management. Furthermore, the automation of agricultural work is expected to advance, leading to the realization of efficient and sustainable agriculture.

[1175] "Climate data" refers to data on weather conditions and environmental factors that affect crop growth, including temperature, humidity, precipitation, wind speed, etc.

[1176] "Soil analysis data" refers to data on the physical and chemical properties of agricultural soil, including pH, humidity, and nutrient concentration.

[1177] "Crop growth status data" refers to data that indicates the growth stage of a crop, including height, number of leaves, color, signs of disease, and the like.

[1178] "Multimodal AI" is an AI technology that integrates and analyzes multiple data modalities (e.g., climate data, soil data, image data, etc.).

[1179] "Analysis results" are the results of analysis generated by multimodal artificial intelligence and are information used to derive specific agricultural methods.

[1180] "Agricultural practices" are specific procedures and techniques used to optimize crop growth and increase yields.

[1181] "Users" are farmers and managers who use the system to manage and optimize agricultural operations.

[1182] "Real-time" refers to a situation in which data is collected and analyzed immediately, and the results are provided to users instantly.

[1183] "Advice" refers to recommendations for agricultural techniques and work provided to users based on analysis results from multimodal artificial intelligence.

[1184] A "personal digital assistant" is a portable electronic device, such as a smartphone or tablet, that allows a user to view information in real time.

[1185] "Machines that automate agricultural work" are devices that perform agricultural work automatically, such as tractors, irrigation systems, and harvesting robots.

[1186] A "central processing unit" is a computer system or server that analyzes collected data and generates optimal agricultural practices.

[1187] The agricultural support system of the present invention collects and analyzes weather data, soil analysis data, and data on the growth status of crops, proposes optimal agricultural methods, and executes those methods using automated machinery. Specific means for realizing this system will be described below.

[1188] 1. Data Collection Module

[1189] User: The user first inputs the type of crop they are growing and the location of their farm, which sets the basis for collecting and analyzing climate data.

[1190] Terminal: The terminal uses soil analysis sensors and crop growth monitoring cameras to periodically measure soil moisture, pH, and crop growth status. This acquired data is sent to the server in real time.

[1191] 2. Data transmission and analysis module

[1192] Server: The server obtains the necessary weather data from an external weather data API. It also receives soil data and crop growth data sent from the device, integrates them, and analyzes them using multimodal AI. This analysis derives optimal agricultural methods that take into account soil moisture, pH value, predicted precipitation, crop growth stage, and other factors.

[1193] 3. Advice Generation Module

[1194] Server: Based on the analysis results, the server generates advice to propose specific agricultural methods. For example, it recommends irrigation if soil moisture is low, or drainage measures if rainfall is forecast. The generated advice is sent to the user's mobile information terminal in real time.

[1195] 4. Robot Control Module

[1196] Terminal: Receives advice sent from the server and carries out the actual agricultural work based on that instruction. For example, if an instruction is given to irrigate, it will automatically water the area, and if drainage measures are required, it will activate the drainage system. This is done using automated machinery on the farm.

[1197] Specific examples

[1198] Let's take the example of an urban farmer growing tomatoes on a rooftop farm in a city.

[1199] 1. User: A user logs into the system and inputs that the crop being grown is "tomatoes" and that the farm is located "inside the city."

[1200] 2. Terminal: The terminal uses sensors to measure the moisture and pH of the soil, and a camera to record the tomato's growth stage. The collected data is sent to the server in real time.

[1201] 3. Server: The server retrieves the latest weather information for the city from an external weather data API, integrates it with soil humidity, acidity, and tomato growth status, and analyzes it. As a result of the analysis, it is determined that the soil humidity is low and that irrigation is necessary.

[1202] 4. Server: Generates irrigation advice and sends it to the user's mobile device in real time.

[1203] 5. Terminal: Receives advice from the server and automatically starts irrigation, providing the right amount of water to the tomatoes.

[1204] Prompt Sentence Examples

[1205] "Build an AI model that recommends irrigation when soil moisture is below 30% and no rain is forecast."

[1206] "Build an application that monitors the growth of tomatoes on an urban farm and suggests optimal farming practices."

[1207] This will enable users to receive analysis results and advice in real time, enabling quick and effective agricultural management. It is also expected that the automation of agricultural work will progress, leading to efficient and sustainable agriculture.

[1208] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1209] Step 1:

[1210] Users log in to the system and input the type of crop they are growing and the location of their farm, which then sets the criteria for collecting and analyzing climate data. The input is the type of crop and location, and the output is the set criteria.

[1211] Step 2:

[1212] The device uses soil analysis sensors and crop growth monitoring cameras to periodically measure soil moisture, pH, and crop growth. The measurement data is sent to the server in real time. The input is the measurement value of each sensor, and the output is data for analysis sent to the server.

[1213] Step 3:

[1214] The server acquires external weather data through the weather data API and integrates it with soil data and crop growth data sent from the terminal. The inputs are weather data, soil data, and crop growth data, and the output is the integrated dataset.

[1215] Step 4:

[1216] The server analyzes the integrated data using multimodal artificial intelligence to determine optimal agricultural practices, taking into account soil moisture, pH, predicted precipitation, crop growth stage, etc. The input is the integrated dataset, and the output is the analysis result (e.g., whether irrigation is required).

[1217] Step 5:

[1218] Based on the analysis results, the server generates advice to propose specific agricultural methods. For example, if the soil moisture is low, it recommends irrigation. The generated advice is sent to the user's mobile information terminal in real time. The input is the analysis results, and the output is the advice sent to the user.

[1219] Step 6:

[1220] The user's mobile information device receives the notification from the server and displays the analysis results and advice to the user. The user can then take the necessary action based on the displayed advice. The input is the notification from the server, and the output is the analysis results and advice presented to the user.

[1221] Step 7:

[1222] The terminal receives advice sent from the server and controls the automated machinery on the farm based on the instructions to carry out agricultural work. For example, if an irrigation command is issued, it will automatically water the farm, and if drainage measures are required, it will activate the drainage system. The input is the advice from the server, and the output is the agricultural work that has been carried out.

[1223] In this way, users can receive real-time advice on optimal farming techniques, leading to efficient and sustainable farming.

[1224] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1225] The agricultural support system of the present invention collects weather data, soil analysis data, and crop growth status data, analyzes this data using multimodal AI, proposes efficient and environmentally friendly agricultural methods, and executes these methods using automated robots. The present invention also incorporates an emotion engine that recognizes the user's emotions, and by utilizing the user's emotion data in the analysis, it is possible to provide more personalized advice. The configuration and operation of the system of the present invention are described in detail below.

[1226] System configuration

[1227] 1. Data Collection Module

[1228] Users log in to the system and enter the type of crop they are growing and the location of their farm, which determines the scope of data to be collected.

[1229] Terminal (robot): Using a soil analysis sensor and a crop growth monitoring camera, it collects data measuring soil humidity, pH value, and crop growth status. It also collects user emotion data through voice recognition and facial expression analysis.

[1230] 2. Data transmission and analysis module

[1231] Server: Receives soil data, crop growth data, and user emotion data sent from the device, and integrates it with weather data obtained from an external weather data API. Using multimodal AI, this data is analyzed to comprehensively assess soil condition, crop status, and weather conditions.

[1232] 3. Advice Generation Module

[1233] Server: Based on the results of multimodal AI analysis, the server generates advice proposing optimal farming methods. It also takes into account the user's emotional data and includes stress reduction measures and warnings as needed.

[1234] 4. Robot Control Module

[1235] Terminal (robot): Receives advice sent from the server and performs agricultural tasks based on the instructions. For example, if an instruction is given to irrigate, it will automatically water the land, and if an instruction is given to take drainage measures, it will activate the drainage system.

[1236] Specific examples

[1237] A specific example of using the system of the present invention will be described below.

[1238] Consider an urban farmer growing tomatoes on a rooftop farm in Tokyo.

[1239] 1. User: The user logs in to the system and inputs that the crop they are growing is "tomatoes" and that the farm is located in "Tokyo." The system also recognizes that the collected emotional data includes the user's voice and facial expressions while working.

[1240] 2. Terminal (Robot): The robot measures the soil moisture and pH value with sensors, records the tomato growth stage with a camera, and simultaneously collects emotional data from the user's voice and facial expressions.

[1241] 3. Server: The server retrieves the latest weather information for Tokyo from the weather data API, and then analyzes it together with soil data, crop growth data, and the user's emotional data. The analysis results indicate that the soil moisture is particularly low and that irrigation is necessary. The emotion engine also detects that the user is feeling stressed.

[1242] 4. Server: Based on the analysis results, generate irrigation advice along with relaxation activity recommendations, for example, "Perform irrigation and take a short break in between."

[1243] 5. Terminal (Robot): The robot starts irrigation and provides the appropriate amount of water. At the same time, a notification is displayed encouraging the user to take a break.

[1244] In this way, this system supports efficient and sustainable agriculture by integrating and analyzing environmental data and user emotional data to propose personalized agricultural methods.

[1245] The processing flow will be explained below.

[1246] Step 1:

[1247] User: Logs into the system and inputs the type of crop being grown (e.g., tomatoes) and the location of the farm (e.g., Tokyo). This sets up the basic information that allows the system to collect and analyze the appropriate data.

[1248] Step 2:

[1249] Terminal (robot): Using a soil analysis sensor, it measures the soil moisture and pH value in the farm. At the same time, it uses a crop growth monitoring camera to record the tomato growth status (e.g., height, number of leaves, color). In addition, it uses voice recognition and facial expression analysis sensors to collect emotional data from the user's voice and facial expressions. The collected data is sent to the server in real time.

[1250] Step 3:

[1251] Server: Obtains the latest weather data for Tokyo (temperature, humidity, precipitation, wind speed, etc.) from an external weather data API. This data is also analyzed along with other collected data.

[1252] Step 4:

[1253] Server: Integrates soil data, crop growth data, and user emotional data sent from the device. This data is input into the multimodal AI for analysis. Specifically, it determines the need for irrigation if the soil moisture is low, the stage of crop growth, and the user's stress level.

[1254] Step 5:

[1255] Server: Based on the analysis results, the server generates specific advice on agricultural techniques. For example, if the soil moisture is low, the server generates the advice "Please irrigate." If the user's emotional data indicates a high stress level, the server adds the recommendation "Please take a break to relax while irrigating."

[1256] Step 6:

[1257] Server: Sends advice to the robot, including specific work instructions (e.g., start irrigation, recommend breaks).

[1258] Step 7:

[1259] Terminal (Robot): Automatically performs agricultural tasks based on advice received from the server. For example, if an irrigation instruction is given, it will automatically start watering. It also displays a notification to the user to take a break to relax.

[1260] This series of steps allows users to easily practice efficient and sustainable farming while also optimally managing their own emotional state. The system aims to achieve both optimal farming practices and user emotional management by collecting and analyzing data in real time.

[1261] Example 2

[1262] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1263] While conventional agricultural support systems collect and analyze weather data, soil analysis data, and crop growth status data, they lack the ability to provide personalized advice or propose agricultural methods that take into account the user's emotional data. Furthermore, there are limited means for implementing automated agricultural work using the collected data, making it difficult to achieve efficient and sustainable agriculture.

[1264] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1265] In this invention, the server includes means for collecting and analyzing weather data, soil analysis data, crop growth status data, and user emotion data, means for generating optimal agricultural techniques and advice for the user on stress reduction measures and cautions based on the analysis results, and means for transmitting the generated advice to the agricultural machine and having the machine perform work based on the instructions. This enables comprehensive data analysis that incorporates user emotion data, making it possible to propose and implement efficient and personalized agricultural techniques.

[1266] "Climate data" refers to information about weather, temperature, humidity, precipitation, wind speed, and other meteorological conditions.

[1267] "Soil analysis data" refers to information about the physical and chemical properties of soil, such as soil moisture, pH, and nutrient content.

[1268] "Crop growth status data" refers to information about the state of growth of crops, such as the stage of growth of the crops, leaf color, and the presence or absence of pests or diseases.

[1269] "User emotion data" refers to information about the user's emotional state analyzed from their voice and facial expressions.

[1270] "Multimodal AI" refers to artificial intelligence that performs integrated analysis of multiple different types of data (e.g., text data, image data, audio data, etc.).

[1271] "Analysis results" refers to information obtained as a result of analysis conducted by AI based on collected data.

[1272] "Agricultural techniques" refer to methods and techniques for carrying out agricultural work efficiently.

[1273] "Agricultural machinery" means an automated machine used to perform agricultural work.

[1274] "Stress reduction measures" are suggestions and measures to reduce the user's stress.

[1275] A "warning" is a notification or warning that prompts the user to take some kind of action.

[1276] The agricultural support system according to the present invention is composed of multiple modules that work in conjunction with each other to provide efficient and personalized agricultural support. This system is made up of a data collection module, a data transmission and analysis module, an advice generation module, and a robot control module. Details and specific operations of each module are explained below.

[1277] Data Collection Module

[1278] User:

[1279] Users first log into the system and input the type of crop they are growing and the location of their farm, which sets the scope of data to be collected.

[1280] Terminal (Robot):

[1281] The device is equipped with a soil analysis sensor and a crop growth monitoring camera. The soil analysis sensor measures the soil's moisture and pH value, and the crop growth monitoring camera records the crop's growth status. It also collects user emotion data through voice recognition and facial expression analysis. All of this data is collected in real time.

[1282] Data Transmission and Analysis Module

[1283] Terminal (Robot):

[1284] The collected soil data, crop growth data, and user emotion data are sent to a server via wireless communication (e.g., Wi-Fi).

[1285] server:

[1286] The server acquires weather data from an external weather data API and integrates it with data sent from the device. Specifically, weather data, soil data, crop growth data, and user emotion data are all stored in a database and analyzed using multimodal AI.

[1287] Advice Generation Module

[1288] server:

[1289] Multimodal AI is used to comprehensively analyze data. The analysis integrates weather data, soil data, growth data, and emotional data to generate results. Based on the analysis results, advice is generated suggesting optimal farming methods. If the user is feeling stressed, advice on stress reduction measures and caution is also included.

[1290] For example, enter the following prompt into the server:

[1291] "Analyze the transmitted sensor and camera data and suggest optimal agricultural methods. If the soil moisture is below 60%, advise irrigation. Also, if the user is feeling stressed, provide advice on how to deal with it."

[1292] Robot Control Module

[1293] Terminal (Robot):

[1294] The system receives advice generated by the server and performs agricultural tasks based on the instructions. For example, it automatically waters the field when irrigation is requested, or activates the drainage system when drainage measures are requested. It also displays notifications encouraging the user to take a break.

[1295] A concrete example of the overall flow

[1296] Suppose an urban farmer is growing tomatoes on a rooftop farm in Tokyo. The user logs into the system and inputs that the crop is tomatoes and that the farm is in Tokyo. The terminal (robot) measures the humidity and pH value of the soil and records the tomato's growth stage with a camera. It also collects emotional data from voice and facial expressions. The server combines this data with weather data and produces an analysis result. Based on the analysis result, it determines that irrigation is necessary and suggests appropriate farming methods to the user. It also instructs the user to take a break if they are feeling stressed.

[1297] In this way, the system of the present invention comprehensively analyzes environmental data and user emotional data, and provides personalized agricultural methods, thereby supporting efficient and sustainable agriculture.

[1298] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1299] Step 1: Data entry

[1300] Users log in to the system and input the type of crop they are growing and the location of their farm. Based on this input data, the system sets the range of data to be collected and the required data items. Specifically, users access a dedicated application from their smartphone or PC and input the type of crop (e.g., tomatoes) and the location of their farm (e.g., Tokyo). The input data is sent to the server and saved in a configuration database.

[1301] Input: type of crop, farm location

[1302] Output: Configuration database updated

[1303] Step 2: Collect environmental data

[1304] The terminal (robot) uses soil analysis sensors and crop growth monitoring cameras to measure soil moisture, pH, and crop growth status. This data is collected periodically. The robot patrols the farm, measuring soil moisture and pH using the soil sensors and saving the results in its internal memory. At the same time, it records the crop growth stages using the monitoring cameras.

[1305] Input: None (data collected periodically)

[1306] Output: Measurement data (humidity, pH value, growth status)

[1307] Step 3: Collecting emotion data

[1308] The terminal (robot) analyzes the user's voice and facial expressions to collect emotional data. Specifically, when the robot is near the user, it activates a voice recognition system and facial expression analysis system. The words spoken and facial expressions of the user while working are analyzed in real time to generate emotional data.

[1309] Input: User's voice, facial expression video

[1310] Output: Emotion data

[1311] Step 4: Send data

[1312] The terminal (robot) transmits the collected soil data, crop growth data, and emotion data to the server via wireless communication (Wi-Fi). When the robot finishes collecting data, it automatically transfers the data to the server via the wireless communication module.

[1313] Input: Measurement data, emotion data

[1314] Output: Send data to the server

[1315] Step 5: Obtaining Weather Data

[1316] The server obtains the latest weather data from an external weather data API. Specifically, the server periodically sends requests to the API to obtain real-time weather data (temperature, humidity, precipitation, etc.). The obtained data is stored in an internal database.

[1317] Input: None (data acquired periodically)

[1318] Output: Weather data

[1319] Step 6: Data integration and analysis

[1320] The server integrates soil data, crop growth data, emotion data, and weather data sent from the device and analyzes them using multimodal AI.The server also extracts various data stored in the database and performs multidimensional analysis using deep learning models.

[1321] Input: soil data, crop growth data, emotion data, weather data

[1322] Output: Analysis results

[1323] Step 7: Advice Generation

[1324] The server generates optimal agricultural methods and advice for users based on the results of multimodal AI analysis. Based on the data derived from the analysis, it creates specific advice on irrigation and fertilizer application timing, pest control, and work efficiency. It also includes stress reduction measures and warnings based on the user's emotional data.

[1325] Specific prompt examples:

[1326] "Analyze the transmitted sensor and camera data and suggest optimal agricultural methods. If the soil moisture is below 60%, advise irrigation. Also, if the user is feeling stressed, provide advice on how to deal with it."

[1327] Input: Analysis results

[1328] Output: Advice content

[1329] Step 8: Submitting Advice

[1330] The server sends the generated advice to the terminal (robot), formats the advice into an appropriate format, and transmits it to the robot via wireless communication.

[1331] Input: Advice content

[1332] Output: Send advice to the robot

[1333] Step 9: Robot execution

[1334] The terminal (robot) performs agricultural work based on instructions from the server. The robot analyzes the advice it receives and begins the necessary agricultural work. For example, if the analysis results indicate that irrigation is necessary, the robot will activate the irrigation system and spray the appropriate amount of water. It will also display a notification encouraging the user to take a break.

[1335] Input: Advice content

[1336] Output: Execution of agricultural work, notification to user

[1337] (Application example 2)

[1338] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1339] In conventional production sites, environmental data and worker emotional data were not fully utilized, resulting in insufficient production efficiency and worker stress management. This made it difficult to find the optimal production method, which could have a negative impact on productivity and the work environment. The present invention aims to solve these problems by providing a system that comprehensively analyzes environmental data and worker emotional data and proposes the optimal production method.

[1340] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting weather data, means for collecting environmental data, means for collecting worker emotion data, means for analyzing the above data using multimodal AI, means for proposing an optimal production method based on the analysis results, and means for controlling equipment that automates the production process. This makes it possible to comprehensively analyze the collected data, improve production efficiency, and manage worker stress.

[1341] "Climate data" is information about weather conditions such as temperature, humidity, air pressure, precipitation, and wind speed.

[1342] "Environmental data" refers to information on various physical environments such as temperature, humidity, vibration, and noise within a factory or production site.

[1343] "Worker's emotional data" is information about the emotional state obtained by analyzing the worker's physiological responses such as voice and facial expressions.

[1344] "Multimodal AI" is an artificial intelligence technology that integrates and analyzes different types of data (e.g., voice, images, text, etc.).

[1345] "Production methods" refer to the procedures and methods of specific production processes and work processes.

[1346] "Equipment that automates production processes" refers to devices and systems that automate production activities within factories or production sites.

[1347] A "server" is a central computer in a system that analyzes and processes data, and stores and provides information.

[1348] The present invention is a system that collects weather data, environmental data, and worker emotion data at a production site, analyzes this data using multimodal AI, proposes optimal production methods, and automates the production process. Specific embodiments of the present invention are described in detail below.

[1349] System configuration

[1350] 1. Data Collection Module

[1351] The server is equipped with sensors and devices to collect weather and environmental data, as well as worker emotional data, including temperature, humidity, and vibration sensors, and an emotion recognition system that analyzes the voices and facial expressions of workers.

[1352] 2. Data transmission and analysis module

[1353] The collected data is sent to a server, which then integrates it and analyzes it using multimodal AI. This analysis uses various sensor APIs, emotion recognition APIs, and an HTTP request library.

[1354] 3. Advice Generation Module

[1355] Based on the analysis results, the server generates advice proposing optimal production methods. Specifically, if the temperature inside the factory is high, instructions are generated to activate the cooling system or encourage workers to take breaks.

[1356] 4. Robot Control Module

[1357] Upon receiving instructions from the server, the terminal (robot) performs automated tasks in the production process, such as operating the cooling system, adjusting humidity, and automatically transporting goods.

[1358] Hardware and software used

[1359] Hardware:

[1360] Temperature Sensor

[1361] Humidity Sensor

[1362] Vibration Sensor

[1363] Emotion Recognition System

[1364] Robots that carry out production processes

[1365] software:

[1366] Sensor API for collecting information

[1367] Emotion recognition API

[1368] HTTP request library

[1369] Multimodal AI analysis API

[1370] Specific examples

[1371] If environmental data on a factory production line indicates high temperature and humidity levels and that workers are feeling stressed, the system will activate the cooling system and send a notification to the worker's smartphone telling them to take a break. In this way, it is possible to maintain the health of workers without reducing production efficiency.

[1372] Prompt Sentence Examples

[1373] An optimization system using robots on factory production lines. Data is collected from temperature, humidity, and vibration sensors, as well as worker emotional data. Multimodal AI is used to analyze the data and generate optimal production methods and robot operation instructions. Instructions include activating the cooling system when temperatures are high, and notifications to encourage workers to take breaks.

[1374] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1375] Step 1:

[1376] Users log in to the system using a smartphone or head-mounted display and input information about the production line and the equipment to be managed. This allows the system to identify the target for monitoring and data collection. The input information includes the type of equipment, its installation location, and worker information. Based on this input data, the system starts operating the necessary sensors and devices.

[1377] Step 2:

[1378] The terminals (sensors) collect environmental data (temperature, humidity, vibration) within the factory and emotional data from workers' voices and facial expressions. Specifically, the temperature sensor detects the temperature on-site, the humidity sensor measures humidity, and the vibration sensor records the vibration state of the equipment. In addition, the emotion recognition system analyzes the workers' faces and voices to generate emotional data. The collected data is formatted and sent to a server in real time.

[1379] Step 3:

[1380] The server receives the environmental data and emotional data sent from the device, integrates them, and stores them in a database. Specifically, the server processes HTTP requests, retrieves data via various data APIs, and stores it. This allows the system to grasp the overall environmental situation and the worker's emotional state.

[1381] Step 4:

[1382] The server analyzes the data stored in the database using a multimodal AI model. Specifically, it receives temperature, humidity, vibration, and emotion data as input and analyzes the correlation between each piece of data. As a result of the analysis, it evaluates how specific environmental conditions and worker status affect production efficiency. The output is generated as optimal production methods and advice.

[1383] Step 5:

[1384] The generated production methods and advice are sent from the server to the terminal (robot). Specific instructions are transmitted to the robot using APIs and communication protocols. Examples include instructions to activate the cooling system if the temperature is high, or notifications to encourage workers to take a break if they are feeling stressed.

[1385] Step 6:

[1386] The terminals (robots) carry out production processes based on instructions received from the server. Specific operations include operating the cooling system, adjusting humidity, and transporting parts. This allows for efficient operation of the production line and optimization of the working environment.

[1387] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1388] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1389] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1390] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1391] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1392] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1393] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1394] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1395] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1396] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1397] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1398] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1399] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1400] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1401] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1402] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1403] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1404] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1405] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1406] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1407] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1408] The following is further disclosed regarding the above embodiment.

[1409] (Claim 1)

[1410] a means for collecting climate data;

[1411] a means for collecting soil analysis data;

[1412] a means for collecting data on the state of growth of the crop;

[1413] A means of analyzing the above data using multimodal AI, and

[1414] A means to propose optimal agricultural methods based on the analysis results, and

[1415] A system including means for controlling a robot that automates agricultural work within a farm.

[1416] (Claim 2)

[1417] means for transmitting the collected weather data, soil analysis data, and crop growth status data to a server;

[1418] A means to analyze the data on the server and generate optimal agricultural methods;

[1419] 10. The system of claim 1, further comprising means for transmitting the generated farming advice to a robot within the farm.

[1420] (Claim 3)

[1421] 2. A system according to claim 1, including means for taking into account, inter alia, the moisture content, pH value of the soil and the stage of growth of the crop when analyzing the collected data.

[1422] "Example 1"

[1423] (Claim 1)

[1424] a means for a user to input crop type and farm location information;

[1425] a means for measuring the moisture and pH value of the soil and the growth status of the crops using a soil analysis sensor and a crop growth monitoring camera;

[1426] means for transmitting said data to a server in real time;

[1427] a means for obtaining climate data from an external weather data API;

[1428] A means of analyzing the collected data using multimodal AI,

[1429] A means to propose optimal agricultural methods based on the analysis results, and

[1430] A system including means for controlling a robot that automatically performs farm work within a farm based on advice sent from a server.

[1431] (Claim 2)

[1432] A means for integrating and transmitting the collected weather data, soil moisture data, pH value data, and crop growth status data to a server for analysis;

[1433] A means for generating optimal agricultural methods as a result of the analysis;

[1434] 2. The system according to claim 1, further comprising means for transmitting the generated advice on agricultural techniques to a robot in a farm and for the robot to carry out farm work based on the advice.

[1435] (Claim 3)

[1436] 2. The system of claim 1, including means for taking into account, inter alia, soil moisture, pH value, expected precipitation, and stage of crop development when analyzing the collected data.

[1437] "Application Example 1"

[1438] (Claim 1)

[1439] a means for collecting climate data;

[1440] a means for collecting soil analysis data;

[1441] a means for collecting data on the state of growth of the crop;

[1442] a means for analyzing said data using multimodal artificial intelligence; and

[1443] A means to propose optimal agricultural methods based on the analysis results, and

[1444] A means to notify users of analysis results and agricultural method advice in real time,

[1445] A system including means for controlling machinery that automates agricultural work within a farm based on the analysis results.

[1446] (Claim 2)

[1447] means for transmitting the collected weather data, soil analysis data and crop growth status data to a central processing unit;

[1448] means for analyzing the data on a central processing unit and generating optimal agricultural practices;

[1449] means for transmitting the generated farming advice to machines within the farm;

[1450] 2. The system according to claim 1, further comprising means for notifying the user of the analysis results and advice on a mobile information terminal.

[1451] (Claim 3)

[1452] 2. The system of claim 1, further comprising means for analyzing the collected data, taking into account, inter alia, soil moisture, acidity and crop growth stage, and means for displaying the results of the analysis on a user's personal digital assistant.

[1453] "Example 2: Combining Emotion Engines"

[1454] (Claim 1)

[1455] a means for collecting climate data;

[1456] a means for collecting soil analysis data;

[1457] a means for collecting data on the state of growth of the crop;

[1458] means for collecting user emotion data;

[1459] The above data will be analyzed using multimodal AI to make a comprehensive judgment.

[1460] A means to propose optimal agricultural methods based on the analysis results, and

[1461] a means for controlling the agricultural machine to execute the proposed agricultural method;

[1462] A system including a means for generating stress reduction and caution advice based on a user's emotional data.

[1463] (Claim 2)

[1464] means for transmitting the collected weather data, soil analysis data, crop growth status data, and user emotion data to a server;

[1465] A means for integrating and analyzing these data on a server to generate optimal agricultural methods and advice for users;

[1466] 2. The system according to claim 1, further comprising means for transmitting the generated advice on agricultural techniques and advice on stress reduction measures and cautions to the agricultural machine, and for the agricultural machine to carry out work based on the instructions.

[1467] (Claim 3)

[1468] 2. The system according to claim 1, further comprising means for taking into account, in particular, soil moisture, pH value, crop growth stage and user emotion data when analyzing the collected data.

[1469] "Application example 2 when combining emotion engines"

[1470] (Claim 1)

[1471] a means for collecting climate data;

[1472] a means for collecting environmental data;

[1473] A means for collecting worker emotion data;

[1474] A means of analyzing the above data using multimodal AI, and

[1475] A means to propose optimal production methods based on the analysis results, and

[1476] A system that includes a means for controlling equipment that automates a production process.

[1477] (Claim 2)

[1478] means for transmitting the collected weather data, environmental data, and worker emotion data to a server;

[1479] A means of analyzing data on a server and generating optimal production methods;

[1480] 2. The system of claim 1, further comprising means for transmitting the generated production technique advice to equipment in the production process.

[1481] (Claim 3)

[1482] 2. A system according to claim 1, including means for taking into account, inter alia, temperature, humidity and the emotional state of the worker when analysing the collected data. [Explanation of symbols]

[1483] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for collecting climate data; a means for collecting soil analysis data; a means for collecting data on the state of growth of the crop; A means of analyzing the above data using multimodal AI, and A means to propose optimal agricultural methods based on the analysis results, and A system including means for controlling a robot that automates agricultural work within a farm.

2. means for transmitting the collected weather data, soil analysis data, and crop growth status data to a server; A means to analyze the data on the server and generate optimal agricultural methods; 2. The system of claim 1, further comprising means for transmitting the generated farming advice to a robot within the farm.

3. 2. A system according to claim 1, including means for taking into account, in particular, the moisture content, pH value of the soil and the stage of growth of the crop when analyzing the collected data.

Images