system

Abstract

Provide a system. A means for photographing the growth of agricultural crops with a camera; means for measuring environmental conditions with sensors; means for transmitting data from said camera and sensor to a server; A means for analyzing the data using a generative artificial intelligence model and generating a proposal for an optimal development environment; A means for adjusting the growing environment based on the analysis results; A means for informing the user of an efficient operation method; a means of receiving feedback; and A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention addresses the need for an automated system that can improve the efficiency of agricultural cultivation and ensure a stable supply of high-quality crops, against the backdrop of the serious problems of an aging population and labor shortages in the agricultural sector. While current technology has made progress in automating some aspects of these issues, it still has problems, such as an inability to flexibly adapt to environmental changes and difficulty in providing customized solutions tailored to the condition of individual crops. These problems make it difficult to achieve sustainable agriculture. [Means for solving the problem]

[0005] To solve these problems, the present invention provides the following means. Specifically, it provides a system including a means for photographing crop growth with a camera, a means for measuring environmental conditions with a sensor, and a means for transmitting this data to a server. The system further includes a means for analyzing this data with a generative artificial intelligence model and generating a proposal for an optimal cultivation environment, a means for adjusting the cultivation environment based on the analysis results, a means for notifying the user of efficient operating methods, and a means for receiving feedback. This makes it possible to propose optimal environmental adjustments and efficient operating methods according to the state of the crops, thereby improving the efficiency and sustainability of agriculture.

[0006] A "camera" is an optical device used to photograph the growing conditions of agricultural crops.

[0007] A "sensor" is a device for measuring surrounding environmental conditions (temperature, humidity, light intensity, etc.).

[0008] A "server" is a computer system that receives, stores, and analyzes data sent from cameras and sensors.

[0009] A "generative artificial intelligence model" is a machine learning algorithm model that analyzes data and generates proposals for the optimal development environment.

[0010] "Data" refers to image data captured by a camera and data on environmental conditions measured by a sensor.

[0011] "Proposal of a growing environment" is information indicating the optimal growing conditions for crops based on the analysis results.

[0012] "Environmental adjustment" refers to specific operations (e.g., watering, fertilizing, adjusting lighting) performed to realize the proposed growing environment.

[0013] "Operation method" refers to procedures and instructions that are communicated to the user for carrying out efficient agricultural operations.

[0014] "Feedback" refers to information about operation results and areas for improvement provided by users to the system. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The system of the present invention is designed to realize efficient and sustainable agricultural management. This system consists of cameras, sensors, generative AI models, servers, terminals, and users.

[0037] 1. Initial Setup

[0038] server

[0039] During initial setup, the server checks the connections of all devices and tests that the cameras and sensors are working properly. It then provides an interface for users to input information about the crops they are growing, and retrieves and stores the input data from the agricultural database. This ensures that the system is ready to provide an optimal growing environment right from the start.

[0040] 2. Data Collection

[0041] Terminal (robot)

[0042] The device patrols the farmland, taking images of the crops with its camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors. This data is sent to a server in real time. Accurate data collected at this stage ensures the accuracy of subsequent analysis.

[0043] 3. Data Analysis

[0044] server

[0045] The server analyzes the received image data and sensor data using a generative artificial intelligence model. Image analysis determines the growth status of crops and checks for disease and pest damage based on leaf color, shape, and size. Environmental assessment evaluates whether the current growing environment is appropriate based on the acquired sensor data and determines whether any deficient environmental conditions need to be corrected. The analysis results are generated as a proposal, and the next action is decided.

[0046] 4. Providing the optimal environment

[0047] Terminal (robot)

[0048] The terminal receives instructions from the server and performs specific operations to optimize the growing environment for the crops, such as watering, fertilizing, and adjusting lighting as needed, ensuring that the crops are grown under optimal conditions.

[0049] 5. Operational suggestions

[0050] server

[0051] Based on the analysis results, the server suggests efficient operation methods to the user. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. For example, specific instructions such as "crops in a specific area require additional watering" are provided.

[0052] 6. Gathering Feedback

[0053] User

[0054] After carrying out actual work based on the suggestions, users provide feedback to the system on the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. This allows the system to continually evolve, enabling more effective agricultural management.

[0055] The above is a specific embodiment of the "Auto Farm Navigation" system. This system not only enables efficient and effective crop cultivation management, but also greatly contributes to the realization of sustainable agriculture. By understanding the specific operating procedures and system flow, users can make the most of this system.

[0056] The processing flow will be explained below.

[0057] Step 1:

[0058] server

[0059] When the system starts up, the server checks the connections of all devices, including cameras, sensors, and robots. It tests each device for normal operation and notifies users if any abnormalities are detected. Users enter information about the crops they are growing through the interface, and based on that, related data is retrieved and saved from the agricultural database.

[0060] Step 2:

[0061] Terminal (robot)

[0062] The terminal (robot) uses a camera to capture images of the growing conditions of the crops at each location. It also uses sensors to measure the soil humidity, temperature, and light intensity. The acquired data is sent to a server in real time.

[0063] Step 3:

[0064] server

[0065] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. It then analyzes the image data using a generative artificial intelligence model. Specifically, it evaluates the color, shape, and size of the leaves and determines the speed of the adult. It also analyzes environmental sensor data to evaluate whether humidity, temperature, and light intensity are within appropriate ranges. Based on the analysis results, it generates a proposal for the optimal growing environment.

[0066] Step 4:

[0067] server

[0068] Based on the generated suggestions, the server generates specific instructions for adjusting the environment, such as "add 500 ml of water to area A" or "apply fertilizer to area B." These instructions are sent to the terminal (robot).

[0069] Step 5:

[0070] Terminal (robot)

[0071] The terminal (robot) adjusts the environment based on instructions from the server. For example, it may water or fertilize the appropriate areas. It may also adjust the lighting. Once the adjustment is complete, it reports the results to the server.

[0072] Step 6:

[0073] server

[0074] The server receives the results of the environmental adjustments from the robot and checks whether the suggestions were implemented properly. Furthermore, it generates suggestions for return work to the user based on the crop growth status. Specifically, the suggestions include, "The harvest time is approaching in a specific zone," or "Signs of pests and diseases are visible and need to be addressed."

[0075] Step 7:

[0076] server

[0077] The system notifies the user of suggestions for efficient operation methods, and provides specific instructions on tasks and precautions the user should take based on the suggestions. For example, it may notify the user that "crops in a specific area require additional watering."

[0078] Step 8:

[0079] User

[0080] The user follows the suggestions from the server and performs actual operations, such as harvesting or pest control. The results of the work and any improvements needed on-site are then fed back to the system, providing reference data for future analyses and to improve the quality of suggestions.

[0081] Through these processing steps, the system of the present invention can streamline crop management and support sustainable agricultural operations.

[0082] Example 1

[0083] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0084] To achieve efficient and sustainable agricultural operations, it is necessary to monitor crop growth and environmental conditions in real time and provide an appropriate growing environment. However, conventional technologies mainly rely on manual monitoring and environmental adjustment, which is labor-intensive and time-consuming, making it difficult to constantly maintain an optimal environment. It is also difficult to respond quickly to changes in environmental conditions, which can affect crop quality and yield.

[0085] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0086] In this invention, the server includes means for photographing the growth of crops with a camera, means for measuring environmental conditions with a sensor, means for transmitting data from the camera and the sensor to an information processing device, means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal cultivation environment, means for adjusting the cultivation environment based on the generated proposal, means for notifying a user of an efficient operation method based on the analysis results, and means for receiving feedback from the user. This makes it possible to monitor the growth of crops in real time, quickly respond to changes in environmental conditions, and provide an optimal cultivation environment.

[0087] A "camera" is an image capturing device for capturing images of crop growth.

[0088] A "sensor" is a device used to measure environmental conditions such as soil humidity, temperature, and light intensity.

[0089] An "information processing device" is a computer system for receiving and processing data transmitted from cameras and sensors.

[0090] A "generative artificial intelligence model" is a machine learning model used to analyze collected data and evaluate the growth status of crops and environmental conditions.

[0091] "Analysis" is the process of using a generative artificial intelligence model to evaluate the content of the received data and determine the condition of the crops and any necessary environmental adjustments.

[0092] "Growing environment" refers to the physical conditions and environmental factors (humidity, temperature, light intensity, etc.) required for crop growth.

[0093] "Feedback" refers to information that is input by the user regarding the results of work performed and improvements to the system's proposals, and is used to improve the system's analysis and proposals from the next time onwards.

[0094] "User" refers to a person or organization that uses this system to grow and manage agricultural crops.

[0095] The system of the present invention is an automated agricultural management system for achieving efficient and sustainable agricultural operations. This system is composed of a camera, sensors, a generative AI model, an information processing device, a terminal, and a user.

[0096] Initial Setup

[0097] server

[0098] During initial setup, the server checks the connectivity of all devices and tests that the cameras and sensors are working properly. Specifically, it takes test images with the cameras and collects initial data (such as temperature and humidity) from the sensors. Next, it provides an interface for users to input information about the crops they are growing. Users enter data such as the type of crop, its growth stage, and specific growing conditions. This information is then retrieved from an agricultural database and stored on the server.

[0099] Data collection

[0100] Terminal (robot)

[0101] The device patrols the farmland, taking images of the crops with its camera. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. This data is sent to a server in real time. For example, the device could take images of the entire farmland at a specific time each day to collect environmental data.

[0102] Data analysis

[0103] server

[0104] The server analyzes the received image data and sensor data using a generative AI model. Image analysis determines the growth status of the crops and checks for disease and pest damage based on leaf color, shape, and size. It also performs environmental assessments based on sensor data to determine whether the current growing environment is appropriate. The generative AI model uses deep learning to improve data accuracy. For example, it can produce analysis results such as, "The leaves of the crops have turned yellow, so there is a high possibility that they are lacking nitrogen fertilizer."

[0105] Providing the optimal environment

[0106] Terminal (robot)

[0107] The device receives instructions from the server and performs operations to optimize the growing environment for crops. For example, if the data measured by the sensor indicates a lack of water, the device will automatically water the crops. It will also apply fertilizer and adjust the lighting as needed. This ensures that crops are grown under optimal conditions.

[0108] Operation suggestions

[0109] server

[0110] Based on the analysis results, the server suggests efficient operation methods to the user. For example, it notifies the user's device of specific instructions such as "crops in a specific area require additional watering." This allows the user to follow the instructions and improve efficiency.

[0111] Collecting feedback

[0112] User

[0113] After carrying out the work based on the suggestions, the user can then provide feedback to the system about the results and areas for improvement. For example, the user can enter specific information such as, "After watering the plant, the leaves suddenly became healthier." This feedback will be used as important data to improve the quality of future analyses and suggestions.

[0114] (Example)

[0115] For example:

[0116] Example prompt:

[0117] Crops are not growing well in the growing environment. Please analyze the data collected by the sensor and suggest the best solution.

[0118] Example result:

[0119] The server uses sensor data to determine if the soil moisture is too low and instructs the device, "Additional watering is required this morning." The device receives instructions from the server and supplies the appropriate amount of water to the specified location. The user then observes changes in growth and provides feedback on the effects.

[0120] This system not only enables efficient and effective crop management, but also enables sustainable agricultural operations.

[0121] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0122] Step 1:

[0123] The server checks the connection of all devices (cameras, sensors, terminals) and tests whether they are working properly. As input, it receives initial data from the cameras and sensors (e.g., test images, temperature, humidity, etc.), and as output, it sends a signal to each device to confirm that the device is working properly.

[0124] Step 2:

[0125] The user inputs information about the crops they are growing using an interface provided by the server. The input details, such as the type of crop, its growth stage, and cultivation conditions, are sent to the server, which then retrieves and stores this information from an agricultural database.

[0126] Step 3:

[0127] The terminal (robot) patrols the farmland, taking images of the crops with a camera. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. It collects crop images and environmental data as input, and sends this data to a server in real time as output.

[0128] Step 4:

[0129] The server receives image data and sensor data sent from the device. The data received as input is sent to a generative AI model, which analyzes it. The output is an evaluation result regarding the growth status of crops and environmental conditions.

[0130] Step 5:

[0131] The server generates optimal growing environment recommendations based on the data analyzed by the generative AI model. It uses the analysis results as input and generates specific growing environment recommendations (e.g., watering amounts, fertilization needs, lighting adjustments, etc.) as output.

[0132] Step 6:

[0133] The server sends the generated cultivation environment proposal to the terminal. It receives the cultivation environment proposal as input and sends instructions as output to the terminal. The terminal performs operations to optimize the crop cultivation environment based on these instructions. Specifically, it automatically waters the crops, applies fertilizer, and adjusts the lights.

[0134] Step 7:

[0135] The server notifies the user of efficient operation methods based on the analysis results. It takes the analysis results and generated suggestions as input and sends notifications about efficient operation methods to the user's device as output. For example, it may instruct the user that "crops in a specific area require additional watering."

[0136] Step 8:

[0137] The user performs the actual work based on the suggestions from the server. The user receives instructions from the server as input and sends the results of the work as feedback to the server as output. For example, the user can input specific information such as "After watering, the leaves of the plant suddenly became healthier."

[0138] Step 9:

[0139] The server receives feedback from users and uses it to improve the quality of the next analysis and proposal. Feedback information is taken in as input, and output is used to improve the generative AI model and increase the accuracy of analysis. This allows the system to continuously evolve, enabling more effective agricultural operations.

[0140] (Application example 1)

[0141] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0142] Modern agriculture requires real-time monitoring of crop growth conditions and the provision of an appropriate growing environment. However, current manual management methods make it difficult to quickly respond to small changes in the growing environment. Furthermore, a lack of technology for efficient agricultural management makes sustainable agricultural operations difficult. Furthermore, managing large areas of farmland requires a great deal of manpower and effort, resulting in high costs.

[0143] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0144] In this invention, the server includes means for photographing crop growth with a camera, means for measuring environmental conditions with a sensor, means for transmitting data from the camera and the sensor to the server, means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal cultivation environment, means for adjusting the cultivation environment based on the analysis results, means for notifying the user of efficient operation methods, means for receiving feedback, means for being mounted on the autonomous agricultural vehicle, and means for transmitting instructions to the autonomous agricultural vehicle. This enables crop growth to be managed in an optimal environment in real time, thereby realizing efficient agricultural management and sustainable agriculture.

[0145] The "camera" is a device that photographs the growth of crops and collects visual data.

[0146] A "sensor" is a device for measuring environmental conditions (such as humidity or temperature).

[0147] A "server" is a central processing unit that receives and analyzes data sent from cameras and sensors.

[0148] A "generative artificial intelligence model" is a machine learning model that analyzes collected data and generates suggestions for growing crops.

[0149] "Optimal growing environment proposals" are proposals based on the analysis results that show the best environmental settings and operation methods to promote crop growth.

[0150] "Means for adjusting the growing environment" refers to devices or methods for actually changing the environmental conditions of agricultural crops (such as the amount of light, water, and temperature) based on the analysis results.

[0151] The "means for notifying the user of an efficient operation method" refers to a method or device for conveying specific operation instructions to the user based on the analysis results.

[0152] "Means for receiving feedback" refers to a method or device for sending back to the system the results of operations performed by the user in accordance with instructions and suggestions for improvement.

[0153] An "autonomous agricultural vehicle" is a vehicle that operates autonomously according to the growth status of crops and environmental conditions, and manages the growing environment.

[0154] "Means for sending instructions to an autonomous agricultural vehicle" refers to a method or device for transmitting specific operational instructions from a server to an autonomous agricultural vehicle.

[0155] The present invention relates to a system for highly automating the cultivation and environmental management of agricultural crops in an autonomous agricultural vehicle. The system for carrying out the present invention is configured as follows.

[0156] 1. Initial Setup

[0157] server

[0158] The server checks the connection of all devices during the initial setup phase, tests whether the cameras and sensors are working properly, provides an interface for users to input information about the crops they are growing, and retrieves and stores the input data from an agricultural database.

[0159] 2. Data Collection

[0160] Autonomous agricultural vehicles

[0161] The autonomous agricultural vehicle travels around the field, taking images of the crops with a camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors, and transmits this data to a server in real time.

[0162] 3. Data Analysis

[0163] server

[0164] The server analyzes the received image data and sensor data using a generative artificial intelligence model. Image analysis determines the growth status of crops and checks for disease and pest damage based on leaf color, shape, and size. Environmental assessment evaluates whether the current growing environment is appropriate based on the acquired sensor data and determines whether any deficient environmental conditions need to be corrected. The analysis results are generated as a proposal, and the next action is decided.

[0165] 4. Providing the optimal environment

[0166] Autonomous agricultural vehicles

[0167] The autonomous agricultural vehicle receives instructions from the server and performs specific operations to optimize the growing environment for crops, such as watering, fertilizing, and adjusting lights as needed, ensuring that crops are grown under optimal growing conditions.

[0168] 5. Operational suggestions

[0169] server

[0170] Based on the analysis results, the server suggests efficient operation methods to the user. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. Specific instructions include information such as "crops in a specific area require additional watering."

[0171] 6. Gathering Feedback

[0172] User

[0173] After carrying out actual work based on the suggestions, users provide feedback to the system on the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. This allows the system to continually evolve, enabling more effective agricultural management.

[0174] Hardware and Software Used

[0175] Hardware: Cameras, humidity sensors, temperature sensors, autonomous agricultural vehicles

[0176] Software: Python, OpenCV (image processing library), requests (HTTP communication library), generative artificial intelligence model using deep learning

[0177] Example prompt

[0178] In a real application, the next action is determined based on the data analysis results and instructions are sent to the vehicle. For example, if data is collected at a humidity of 45% and a temperature of 24°C, the prompt sentence would be as follows:

[0179] "Data Collection: Environmental data was collected at 45% humidity and 24°C. Based on the analysis, area A1 requires additional watering. Instructions: Add 2 liters of water to area A1."

[0180] This will enable optimal environmental management of crops in real time, resulting in efficient agricultural operations.

[0181] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0182] Step 1:

[0183] As an initial setup, the server checks the connection of all devices and checks that the cameras and sensors are working properly. It also obtains information about the crops being grown based on user input, accesses an agricultural database, and stores the necessary data.

[0184] Input: User input of crop information, hardware operating status

[0185] Output: Device connection status report, saved crop data

[0186] Operation: The server checks the device connection and accesses data from the agricultural database

[0187] Step 2:

[0188] The autonomous agricultural vehicle patrols farmland, taking images of crops with a camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors, and transmits the acquired data to a server in real time.

[0189] Input: Farmland environmental conditions, camera and sensor information

[0190] Output: Image data captured, environmental data measured

[0191] Operation: Taking images and measuring data using an autonomous agricultural vehicle, and sending the data to a server

[0192] Step 3:

[0193] The server analyzes the received image data and sensor data using a generative artificial intelligence model. Image analysis determines the growth status of the crops and the presence of diseases and pests. Environmental assessment uses the sensor data to evaluate whether the growing environment is appropriate and determine whether any corrections are necessary.

[0194] Input: Image data, sensor data

[0195] Output: Analysis result report, environmental assessment report

[0196] Operation: Data analysis by server, evaluation of growth status and environment

[0197] Step 4:

[0198] Based on the analysis results, the server instructs the autonomous agricultural vehicle on specific operations to provide an optimal growing environment, such as watering, fertilizing, and adjusting lights as needed.

[0199] Input: Analysis results

[0200] Output: Operation instructions for the autonomous agricultural vehicle

[0201] Operation: Sending instructions from the server to the autonomous agricultural vehicle and executing specific operations

[0202] Step 5:

[0203] Based on the analysis results, the server suggests efficient operation methods to the user. The suggestions are sent to the user's device, and the user then performs the operation. For example, specific instructions such as "crops in a specific area require additional watering" are provided.

[0204] Input: Analysis results, operation suggestions

[0205] Output: Notification of suggested action to user

[0206] Action: Notification from the server to the user device, execution of user operation

[0207] Step 6:

[0208] After the user performs the actual work based on the suggestions, they provide feedback to the system regarding the results and improvements. This feedback is used to improve the accuracy of future analyses and suggestions.

[0209] Input: Results of user actions and improvements

[0210] Output: Feedback data sent to the server

[0211] How it works: User sends feedback, server receives data and improves analysis accuracy

[0212] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0213] The system of the present invention is designed to achieve efficient and sustainable agricultural management. This system is composed of cameras, sensors, generative AI models, servers, terminals (robots), emotion engines, and users. The combination of these components is explained in detail below.

[0214] 1. Initial Setup

[0215] server

[0216] During the initial setup, the server checks the connections of all devices and tests that the cameras, sensors, robots, and emotion engine are working properly. It also provides an interface for users to input information about the crops they are growing, and retrieves and stores the input data from the agricultural database.

[0217] 2. Data Collection

[0218] Terminal (robot)

[0219] The terminal (robot) patrols the farmland, taking pictures of the growing crops with a camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors. The acquired data is sent to a server in real time. In addition, an emotion engine analyzes the user's voice and facial expressions to obtain emotional data.

[0220] 3. Data Analysis

[0221] server

[0222] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. It then analyzes the image data using a generative artificial intelligence model. Specifically, it evaluates the color, shape, and size of the leaves to determine the rate of growth. It also analyzes environmental sensor data to determine whether humidity, temperature, and light intensity are within appropriate ranges. Based on the analysis results, it generates suggestions for the optimal growing environment. It also evaluates the user's stress level based on the user's emotional data obtained by the emotion engine.

[0223] 4. Providing the optimal environment

[0224] Terminal (robot)

[0225] The terminal (robot) receives instructions from the server and performs specific operations to optimize the growing environment for the crops. For example, it waters, fertilizes, adjusts lighting, etc. as needed, ensuring that the crops are grown under optimal conditions.

[0226] 5. Operational suggestions

[0227] server

[0228] The server then suggests efficient operation methods to the user based on the analysis results and the user's emotional data. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. For example, along with specific instructions such as "crops in a specific area require additional watering," the server may provide suggestions such as "your current stress level is high, so we recommend you take a moderate break."

[0229] 6. Gathering Feedback

[0230] User

[0231] After the user performs the actual work based on the suggestions, they provide feedback to the system about the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. At the same time, the emotion engine continuously monitors changes in the user's emotions, and this data is also sent to the server.

[0232] Through these steps, the Auto Farm Navigation system not only streamlines crop cultivation management and supports sustainable agricultural operations, but also reduces the mental burden on users. For example, when users harvest, the system provides appropriate guidance on harvesting areas and adjusts work times according to the user's stress level, making farming more efficient and less stressful.

[0233] The processing flow will be explained below.

[0234] Step 1:

[0235] server

[0236] When the system starts up, the server checks the connection of all devices. It tests the normal operation of the cameras, sensors, robots, and emotion engine, and notifies the user if any abnormalities are detected. Next, the user inputs information about the crops they are growing through the interface, which retrieves and stores related data from the agricultural database.

[0237] Step 2:

[0238] Terminal (robot)

[0239] The terminal (robot) patrols the farmland, taking pictures of the growth of each crop with a camera. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. This data is sent to a server in real time.

[0240] Step 3:

[0241] Terminal (emotion engine)

[0242] The emotion engine analyzes the user's voice and facial expressions to obtain emotional data, which is then immediately sent to the server.

[0243] Step 4:

[0244] server

[0245] The server receives image data and sensor data sent from the terminal (robot), as well as emotion data sent from the emotion engine, and stores them in a database. It then uses a generative AI model to analyze the image data. Specifically, it evaluates the color, shape, and size of the leaves to determine the rate of growth. It also analyzes environmental sensor data to determine whether humidity, temperature, and light intensity are within appropriate ranges. It uses the emotion data to evaluate the user's stress level, and based on all this data, it generates suggestions for the optimal growing environment and operation methods.

[0246] Step 5:

[0247] server

[0248] Based on the generated suggestions, the server generates specific instructions for adjusting the environment. For example, it creates specific instructions such as "Add 500 ml of water to area A" or "Apply fertilizer to area B" and sends them to the terminal (robot). It also generates suggestions based on emotion data, such as "Your current stress level is high, so we recommend you take a moderate break."

[0249] Step 6:

[0250] Terminal (robot)

[0251] The terminal (robot) adjusts the environment based on instructions from the server. For example, it may water or fertilize the appropriate areas. It may also adjust the lighting. Once the adjustment is complete, it reports the results to the server.

[0252] Step 7:

[0253] server

[0254] The server receives the results of the environmental adjustments from the robot and checks whether the suggestions were implemented properly. Based on the crop growth status, the server generates suggestions for further restoration work for the user. Specifically, it may suggest that "harvesting time is approaching in a specific zone." It also suggests ways to reduce stress levels based on emotion data.

[0255] Step 8:

[0256] server

[0257] The suggestions are sent to the user's device, allowing them to understand the necessary operations and timing for breaks. Specific instructions include "crops in a specific area need additional watering" and "your current stress level is high, so please take a moderate break."

[0258] Step 9:

[0259] User

[0260] The user follows the suggestions from the server and performs actual operations, such as harvesting or pest control. Afterwards, the results of the work and any improvements needed on-site are fed back to the system. The emotion engine continues to monitor the user's emotion data and also sends that data to the server.

[0261] Through these processing steps, the Auto Farm Navigation system not only streamlines crop cultivation management and supports sustainable agricultural operations, but also reduces the user's mental burden. For example, when harvesting, the system provides appropriate judgment of the harvesting area and adjusts work time according to the user's stress level, making agricultural work more efficient and less stressful.

[0262] Example 2

[0263] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0264] In modern agriculture, while there is a demand for improving production efficiency and ensuring the quality of agricultural products, there is also a need to reduce the burden of farm work and achieve sustainable operations. Conventional systems have the technology to collect and analyze environmental and growth data, but they lack mechanisms for improving work efficiency and reducing the burden by taking user emotional data into account. This can lead to the accumulation of fatigue and stress, which can lead to reduced productivity. Furthermore, proposals for appropriate growing environments and work methods are limited, resulting in a lack of flexible and comprehensive support for agricultural operations.

[0265] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0266] In this invention, the server includes a means for capturing images of crop growth with a camera, a means for measuring environmental conditions with a sensor, a means for transmitting data from the camera and the sensor to the server, a means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal growing environment, a means for adjusting the growing environment based on the analysis results, a means for informing the user of efficient operation methods, a means for receiving feedback, a means for acquiring and analyzing the user's emotional data with an emotion analysis engine, and a means for evaluating the user's stress level and proposing appropriate working hours based on the user's emotional data. This not only improves the efficiency of agricultural work, but also reduces user stress and provides an appropriate working environment. Furthermore, by comprehensively analyzing the acquired data, maintaining an optimal growing environment, and proposing agricultural work methods, sustainable agricultural operations can be realized.

[0267] A "camera" is a device used to photograph the growth of crops.

[0268] A "sensor" is a device for measuring environmental conditions (humidity, temperature, light intensity, etc.).

[0269] "Server" is a computer system that receives, stores, and analyzes data sent from devices, and generates and provides various proposals.

[0270] A "generative artificial intelligence model" is an AI algorithm that analyzes the transmitted data and generates suggestions for the optimal development environment.

[0271] "Adjusting the growing environment" refers to the operation of adjusting the environment necessary for crop growth based on the analysis results.

[0272] "Efficient operation methods" refer to suggestions and instructions that allow users to manage crops and adjust the environment more effectively.

[0273] "Feedback" refers to information returned to the system regarding the results of the work performed by the user and areas for improvement.

[0274] An "emotion analysis engine" is an algorithm that analyzes a user's voice and facial expressions to obtain and evaluate emotional data.

[0275] "Emotion data" is data that indicates the user's emotional state and stress level.

[0276] "Stress level" is an index that represents the degree of stress that the user is feeling.

[0277] The system of the present invention is designed to achieve efficient and sustainable agricultural management. This system is composed of a combination of multiple hardware and software components, including cameras, sensors, generative AI models, servers, terminals (robots), an emotion analysis engine, and users.

[0278] 1. Initial Setup

[0279] server

[0280] During initial setup, the server checks the connections of all devices and tests that the cameras, sensors, terminals (robots), and sentiment analysis engine are working properly. The server also provides an interface for users to input information about the crops they are growing, and retrieves and stores the input crop data from the agricultural database. For example, if a user is growing tomatoes, the server retrieves the optimal growing conditions for tomatoes from the agricultural database and reflects them in the system.

[0281] 2. Data Collection

[0282] Terminal (robot)

[0283] The terminal (robot) patrols the farmland, using a camera to take pictures of the growing state of the crops. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. This data is sent to a server in real time. The terminal (robot) also uses an emotion analysis engine to analyze the user's voice and facial expressions to obtain emotional data about the user. For example, while the user is harvesting, the robot measures the soil humidity and temperature and sends the data to the server.

[0284] 3. Data Analysis

[0285] server

[0286] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. It then uses a generative AI model to analyze the image data. Specifically, it evaluates the color, shape, and size of the leaves and determines the rate of growth. It also analyzes environmental sensor data to determine whether humidity, temperature, and light intensity are within appropriate ranges. It also evaluates the user's stress level based on the user's emotional data obtained by an emotion analysis engine.

[0287] Proposal generation

[0288] Based on the analysis results, the server generates recommendations for the optimal growing environment. For example, it generates specific instructions such as, "The current soil humidity is low, so additional watering is required." It also makes suggestions such as, "We recommend taking moderate breaks" if the user's stress level is high.

[0289] 4. Providing the optimal environment

[0290] Terminal (robot)

[0291] The terminal (robot) receives instructions from the server and performs specific operations to optimize the growing environment for the crops. For example, it performs watering, fertilization, and light adjustments. This ensures that the crops are grown under optimal conditions. If the robot detects a nutrient deficiency in the tomatoes, it can automatically supply the appropriate amount of fertilizer.

[0292] 5. Operational suggestions

[0293] server

[0294] The server then suggests efficient ways to operate the system based on the analysis results and the user's emotional data. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. For example, the server might suggest, "Your current stress level is high, so we recommend you take a moderate break."

[0295] 6. Gathering Feedback

[0296] User

[0297] The user performs the work based on the suggestions and provides feedback to the system with the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. The sentiment analysis engine also continuously monitors changes in the user's emotions, and this data is also sent to the server. If a user provides feedback such as "The additional fertilizer has helped the tomatoes grow better," this will be reflected in future fertilizer management.

[0298] Specific examples of prompts to input to generative AI models

[0299] 1. "Evaluate the current growth status of your tomato plants based on the color and shape of their leaves."

[0300] 2. "Based on recent sensor data, determine if soil moisture and temperature are adequate."

[0301] 3. "Please analyze the user's current stress level from their voice data."

[0302] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0303] Step 1: Performing initial setup

[0304] Server: Check device connection

[0305] Input: Connection status of the onboard camera, sensors, terminal (robot), and emotion analysis engine

[0306] Processing: The server checks whether each device is properly connected and tests whether it is working properly. For example, it checks whether a camera is connected to the network and can transmit images in real time. It also checks the operation status of sensors and robots.

[0307] Output: Device connection status and operation check results

[0308] Specific operations: Test whether the camera can capture images properly, whether the sensor can measure data accurately, and whether the robot can operate correctly. If there is an abnormality, the user will be notified.

[0309] Server: Provides the user interface

[0310] Input: Information about the crops the user grows

[0311] Processing: The server provides a web interface for inputting information about the crops to be grown. When a user inputs the tomato variety and growing conditions, the server retrieves the relevant data from an agricultural database and stores them.

[0312] Output: Crop information and growing conditions data

[0313] Specific operation: The user inputs tomato cultivation information through the interface, and the server stores it in the database.

[0314] Step 2: Data collection

[0315] Terminal (robot): Photographing crops

[0316] Input: Crop growth status

[0317] Processing: The terminal (robot) periodically patrols the farmland and uses a camera to take pictures of the growing conditions of the crops.

[0318] Output: Captured image data

[0319] What it does: The robot patrols farmland and uses cameras to collect detailed images, such as capturing leaf color, shape, and size.

[0320] Terminal (robot): Measurement of environmental data

[0321] Input: soil moisture, temperature, light intensity

[0322] Processing: The terminal (robot) uses sensors during its patrol to measure soil humidity, temperature, and light intensity.

[0323] Output: Measured environmental data

[0324] Specific operation: The robot uses a humidity sensor to measure the moisture content of the soil and a temperature sensor to measure the environmental temperature.

[0325] Terminal (robot): Acquire emotion data

[0326] Input: User's voice and facial expressions

[0327] Processing: The terminal (robot) uses an emotion analysis engine to analyze the user's voice and facial expressions and obtain emotional data.

[0328] Output: Obtained emotion data

[0329] Specific operation: While the user is working, the robot collects and analyzes voice and facial expression data.

[0330] Terminal (robot): Sending data

[0331] Input: photographed image data, measured environmental data, acquired emotion data

[0332] Processing: The terminal (robot) sends these data to the server in real time.

[0333] Output: Data sent to server completed

[0334] Specific operation: The terminal (robot) uses wireless communication to send all collected data to the server.

[0335] Step 3: Data analysis

[0336] Server: Receives and stores data

[0337] Input: Transmitted image data, environmental data, emotional data

[0338] Processing: The server receives the data sent from the terminal (robot) and stores it in a database.

[0339] Output: Data stored in the database

[0340] Specific operation: The server stores all received data in a database in the appropriate format.

[0341] Server: Image data analysis

[0342] Input: Saved image data

[0343] Processing: The server uses a generative artificial intelligence model to analyze the image data, assessing leaf color, shape, and size to determine growth rate.

[0344] Output: Growth assessment results based on image data

[0345] Specific operation: Image analysis AI analyzes the color and shape of tomato leaves and measures their growth status.

[0346] Server: Sensor data analysis

[0347] Input: Stored environmental data

[0348] Processing: The server analyzes the environmental sensor data sent and evaluates whether the humidity, temperature, and light level are within appropriate ranges.

[0349] Output: Environmental condition assessment results

[0350] Specific operation: The data analysis system analyzes humidity, temperature, and light intensity data to determine whether the growing conditions are appropriate.

[0351] Server: Proposal Generation

[0352] Input: Image data growth assessment results, environmental condition assessment results

[0353] Processing: The server generates a proposal for the optimal growing environment based on the results of analyzing image data and sensor data.

[0354] Output: Proposed development environment

[0355] Specific action: The server generates specific instructions such as "Current soil moisture is low and additional watering is required."

[0356] Server: Emotion data analysis

[0357] Input: Acquired emotion data

[0358] Processing: The server evaluates the user's stress level based on data from the emotion analysis engine.

[0359] Output: User's stress level assessment result

[0360] Specific operation: Emotion analysis AI analyzes the user's stress level and, if necessary, makes suggestions such as "We recommend you take a moderate break."

[0361] Step 4: Providing an optimal environment

[0362] Terminal (robot): Receives operation instructions

[0363] Input: Instructions from the server

[0364] Processing: The terminal (robot) receives the operation instructions sent from the server.

[0365] Output: Actions based on received instructions

[0366] Specific action: The robot receives the instruction to "water the tomato trellis."

[0367] Terminal (robot): Performing environmental adjustments

[0368] Input: Specific operation instructions from the server

[0369] Processing: The terminal (robot) adjusts the environment according to the received instructions.

[0370] Output: Applied environmental adjustments

[0371] Specific actions: The robot waters and fertilizes the plants, as well as adjusts the lights and controls the temperature.

[0372] Step 5: Suggested operation

[0373] Server: Proposal Notification

[0374] Input: Analysis results, emotion data

[0375] Processing: The server proposes efficient operation methods to the user based on the analysis results and the user's emotional data.

[0376] Output: User is notified of the suggestion

[0377] Specific operation: The server notifies the user via the terminal with specific suggestions such as "Tomatoes in a specific area need additional watering."

[0378] User: Operate based on instructions

[0379] Input: Operation instructions from the server

[0380] Processing: The user follows instructions from the server to manage crops and adjust the environment.

[0381] Output: The result of the operation performed

[0382] Specific operation: The user receives instructions from the server and manually waters the plants, adds fertilizer, adjusts the lights, etc.

[0383] Step 6: Gather feedback

[0384] User: Feedback on work results

[0385] Input: Result of the operation performed

[0386] Processing: The user performs actual work based on the suggestions and provides feedback to the system on the results and improvements.

[0387] Output: Feedback data to the system

[0388] Specific action: The user comments, "The tomatoes are growing better thanks to the additional fertilizer," and enters this into the system.

[0389] Server: Update emotion data

[0390] Input: Emotional data after the user's task

[0391] Processing: The sentiment analysis engine continuously monitors the user's emotional changes and sends the data to the server.

[0392] Output: Updated emotion data

[0393] Specific operation: Emotion analysis AI analyzes the user's emotional data and evaluates changes in stress levels after work.

[0394] (Application example 2)

[0395] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0396] Conventional agricultural management systems primarily monitor crop growth and environmental conditions, but do not address efficient operations in industrial manufacturing sites. Furthermore, there was a lack of a system that simultaneously optimized the quality control of the manufacturing process and the health of workers, creating a challenge to balance the efficiency of factory operations with worker health management.

[0397] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0398] In this invention, the server includes means for photographing crop growth with a camera, means for measuring environmental conditions with a sensor, means for transmitting data from the camera and the sensor to the server, means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal cultivation environment, means for adjusting the cultivation environment based on the analysis results, means for notifying the user of efficient operation methods, means for receiving feedback, and means for monitoring the production process and the health of workers and providing optimal operations. This makes it possible to achieve both quality control of the production process and health management of workers.

[0399] A "camera" is a device that photographs an object and acquires its image data.

[0400] "Crop" refers to cultivated plants, especially those grown in agriculture.

[0401] "Growth" refers to the process by which crops grow and develop over time.

[0402] A "sensor" is a device that detects a physical or environmental condition and outputs it as an electrical signal.

[0403] "Environmental conditions" refers to factors that affect crops and the working environment, such as temperature, humidity, and light intensity.

[0404] "Server" refers to a central computer system for receiving data, analyzing data, and sending instructions.

[0405] A "generative artificial intelligence model" is a machine learning model that analyzes data based on specific algorithms and generates new suggestions and predictions.

[0406] "Analysis" refers to the process of processing acquired data and images to extract specific information and patterns.

[0407] "Growing environment" refers to the environmental conditions that promote optimal growth and production of agricultural crops and manufacturing processes.

[0408] "Adjustment" refers to the act of changing the conditions of a machine or environment to an optimal state in order to achieve a specific purpose.

[0409] "How to use" refers to documentation or interfaces that provide procedures or techniques for users to use a system to perform specific tasks.

[0410] "Users" refer to people who interact with the system and act on its instructions and suggestions.

[0411] "Notification" refers to the act or function of informing a user of specific information.

[0412] "Feedback" refers to results and opinions collected from users, and is information that is used to improve the performance and functionality of the system.

[0413] "Manufacturing process" refers to the series of operations and steps that create a product using raw materials.

[0414] "Health status" refers to the state of a worker's physical and mental health, including an assessment of the worker's condition in the working environment.

[0415] "Optimization" refers to adjusting a system or environment to its best state in order to achieve a specific purpose.

[0416] The system of the present invention is designed to optimize the manufacturing process and worker health in a factory, supporting efficient operations. This system is composed of cameras, sensors, a generative AI model, a server, a terminal (robot), an emotion engine, and a user. The combination of these components is described in detail below.

[0417] 1. Initial Setup

[0418] server

[0419] The server first checks whether all devices (cameras, sensors, robots, and emotion engines) are operating normally. The camera used here is a Logitech C920 HD Pro Webcam, and the temperature and humidity sensor is a DHT11. Next, it inputs initial data about the factory's manufacturing process and stores it in a database. The server then uses a web framework such as Flask or Django to build an API and establish communication with the devices.

[0420] 2. Data Collection

[0421] Terminal (robot)

[0422] The terminal (robot) patrols the factory, checking product quality with a camera and measuring environmental conditions (temperature, humidity, etc.) with sensors. It is equipped with an emotion engine and acquires emotional data from the facial expressions and voices of workers. This data is sent to the server in real time.

[0423] 3. Data Analysis

[0424] server

[0425] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. A generative artificial intelligence model using TENSORFLOW (registered trademark) analyzes the image data and evaluates the quality of the product. It also analyzes environmental sensor data to determine whether temperature and humidity are within appropriate ranges. It also simultaneously evaluates the user's emotional data obtained by the emotion engine and measures the user's stress level.

[0426] 4. Providing the optimal environment

[0427] Terminal (robot)

[0428] Based on the analysis results, the server sends specific instructions to the terminal (robot), such as adjusting the lighting, adjusting the temperature and humidity, or suggesting that the worker take a break.

[0429] 5. Operational suggestions

[0430] server

[0431] The server then informs the user of the optimal operation method based on the analysis results, for example, providing specific instructions such as "Additional inspection of products in a specific area is required" along with suggestions such as "Your current stress level is high, so we recommend you take a moderate break."

[0432] 6. Gathering Feedback

[0433] User

[0434] The user performs actual tasks based on the suggestions and provides feedback on the results and improvements to the system. This feedback is used to improve the quality of future analyses and suggestions. At the same time, the emotion engine continuously monitors changes in the user's emotions, and this data is also sent to the server.

[0435] For example, a robot monitors the production line with a camera and measures environmental conditions in real time using a DHT11 sensor. It also captures the user's (worker's) face with a camera and analyzes the facial expression data using an emotion engine. The results are sent to a server, where a generative AI model predicts and suggests optimal operations and improvements. Specific instructions, such as "make the lighting a little brighter," "add more moisture to the product," or "encourage workers to take breaks," can be obtained.

[0436] Example prompt sentence:

[0437] "Detect scratches and defects on products from images"

[0438] "Please evaluate how these environmental conditions (humidity, temperature) affect the quality of the product."

[0439] "Assess workers' stress levels and suggest break needs."

[0440] This system simultaneously improves the efficiency of the manufacturing process and manages the health of workers.

[0441] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0442] Step 1:

[0443] The server checks whether all devices (cameras, sensors, robots, emotion engines) are working properly.

[0444] Input: Device connection status

[0445] Output: Device connection check result

[0446] How it works: The server pings each device and checks if there is a response. It uses Flask or Django to check the device connection status via an API.

[0447] Step 2:

[0448] The server inputs initial data about the factory's manufacturing process and stores it in a database.

[0449] Input: Initial data about the manufacturing process

[0450] Output: Initial data stored in the database

[0451] Operation: The server receives manufacturing process information entered through a user interface and stores it in a database, including information such as product specifications and process sequences.

[0452] Step 3:

[0453] The terminal (robot) patrols the factory, checking product quality with cameras and measuring environmental conditions with sensors.

[0454] Input: Camera footage, environmental data measured by sensors (temperature, humidity, etc.)

[0455] Output: Photographed image data, measured environmental data

[0456] Operation: The terminal (robot) moves autonomously within the factory, taking images of products using a camera and measuring environmental conditions such as temperature and humidity using sensors.

[0457] Step 4:

[0458] The terminal (robot) uses an emotion engine to obtain emotional data from the worker's facial expressions and voice.

[0459] Input: Worker's face video and voice

[0460] Output: Parsed emotion data

[0461] How it works: The device captures the worker's face with a camera and collects their voice with a microphone. It then uses an emotion engine to analyze this data and generate emotion data.

[0462] Step 5:

[0463] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database.

[0464] Input: Image data, sensor data

[0465] Output: Raw data stored in a database

[0466] Operation: The server receives data sent from the terminal using a communication protocol and stores it in a database.

[0467] Step 6:

[0468] The server analyzes image data using a generative artificial intelligence model to evaluate product quality, as well as environmental sensor data and emotion data.

[0469] Input: Image data, environmental sensor data, emotion data

[0470] Output: Analysis results (quality evaluation, environment evaluation, emotion evaluation)

[0471] How it works: The server uses TensorFlow to analyze image data and input prompts (e.g., "Detect scratches and defects on the product from the image"), as well as environmental data such as temperature and humidity (e.g., "Evaluate how these environmental conditions affect the product's quality") and emotional data (e.g., "Evaluate the worker's stress level and suggest the need for a break").

[0472] Step 7:

[0473] Based on the analysis results, the server sends specific operation instructions to the terminal (robot).

[0474] Input: Analysis results

[0475] Output: Instructions to the terminal

[0476] Operation: Based on the analysis results, the server generates specific operational instructions, such as "brighten the lights" or "adjust the temperature and humidity," and sends them to the terminal.

[0477] Step 8:

[0478] The terminal (robot) performs specific operations on-site based on operation instructions from the server.

[0479] Input: Operation instructions from the server

[0480] Output: The result of the operation performed

[0481] Operation: The terminal follows instructions from the server and performs operations such as "adjusting lighting," "increasing humidity," and "notifying workers to take a break."

[0482] Step 9:

[0483] Based on the analysis results, the server notifies the user of efficient operation methods.

[0484] Input: Analysis results

[0485] Output: A message to inform the user

[0486] How it works: The server analyzes the results and sends specific instructions to the user's device, such as "Products in a specific area require additional testing," and suggestions, such as "We recommend taking appropriate breaks."

[0487] Step 10:

[0488] The user performs actual work based on the suggestions and provides feedback on the results and improvements to the system.

[0489] Input: User feedback data

[0490] Output: accumulation and analysis of feedback data

[0491] How it works: The user follows the suggestions and reports the results to the system. This allows the system to improve the quality of its analysis and suggestions in the future. The emotion engine also continuously monitors the user's emotional changes and sends the data to the server.

[0492] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0493] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0494] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0495] [Second embodiment]

[0496] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0497] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0498] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0499] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0500] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0501] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0502] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0503] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0504] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0505] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0506] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0507] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0508] The system of the present invention is designed to realize efficient and sustainable agricultural management. This system consists of cameras, sensors, generative AI models, servers, terminals, and users.

[0509] 1. Initial Setup

[0510] server

[0511] During initial setup, the server checks the connections of all devices and tests that the cameras and sensors are working properly. It then provides an interface for users to input information about the crops they are growing, and retrieves and stores the input data from the agricultural database. This ensures that the system is ready to provide an optimal growing environment right from the start.

[0512] 2. Data Collection

[0513] Terminal (robot)

[0514] The device patrols the farmland, taking images of the crops with its camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors. This data is sent to a server in real time. Accurate data collected at this stage ensures the accuracy of subsequent analysis.

[0515] 3. Data Analysis

[0516] server

[0517] The server analyzes the received image data and sensor data using a generative artificial intelligence model. Image analysis determines the growth status of crops and checks for disease and pest damage based on leaf color, shape, and size. Environmental assessment evaluates whether the current growing environment is appropriate based on the acquired sensor data and determines whether any deficient environmental conditions need to be corrected. The analysis results are generated as a proposal, and the next action is decided.

[0518] 4. Providing the optimal environment

[0519] Terminal (robot)

[0520] The terminal receives instructions from the server and performs specific operations to optimize the growing environment for the crops, such as watering, fertilizing, and adjusting lighting as needed, ensuring that the crops are grown under optimal conditions.

[0521] 5. Operational suggestions

[0522] server

[0523] Based on the analysis results, the server suggests efficient operation methods to the user. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. For example, specific instructions such as "crops in a specific area require additional watering" are provided.

[0524] 6. Gathering Feedback

[0525] User

[0526] After carrying out actual work based on the suggestions, users provide feedback to the system on the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. This allows the system to continually evolve, enabling more effective agricultural management.

[0527] The above is a specific embodiment of the "Auto Farm Navigation" system. This system not only enables efficient and effective crop cultivation management, but also greatly contributes to the realization of sustainable agriculture. By understanding the specific operating procedures and system flow, users can make the most of this system.

[0528] The processing flow will be explained below.

[0529] Step 1:

[0530] server

[0531] When the system starts up, the server checks the connections of all devices, including cameras, sensors, and robots. It tests each device for normal operation and notifies users if any abnormalities are detected. Users enter information about the crops they are growing through the interface, and based on that, related data is retrieved and saved from the agricultural database.

[0532] Step 2:

[0533] Terminal (robot)

[0534] The terminal (robot) uses a camera to capture images of the growing conditions of the crops at each location. It also uses sensors to measure the soil humidity, temperature, and light intensity. The acquired data is sent to a server in real time.

[0535] Step 3:

[0536] server

[0537] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. It then analyzes the image data using a generative artificial intelligence model. Specifically, it evaluates the color, shape, and size of the leaves and determines the speed of the adult. It also analyzes environmental sensor data to evaluate whether humidity, temperature, and light intensity are within appropriate ranges. Based on the analysis results, it generates a proposal for the optimal growing environment.

[0538] Step 4:

[0539] server

[0540] Based on the generated suggestions, the server generates specific instructions for adjusting the environment, such as "add 500 ml of water to area A" or "apply fertilizer to area B." These instructions are sent to the terminal (robot).

[0541] Step 5:

[0542] Terminal (robot)

[0543] The terminal (robot) adjusts the environment based on instructions from the server. For example, it may water or fertilize the appropriate areas. It may also adjust the lighting. Once the adjustment is complete, it reports the results to the server.

[0544] Step 6:

[0545] server

[0546] The server receives the results of the environmental adjustments from the robot and checks whether the suggestions were implemented properly. Furthermore, it generates suggestions for return work to the user based on the crop growth status. Specifically, the suggestions include, "The harvest time is approaching in a specific zone," or "Signs of pests and diseases are visible and need to be addressed."

[0547] Step 7:

[0548] server

[0549] The system notifies the user of suggestions for efficient operation methods, and provides specific instructions on tasks and precautions the user should take based on the suggestions. For example, it may notify the user that "crops in a specific area require additional watering."

[0550] Step 8:

[0551] User

[0552] The user follows the suggestions from the server and performs actual operations, such as harvesting or pest control. The results of the work and any improvements needed on-site are then fed back to the system, providing reference data for future analyses and to improve the quality of suggestions.

[0553] Through these processing steps, the system of the present invention can streamline crop management and support sustainable agricultural operations.

[0554] Example 1

[0555] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0556] To achieve efficient and sustainable agricultural operations, it is necessary to monitor crop growth and environmental conditions in real time and provide an appropriate growing environment. However, conventional technologies mainly rely on manual monitoring and environmental adjustment, which is labor-intensive and time-consuming, making it difficult to constantly maintain an optimal environment. It is also difficult to respond quickly to changes in environmental conditions, which can affect crop quality and yield.

[0557] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0558] In this invention, the server includes means for photographing the growth of crops with a camera, means for measuring environmental conditions with a sensor, means for transmitting data from the camera and the sensor to an information processing device, means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal cultivation environment, means for adjusting the cultivation environment based on the generated proposal, means for notifying a user of an efficient operation method based on the analysis results, and means for receiving feedback from the user. This makes it possible to monitor the growth of crops in real time, quickly respond to changes in environmental conditions, and provide an optimal cultivation environment.

[0559] A "camera" is an image capturing device for capturing images of crop growth.

[0560] A "sensor" is a device used to measure environmental conditions such as soil humidity, temperature, and light intensity.

[0561] An "information processing device" is a computer system for receiving and processing data transmitted from cameras and sensors.

[0562] A "generative artificial intelligence model" is a machine learning model used to analyze collected data and evaluate the growth status of crops and environmental conditions.

[0563] "Analysis" is the process of using a generative artificial intelligence model to evaluate the content of the received data and determine the condition of the crops and any necessary environmental adjustments.

[0564] "Growing environment" refers to the physical conditions and environmental factors (humidity, temperature, light intensity, etc.) required for crop growth.

[0565] "Feedback" refers to information that is input by the user regarding the results of work performed and improvements to the system's proposals, and is used to improve the system's analysis and proposals from the next time onwards.

[0566] "User" refers to a person or organization that uses this system to grow and manage agricultural crops.

[0567] The system of the present invention is an automated agricultural management system for achieving efficient and sustainable agricultural operations. This system is composed of a camera, sensors, a generative AI model, an information processing device, a terminal, and a user.

[0568] Initial Setup

[0569] server

[0570] During initial setup, the server checks the connectivity of all devices and tests that the cameras and sensors are working properly. Specifically, it takes test images with the cameras and collects initial data (such as temperature and humidity) from the sensors. Next, it provides an interface for users to input information about the crops they are growing. Users enter data such as the type of crop, its growth stage, and specific growing conditions. This information is then retrieved from an agricultural database and stored on the server.

[0571] Data collection

[0572] Terminal (robot)

[0573] The device patrols the farmland, taking images of the crops with its camera. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. This data is sent to a server in real time. For example, the device could take images of the entire farmland at a specific time each day to collect environmental data.

[0574] Data analysis

[0575] server

[0576] The server analyzes the received image data and sensor data using a generative AI model. Image analysis determines the growth status of the crops and checks for disease and pest damage based on leaf color, shape, and size. It also performs environmental assessments based on sensor data to determine whether the current growing environment is appropriate. The generative AI model uses deep learning to improve data accuracy. For example, it can produce analysis results such as, "The leaves of the crops have turned yellow, so there is a high possibility that they are lacking nitrogen fertilizer."

[0577] Providing the optimal environment

[0578] Terminal (robot)

[0579] The device receives instructions from the server and performs operations to optimize the growing environment for crops. For example, if the data measured by the sensor indicates a lack of water, the device will automatically water the crops. It will also apply fertilizer and adjust the lighting as needed. This ensures that crops are grown under optimal conditions.

[0580] Operation suggestions

[0581] server

[0582] Based on the analysis results, the server suggests efficient operation methods to the user. For example, it notifies the user's device of specific instructions such as "crops in a specific area require additional watering." This allows the user to follow the instructions and improve efficiency.

[0583] Collecting feedback

[0584] User

[0585] After carrying out the work based on the suggestions, the user can then provide feedback to the system about the results and areas for improvement. For example, the user can enter specific information such as, "After watering the plant, the leaves suddenly became healthier." This feedback will be used as important data to improve the quality of future analyses and suggestions.

[0586] (Example)

[0587] For example:

[0588] Example prompt:

[0589] Crops are not growing well in the growing environment. Please analyze the data collected by the sensor and suggest the best solution.

[0590] Example result:

[0591] The server uses sensor data to determine if the soil moisture is too low and instructs the device, "Additional watering is required this morning." The device receives instructions from the server and supplies the appropriate amount of water to the specified location. The user then observes changes in growth and provides feedback on the effects.

[0592] This system not only enables efficient and effective crop management, but also enables sustainable agricultural operations.

[0593] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0594] Step 1:

[0595] The server checks the connection of all devices (cameras, sensors, terminals) and tests whether they are working properly. As input, it receives initial data from the cameras and sensors (e.g., test images, temperature, humidity, etc.), and as output, it sends a signal to each device to confirm that the device is working properly.

[0596] Step 2:

[0597] The user inputs information about the crops they are growing using an interface provided by the server. The input details, such as the type of crop, its growth stage, and cultivation conditions, are sent to the server, which then retrieves and stores this information from an agricultural database.

[0598] Step 3:

[0599] The terminal (robot) patrols the farmland, taking images of the crops with a camera. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. It collects crop images and environmental data as input, and sends this data to a server in real time as output.

[0600] Step 4:

[0601] The server receives image data and sensor data sent from the device. The data received as input is sent to a generative AI model, which analyzes it. The output is an evaluation result regarding the growth status of crops and environmental conditions.

[0602] Step 5:

[0603] The server generates optimal growing environment recommendations based on the data analyzed by the generative AI model. It uses the analysis results as input and generates specific growing environment recommendations (e.g., watering amounts, fertilization needs, lighting adjustments, etc.) as output.

[0604] Step 6:

[0605] The server sends the generated cultivation environment proposal to the terminal. It receives the cultivation environment proposal as input and sends instructions as output to the terminal. The terminal performs operations to optimize the crop cultivation environment based on these instructions. Specifically, it automatically waters the crops, applies fertilizer, and adjusts the lights.

[0606] Step 7:

[0607] The server notifies the user of efficient operation methods based on the analysis results. It takes the analysis results and generated suggestions as input and sends notifications about efficient operation methods to the user's device as output. For example, it may instruct the user that "crops in a specific area require additional watering."

[0608] Step 8:

[0609] The user performs the actual work based on the suggestions from the server. The user receives instructions from the server as input and sends the results of the work as feedback to the server as output. For example, the user can input specific information such as "After watering, the leaves of the plant suddenly became healthier."

[0610] Step 9:

[0611] The server receives feedback from users and uses it to improve the quality of the next analysis and proposal. Feedback information is taken in as input, and output is used to improve the generative AI model and increase the accuracy of analysis. This allows the system to continuously evolve, enabling more effective agricultural operations.

[0612] (Application example 1)

[0613] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0614] Modern agriculture requires real-time monitoring of crop growth conditions and the provision of an appropriate growing environment. However, current manual management methods make it difficult to quickly respond to small changes in the growing environment. Furthermore, a lack of technology for efficient agricultural management makes sustainable agricultural operations difficult. Furthermore, managing large areas of farmland requires a great deal of manpower and effort, resulting in high costs.

[0615] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0616] In this invention, the server includes means for photographing crop growth with a camera, means for measuring environmental conditions with a sensor, means for transmitting data from the camera and the sensor to the server, means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal cultivation environment, means for adjusting the cultivation environment based on the analysis results, means for notifying the user of efficient operation methods, means for receiving feedback, means for being mounted on the autonomous agricultural vehicle, and means for transmitting instructions to the autonomous agricultural vehicle. This enables crop growth to be managed in an optimal environment in real time, thereby realizing efficient agricultural management and sustainable agriculture.

[0617] The "camera" is a device that photographs the growth of crops and collects visual data.

[0618] A "sensor" is a device for measuring environmental conditions (such as humidity or temperature).

[0619] A "server" is a central processing unit that receives and analyzes data sent from cameras and sensors.

[0620] A "generative artificial intelligence model" is a machine learning model that analyzes collected data and generates suggestions for growing crops.

[0621] "Optimal growing environment proposals" are proposals based on the analysis results that show the best environmental settings and operation methods to promote crop growth.

[0622] "Means for adjusting the growing environment" refers to devices or methods for actually changing the environmental conditions of agricultural crops (such as the amount of light, water, and temperature) based on the analysis results.

[0623] The "means for notifying the user of an efficient operation method" refers to a method or device for conveying specific operation instructions to the user based on the analysis results.

[0624] "Means for receiving feedback" refers to a method or device for sending back to the system the results of operations performed by the user in accordance with instructions and suggestions for improvement.

[0625] An "autonomous agricultural vehicle" is a vehicle that operates autonomously according to the growth status of crops and environmental conditions, and manages the growing environment.

[0626] "Means for sending instructions to an autonomous agricultural vehicle" refers to a method or device for transmitting specific operational instructions from a server to an autonomous agricultural vehicle.

[0627] The present invention relates to a system for highly automating the cultivation and environmental management of agricultural crops in an autonomous agricultural vehicle. The system for carrying out the present invention is configured as follows.

[0628] 1. Initial Setup

[0629] server

[0630] The server checks the connection of all devices during the initial setup phase, tests whether the cameras and sensors are working properly, provides an interface for users to input information about the crops they are growing, and retrieves and stores the input data from an agricultural database.

[0631] 2. Data Collection

[0632] Autonomous agricultural vehicles

[0633] The autonomous agricultural vehicle travels around the field, taking images of the crops with a camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors, and transmits this data to a server in real time.

[0634] 3. Data Analysis

[0635] server

[0636] The server analyzes the received image data and sensor data using a generative artificial intelligence model. Image analysis determines the growth status of crops and checks for disease and pest damage based on leaf color, shape, and size. Environmental assessment evaluates whether the current growing environment is appropriate based on the acquired sensor data and determines whether any deficient environmental conditions need to be corrected. The analysis results are generated as a proposal, and the next action is decided.

[0637] 4. Providing the optimal environment

[0638] Autonomous agricultural vehicles

[0639] The autonomous agricultural vehicle receives instructions from the server and performs specific operations to optimize the growing environment for crops, such as watering, fertilizing, and adjusting lights as needed, ensuring that crops are grown under optimal growing conditions.

[0640] 5. Operational suggestions

[0641] server

[0642] Based on the analysis results, the server suggests efficient operation methods to the user. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. Specific instructions include information such as "crops in a specific area require additional watering."

[0643] 6. Gathering Feedback

[0644] User

[0645] After carrying out actual work based on the suggestions, users provide feedback to the system on the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. This allows the system to continually evolve, enabling more effective agricultural management.

[0646] Hardware and Software Used

[0647] Hardware: Cameras, humidity sensors, temperature sensors, autonomous agricultural vehicles

[0648] Software: Python, OpenCV (image processing library), requests (HTTP communication library), generative artificial intelligence model using deep learning

[0649] Example prompt

[0650] In a real application, the next action is determined based on the data analysis results and instructions are sent to the vehicle. For example, if data is collected at a humidity of 45% and a temperature of 24°C, the prompt sentence would be as follows:

[0651] "Data Collection: Environmental data was collected at 45% humidity and 24°C. Based on the analysis, area A1 requires additional watering. Instructions: Add 2 liters of water to area A1."

[0652] This will enable optimal environmental management of crops in real time, resulting in efficient agricultural operations.

[0653] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0654] Step 1:

[0655] As an initial setup, the server checks the connection of all devices and checks that the cameras and sensors are working properly. It also obtains information about the crops being grown based on user input, accesses an agricultural database, and stores the necessary data.

[0656] Input: User input of crop information, hardware operating status

[0657] Output: Device connection status report, saved crop data

[0658] Operation: The server checks the device connection and accesses data from the agricultural database

[0659] Step 2:

[0660] The autonomous agricultural vehicle patrols farmland, taking images of crops with a camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors, and transmits the acquired data to a server in real time.

[0661] Input: Farmland environmental conditions, camera and sensor information

[0662] Output: Image data captured, environmental data measured

[0663] Operation: Taking images and measuring data using an autonomous agricultural vehicle, and sending the data to a server

[0664] Step 3:

[0665] The server analyzes the received image data and sensor data using a generative artificial intelligence model. Image analysis determines the growth status of the crops and the presence of diseases and pests. Environmental assessment uses the sensor data to evaluate whether the growing environment is appropriate and determine whether any corrections are necessary.

[0666] Input: Image data, sensor data

[0667] Output: Analysis result report, environmental assessment report

[0668] Operation: Data analysis by server, evaluation of growth status and environment

[0669] Step 4:

[0670] Based on the analysis results, the server instructs the autonomous agricultural vehicle on specific operations to provide an optimal growing environment, such as watering, fertilizing, and adjusting lights as needed.

[0671] Input: Analysis results

[0672] Output: Operation instructions for the autonomous agricultural vehicle

[0673] Operation: Sending instructions from the server to the autonomous agricultural vehicle and executing specific operations

[0674] Step 5:

[0675] Based on the analysis results, the server suggests efficient operation methods to the user. The suggestions are sent to the user's device, and the user then performs the operation. For example, specific instructions such as "crops in a specific area require additional watering" are provided.

[0676] Input: Analysis results, operation suggestions

[0677] Output: Notification of suggested action to user

[0678] Action: Notification from the server to the user device, execution of user operation

[0679] Step 6:

[0680] After the user performs the actual work based on the suggestions, they provide feedback to the system regarding the results and improvements. This feedback is used to improve the accuracy of future analyses and suggestions.

[0681] Input: Results of user actions and improvements

[0682] Output: Feedback data sent to the server

[0683] How it works: User sends feedback, server receives data and improves analysis accuracy

[0684] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0685] The system of the present invention is designed to achieve efficient and sustainable agricultural management. This system is composed of cameras, sensors, generative AI models, servers, terminals (robots), emotion engines, and users. The combination of these components is explained in detail below.

[0686] 1. Initial Setup

[0687] server

[0688] During the initial setup, the server checks the connections of all devices and tests that the cameras, sensors, robots, and emotion engine are working properly. It also provides an interface for users to input information about the crops they are growing, and retrieves and stores the input data from the agricultural database.

[0689] 2. Data Collection

[0690] Terminal (robot)

[0691] The terminal (robot) patrols the farmland, taking pictures of the growing crops with a camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors. The acquired data is sent to a server in real time. In addition, an emotion engine analyzes the user's voice and facial expressions to obtain emotional data.

[0692] 3. Data Analysis

[0693] server

[0694] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. It then analyzes the image data using a generative artificial intelligence model. Specifically, it evaluates the color, shape, and size of the leaves to determine the rate of growth. It also analyzes environmental sensor data to determine whether humidity, temperature, and light intensity are within appropriate ranges. Based on the analysis results, it generates suggestions for the optimal growing environment. It also evaluates the user's stress level based on the user's emotional data obtained by the emotion engine.

[0695] 4. Providing the optimal environment

[0696] Terminal (robot)

[0697] The terminal (robot) receives instructions from the server and performs specific operations to optimize the growing environment for the crops. For example, it waters, fertilizes, adjusts lighting, etc. as needed, ensuring that the crops are grown under optimal conditions.

[0698] 5. Operational suggestions

[0699] server

[0700] The server then suggests efficient operation methods to the user based on the analysis results and the user's emotional data. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. For example, along with specific instructions such as "crops in a specific area require additional watering," the server may provide suggestions such as "your current stress level is high, so we recommend you take a moderate break."

[0701] 6. Gathering Feedback

[0702] User

[0703] After the user performs the actual work based on the suggestions, they provide feedback to the system about the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. At the same time, the emotion engine continuously monitors changes in the user's emotions, and this data is also sent to the server.

[0704] Through these steps, the Auto Farm Navigation system not only streamlines crop cultivation management and supports sustainable agricultural operations, but also reduces the mental burden on users. For example, when users harvest, the system provides appropriate guidance on harvesting areas and adjusts work times according to the user's stress level, making farming more efficient and less stressful.

[0705] The processing flow will be explained below.

[0706] Step 1:

[0707] server

[0708] When the system starts up, the server checks the connection of all devices. It tests the normal operation of the cameras, sensors, robots, and emotion engine, and notifies the user if any abnormalities are detected. Next, the user inputs information about the crops they are growing through the interface, which retrieves and stores related data from the agricultural database.

[0709] Step 2:

[0710] Terminal (robot)

[0711] The terminal (robot) patrols the farmland, taking pictures of the growth of each crop with a camera. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. This data is sent to a server in real time.

[0712] Step 3:

[0713] Terminal (emotion engine)

[0714] The emotion engine analyzes the user's voice and facial expressions to obtain emotional data, which is then immediately sent to the server.

[0715] Step 4:

[0716] server

[0717] The server receives image data and sensor data sent from the terminal (robot), as well as emotion data sent from the emotion engine, and stores them in a database. It then uses a generative AI model to analyze the image data. Specifically, it evaluates the color, shape, and size of the leaves to determine the rate of growth. It also analyzes environmental sensor data to determine whether humidity, temperature, and light intensity are within appropriate ranges. It uses the emotion data to evaluate the user's stress level, and based on all this data, it generates suggestions for the optimal growing environment and operation methods.

[0718] Step 5:

[0719] server

[0720] Based on the generated suggestions, the server generates specific instructions for adjusting the environment. For example, it creates specific instructions such as "Add 500 ml of water to area A" or "Apply fertilizer to area B" and sends them to the terminal (robot). It also generates suggestions based on emotion data, such as "Your current stress level is high, so we recommend you take a moderate break."

[0721] Step 6:

[0722] Terminal (robot)

[0723] The terminal (robot) adjusts the environment based on instructions from the server. For example, it may water or fertilize the appropriate areas. It may also adjust the lighting. Once the adjustment is complete, it reports the results to the server.

[0724] Step 7:

[0725] server

[0726] The server receives the results of the environmental adjustments from the robot and checks whether the suggestions were implemented properly. Based on the crop growth status, the server generates suggestions for further restoration work for the user. Specifically, it may suggest that "harvesting time is approaching in a specific zone." It also suggests ways to reduce stress levels based on emotion data.

[0727] Step 8:

[0728] server

[0729] The suggestions are sent to the user's device, allowing them to understand the necessary operations and timing for breaks. Specific instructions include "crops in a specific area need additional watering" and "your current stress level is high, so please take a moderate break."

[0730] Step 9:

[0731] User

[0732] The user follows the suggestions from the server and performs actual operations, such as harvesting or pest control. Afterwards, the results of the work and any improvements needed on-site are fed back to the system. The emotion engine continues to monitor the user's emotion data and also sends that data to the server.

[0733] Through these processing steps, the Auto Farm Navigation system not only streamlines crop cultivation management and supports sustainable agricultural operations, but also reduces the user's mental burden. For example, when harvesting, the system provides appropriate judgment of the harvesting area and adjusts work time according to the user's stress level, making agricultural work more efficient and less stressful.

[0734] Example 2

[0735] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0736] In modern agriculture, while there is a demand for improving production efficiency and ensuring the quality of agricultural products, there is also a need to reduce the burden of farm work and achieve sustainable operations. Conventional systems have the technology to collect and analyze environmental and growth data, but they lack mechanisms for improving work efficiency and reducing the burden by taking user emotional data into account. This can lead to the accumulation of fatigue and stress, which can lead to reduced productivity. Furthermore, proposals for appropriate growing environments and work methods are limited, resulting in a lack of flexible and comprehensive support for agricultural operations.

[0737] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0738] In this invention, the server includes a means for capturing images of crop growth with a camera, a means for measuring environmental conditions with a sensor, a means for transmitting data from the camera and the sensor to the server, a means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal growing environment, a means for adjusting the growing environment based on the analysis results, a means for informing the user of efficient operation methods, a means for receiving feedback, a means for acquiring and analyzing the user's emotional data with an emotion analysis engine, and a means for evaluating the user's stress level and proposing appropriate working hours based on the user's emotional data. This not only improves the efficiency of agricultural work, but also reduces user stress and provides an appropriate working environment. Furthermore, by comprehensively analyzing the acquired data, maintaining an optimal growing environment, and proposing agricultural work methods, sustainable agricultural operations can be realized.

[0739] A "camera" is a device used to photograph the growth of crops.

[0740] A "sensor" is a device for measuring environmental conditions (humidity, temperature, light intensity, etc.).

[0741] "Server" is a computer system that receives, stores, and analyzes data sent from devices, and generates and provides various proposals.

[0742] A "generative artificial intelligence model" is an AI algorithm that analyzes the transmitted data and generates suggestions for the optimal development environment.

[0743] "Adjusting the growing environment" refers to the operation of adjusting the environment necessary for crop growth based on the analysis results.

[0744] "Efficient operation methods" refer to suggestions and instructions that allow users to manage crops and adjust the environment more effectively.

[0745] "Feedback" refers to information returned to the system regarding the results of the work performed by the user and areas for improvement.

[0746] An "emotion analysis engine" is an algorithm that analyzes a user's voice and facial expressions to obtain and evaluate emotional data.

[0747] "Emotion data" is data that indicates the user's emotional state and stress level.

[0748] "Stress level" is an index that represents the degree of stress that the user is feeling.

[0749] The system of the present invention is designed to achieve efficient and sustainable agricultural management. This system is composed of a combination of multiple hardware and software components, including cameras, sensors, generative AI models, servers, terminals (robots), an emotion analysis engine, and users.

[0750] 1. Initial Setup

[0751] server

[0752] During initial setup, the server checks the connections of all devices and tests that the cameras, sensors, terminals (robots), and sentiment analysis engine are working properly. The server also provides an interface for users to input information about the crops they are growing, and retrieves and stores the input crop data from the agricultural database. For example, if a user is growing tomatoes, the server retrieves the optimal growing conditions for tomatoes from the agricultural database and reflects them in the system.

[0753] 2. Data Collection

[0754] Terminal (robot)

[0755] The terminal (robot) patrols the farmland, using a camera to take pictures of the growing state of the crops. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. This data is sent to a server in real time. The terminal (robot) also uses an emotion analysis engine to analyze the user's voice and facial expressions to obtain emotional data about the user. For example, while the user is harvesting, the robot measures the soil humidity and temperature and sends the data to the server.

[0756] 3. Data Analysis

[0757] server

[0758] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. It then uses a generative AI model to analyze the image data. Specifically, it evaluates the color, shape, and size of the leaves and determines the rate of growth. It also analyzes environmental sensor data to determine whether humidity, temperature, and light intensity are within appropriate ranges. It also evaluates the user's stress level based on the user's emotional data obtained by an emotion analysis engine.

[0759] Proposal generation

[0760] Based on the analysis results, the server generates recommendations for the optimal growing environment. For example, it generates specific instructions such as, "The current soil humidity is low, so additional watering is required." It also makes suggestions such as, "We recommend taking moderate breaks" if the user's stress level is high.

[0761] 4. Providing the optimal environment

[0762] Terminal (robot)

[0763] The terminal (robot) receives instructions from the server and performs specific operations to optimize the growing environment for the crops. For example, it performs watering, fertilization, and light adjustments. This ensures that the crops are grown under optimal conditions. If the robot detects a nutrient deficiency in the tomatoes, it can automatically supply the appropriate amount of fertilizer.

[0764] 5. Operational suggestions

[0765] server

[0766] The server then suggests efficient ways to operate the system based on the analysis results and the user's emotional data. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. For example, the server might suggest, "Your current stress level is high, so we recommend you take a moderate break."

[0767] 6. Gathering Feedback

[0768] User

[0769] The user performs the work based on the suggestions and provides feedback to the system with the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. The sentiment analysis engine also continuously monitors changes in the user's emotions, and this data is also sent to the server. If a user provides feedback such as "The additional fertilizer has helped the tomatoes grow better," this will be reflected in future fertilizer management.

[0770] Specific examples of prompts to input to generative AI models

[0771] 1. "Evaluate the current growth status of your tomato plants based on the color and shape of their leaves."

[0772] 2. "Based on recent sensor data, determine if soil moisture and temperature are adequate."

[0773] 3. "Please analyze the user's current stress level from their voice data."

[0774] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0775] Step 1: Performing initial setup

[0776] Server: Check device connection

[0777] Input: Connection status of the onboard camera, sensors, terminal (robot), and emotion analysis engine

[0778] Processing: The server checks whether each device is properly connected and tests whether it is working properly. For example, it checks whether a camera is connected to the network and can transmit images in real time. It also checks the operation status of sensors and robots.

[0779] Output: Device connection status and operation check results

[0780] Specific operations: Test whether the camera can capture images properly, whether the sensor can measure data accurately, and whether the robot can operate correctly. If there is an abnormality, the user will be notified.

[0781] Server: Provides the user interface

[0782] Input: Information about the crops the user grows

[0783] Processing: The server provides a web interface for inputting information about the crops to be grown. When a user inputs the tomato variety and growing conditions, the server retrieves the relevant data from an agricultural database and stores them.

[0784] Output: Crop information and growing conditions data

[0785] Specific operation: The user inputs tomato cultivation information through the interface, and the server stores it in the database.

[0786] Step 2: Data collection

[0787] Terminal (robot): Photographing crops

[0788] Input: Crop growth status

[0789] Processing: The terminal (robot) periodically patrols the farmland and uses a camera to take pictures of the growing conditions of the crops.

[0790] Output: Captured image data

[0791] What it does: The robot patrols farmland and uses cameras to collect detailed images, such as capturing leaf color, shape, and size.

[0792] Terminal (robot): Measurement of environmental data

[0793] Input: soil moisture, temperature, light intensity

[0794] Processing: The terminal (robot) uses sensors during its patrol to measure soil humidity, temperature, and light intensity.

[0795] Output: Measured environmental data

[0796] Specific operation: The robot uses a humidity sensor to measure the moisture content of the soil and a temperature sensor to measure the environmental temperature.

[0797] Terminal (robot): Acquire emotion data

[0798] Input: User's voice and facial expressions

[0799] Processing: The terminal (robot) uses an emotion analysis engine to analyze the user's voice and facial expressions and obtain emotional data.

[0800] Output: Obtained emotion data

[0801] Specific operation: While the user is working, the robot collects and analyzes voice and facial expression data.

[0802] Terminal (robot): Sending data

[0803] Input: photographed image data, measured environmental data, acquired emotion data

[0804] Processing: The terminal (robot) sends these data to the server in real time.

[0805] Output: Data sent to server completed

[0806] Specific operation: The terminal (robot) uses wireless communication to send all collected data to the server.

[0807] Step 3: Data analysis

[0808] Server: Receives and stores data

[0809] Input: Transmitted image data, environmental data, emotional data

[0810] Processing: The server receives the data sent from the terminal (robot) and stores it in a database.

[0811] Output: Data stored in the database

[0812] Specific operation: The server stores all received data in a database in the appropriate format.

[0813] Server: Image data analysis

[0814] Input: Saved image data

[0815] Processing: The server uses a generative artificial intelligence model to analyze the image data, assessing leaf color, shape, and size to determine growth rate.

[0816] Output: Growth assessment results based on image data

[0817] Specific operation: Image analysis AI analyzes the color and shape of tomato leaves and measures their growth status.

[0818] Server: Sensor data analysis

[0819] Input: Stored environmental data

[0820] Processing: The server analyzes the environmental sensor data sent and evaluates whether the humidity, temperature, and light level are within appropriate ranges.

[0821] Output: Environmental condition assessment results

[0822] Specific operation: The data analysis system analyzes humidity, temperature, and light intensity data to determine whether the growing conditions are appropriate.

[0823] Server: Proposal Generation

[0824] Input: Image data growth assessment results, environmental condition assessment results

[0825] Processing: The server generates a proposal for the optimal growing environment based on the results of analyzing image data and sensor data.

[0826] Output: Proposed development environment

[0827] Specific action: The server generates specific instructions such as "Current soil moisture is low and additional watering is required."

[0828] Server: Emotion data analysis

[0829] Input: Acquired emotion data

[0830] Processing: The server evaluates the user's stress level based on data from the emotion analysis engine.

[0831] Output: User's stress level assessment result

[0832] Specific operation: Emotion analysis AI analyzes the user's stress level and, if necessary, makes suggestions such as "We recommend you take a moderate break."

[0833] Step 4: Providing an optimal environment

[0834] Terminal (robot): Receives operation instructions

[0835] Input: Instructions from the server

[0836] Processing: The terminal (robot) receives the operation instructions sent from the server.

[0837] Output: Actions based on received instructions

[0838] Specific action: The robot receives the instruction to "water the tomato trellis."

[0839] Terminal (robot): Performing environmental adjustments

[0840] Input: Specific operation instructions from the server

[0841] Processing: The terminal (robot) adjusts the environment according to the received instructions.

[0842] Output: Applied environmental adjustments

[0843] Specific actions: The robot waters and fertilizes the plants, as well as adjusts the lights and controls the temperature.

[0844] Step 5: Suggested operation

[0845] Server: Proposal Notification

[0846] Input: Analysis results, emotion data

[0847] Processing: The server proposes efficient operation methods to the user based on the analysis results and the user's emotional data.

[0848] Output: User is notified of the suggestion

[0849] Specific operation: The server notifies the user via the terminal with specific suggestions such as "Tomatoes in a specific area need additional watering."

[0850] User: Operate based on instructions

[0851] Input: Operation instructions from the server

[0852] Processing: The user follows instructions from the server to manage crops and adjust the environment.

[0853] Output: The result of the operation performed

[0854] Specific operation: The user receives instructions from the server and manually waters the plants, adds fertilizer, adjusts the lights, etc.

[0855] Step 6: Gather feedback

[0856] User: Feedback on work results

[0857] Input: Result of the operation performed

[0858] Processing: The user performs actual work based on the suggestions and provides feedback to the system on the results and improvements.

[0859] Output: Feedback data to the system

[0860] Specific action: The user comments, "The tomatoes are growing better thanks to the additional fertilizer," and enters this into the system.

[0861] Server: Update emotion data

[0862] Input: Emotional data after the user's task

[0863] Processing: The sentiment analysis engine continuously monitors the user's emotional changes and sends the data to the server.

[0864] Output: Updated emotion data

[0865] Specific operation: Emotion analysis AI analyzes the user's emotional data and evaluates changes in stress levels after work.

[0866] (Application example 2)

[0867] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0868] Conventional agricultural management systems primarily monitor crop growth and environmental conditions, but do not address efficient operations in industrial manufacturing sites. Furthermore, there was a lack of a system that simultaneously optimized the quality control of the manufacturing process and the health of workers, creating a challenge to balance the efficiency of factory operations with worker health management.

[0869] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0870] In this invention, the server includes means for photographing crop growth with a camera, means for measuring environmental conditions with a sensor, means for transmitting data from the camera and the sensor to the server, means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal cultivation environment, means for adjusting the cultivation environment based on the analysis results, means for notifying the user of efficient operation methods, means for receiving feedback, and means for monitoring the production process and the health of workers and providing optimal operations. This makes it possible to achieve both quality control of the production process and health management of workers.

[0871] A "camera" is a device that photographs an object and acquires its image data.

[0872] "Crop" refers to cultivated plants, especially those grown in agriculture.

[0873] "Growth" refers to the process by which crops grow and develop over time.

[0874] A "sensor" is a device that detects a physical or environmental condition and outputs it as an electrical signal.

[0875] "Environmental conditions" refers to factors that affect crops and the working environment, such as temperature, humidity, and light intensity.

[0876] "Server" refers to a central computer system for receiving data, analyzing data, and sending instructions.

[0877] A "generative artificial intelligence model" is a machine learning model that analyzes data based on specific algorithms and generates new suggestions and predictions.

[0878] "Analysis" refers to the process of processing acquired data and images to extract specific information and patterns.

[0879] "Growing environment" refers to the environmental conditions that promote optimal growth and production of agricultural crops and manufacturing processes.

[0880] "Adjustment" refers to the act of changing the conditions of a machine or environment to an optimal state in order to achieve a specific purpose.

[0881] "How to use" refers to documentation or interfaces that provide procedures or techniques for users to use a system to perform specific tasks.

[0882] "Users" refer to people who interact with the system and act on its instructions and suggestions.

[0883] "Notification" refers to the act or function of informing a user of specific information.

[0884] "Feedback" refers to results and opinions collected from users, and is information that is used to improve the performance and functionality of the system.

[0885] "Manufacturing process" refers to the series of operations and steps that create a product using raw materials.

[0886] "Health status" refers to the state of a worker's physical and mental health, including an assessment of the worker's condition in the working environment.

[0887] "Optimization" refers to adjusting a system or environment to its best state in order to achieve a specific purpose.

[0888] The system of the present invention is designed to optimize the manufacturing process and worker health in a factory, supporting efficient operations. This system is composed of cameras, sensors, a generative AI model, a server, a terminal (robot), an emotion engine, and a user. The combination of these components is described in detail below.

[0889] 1. Initial Setup

[0890] server

[0891] The server first checks whether all devices (cameras, sensors, robots, and emotion engines) are operating normally. The camera used here is a Logitech C920 HD Pro Webcam, and the temperature and humidity sensor is a DHT11. Next, it inputs initial data about the factory's manufacturing process and stores it in a database. The server then uses a web framework such as Flask or Django to build an API and establish communication with the devices.

[0892] 2. Data Collection

[0893] Terminal (robot)

[0894] The terminal (robot) patrols the factory, checking product quality with a camera and measuring environmental conditions (temperature, humidity, etc.) with sensors. It is equipped with an emotion engine and acquires emotional data from the facial expressions and voices of workers. This data is sent to the server in real time.

[0895] 3. Data Analysis

[0896] server

[0897] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. A generative AI model using TensorFlow analyzes the image data and evaluates the quality of the product. It also analyzes environmental sensor data to determine whether temperature and humidity are within appropriate ranges. It also simultaneously evaluates the user's emotional data obtained by the emotion engine and measures the user's stress level.

[0898] 4. Providing the optimal environment

[0899] Terminal (robot)

[0900] Based on the analysis results, the server sends specific instructions to the terminal (robot), such as adjusting the lighting, adjusting the temperature and humidity, or suggesting that the worker take a break.

[0901] 5. Operational suggestions

[0902] server

[0903] The server then informs the user of the optimal operation method based on the analysis results, for example, providing specific instructions such as "Additional inspection of products in a specific area is required" along with suggestions such as "Your current stress level is high, so we recommend you take a moderate break."

[0904] 6. Gathering Feedback

[0905] User

[0906] The user performs actual tasks based on the suggestions and provides feedback on the results and improvements to the system. This feedback is used to improve the quality of future analyses and suggestions. At the same time, the emotion engine continuously monitors changes in the user's emotions, and this data is also sent to the server.

[0907] For example, a robot monitors the production line with a camera and measures environmental conditions in real time using a DHT11 sensor. It also captures the user's (worker's) face with a camera and analyzes the facial expression data using an emotion engine. The results are sent to a server, where a generative AI model predicts and suggests optimal operations and improvements. Specific instructions, such as "make the lighting a little brighter," "add more moisture to the product," or "encourage workers to take breaks," can be obtained.

[0908] Example prompt sentence:

[0909] "Detect scratches and defects on products from images"

[0910] "Please evaluate how these environmental conditions (humidity, temperature) affect the quality of the product."

[0911] "Assess workers' stress levels and suggest break needs."

[0912] This system simultaneously improves the efficiency of the manufacturing process and manages the health of workers.

[0913] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0914] Step 1:

[0915] The server checks whether all devices (cameras, sensors, robots, emotion engines) are working properly.

[0916] Input: Device connection status

[0917] Output: Device connection check result

[0918] How it works: The server pings each device and checks if there is a response. It uses Flask or Django to check the device connection status via an API.

[0919] Step 2:

[0920] The server inputs initial data about the factory's manufacturing process and stores it in a database.

[0921] Input: Initial data about the manufacturing process

[0922] Output: Initial data stored in the database

[0923] Operation: The server receives manufacturing process information entered through a user interface and stores it in a database, including information such as product specifications and process sequences.

[0924] Step 3:

[0925] The terminal (robot) patrols the factory, checking product quality with cameras and measuring environmental conditions with sensors.

[0926] Input: Camera footage, environmental data measured by sensors (temperature, humidity, etc.)

[0927] Output: Photographed image data, measured environmental data

[0928] Operation: The terminal (robot) moves autonomously within the factory, taking images of products using a camera and measuring environmental conditions such as temperature and humidity using sensors.

[0929] Step 4:

[0930] The terminal (robot) uses an emotion engine to obtain emotional data from the worker's facial expressions and voice.

[0931] Input: Worker's face video and voice

[0932] Output: Parsed emotion data

[0933] How it works: The device captures the worker's face with a camera and collects their voice with a microphone. It then uses an emotion engine to analyze this data and generate emotion data.

[0934] Step 5:

[0935] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database.

[0936] Input: Image data, sensor data

[0937] Output: Raw data stored in a database

[0938] Operation: The server receives data sent from the terminal using a communication protocol and stores it in a database.

[0939] Step 6:

[0940] The server analyzes image data using a generative artificial intelligence model to evaluate product quality, as well as environmental sensor data and emotion data.

[0941] Input: Image data, environmental sensor data, emotion data

[0942] Output: Analysis results (quality evaluation, environment evaluation, emotion evaluation)

[0943] How it works: The server uses TensorFlow to analyze image data and input prompts (e.g., "Detect scratches and defects on the product from the image"), as well as environmental data such as temperature and humidity (e.g., "Evaluate how these environmental conditions affect the product's quality") and emotional data (e.g., "Evaluate the worker's stress level and suggest the need for a break").

[0944] Step 7:

[0945] Based on the analysis results, the server sends specific operation instructions to the terminal (robot).

[0946] Input: Analysis results

[0947] Output: Instructions to the terminal

[0948] Operation: Based on the analysis results, the server generates specific operational instructions, such as "brighten the lights" or "adjust the temperature and humidity," and sends them to the terminal.

[0949] Step 8:

[0950] The terminal (robot) performs specific operations on-site based on operation instructions from the server.

[0951] Input: Operation instructions from the server

[0952] Output: The result of the operation performed

[0953] Operation: The terminal follows instructions from the server and performs operations such as "adjusting lighting," "increasing humidity," and "notifying workers to take a break."

[0954] Step 9:

[0955] Based on the analysis results, the server notifies the user of efficient operation methods.

[0956] Input: Analysis results

[0957] Output: A message to inform the user

[0958] How it works: The server analyzes the results and sends specific instructions to the user's device, such as "Products in a specific area require additional testing," and suggestions, such as "We recommend taking appropriate breaks."

[0959] Step 10:

[0960] The user performs actual work based on the suggestions and provides feedback on the results and improvements to the system.

[0961] Input: User feedback data

[0962] Output: accumulation and analysis of feedback data

[0963] How it works: The user follows the suggestions and reports the results to the system. This allows the system to improve the quality of its analysis and suggestions in the future. The emotion engine also continuously monitors the user's emotional changes and sends the data to the server.

[0964] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0965] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0966] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0967] [Third embodiment]

[0968] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0969] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0970] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0971] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0972] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0973] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0974] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0975] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0976] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0977] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0978] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0979] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0980] The system of the present invention is designed to realize efficient and sustainable agricultural management. This system consists of cameras, sensors, generative AI models, servers, terminals, and users.

[0981] 1. Initial Setup

[0982] server

[0983] During initial setup, the server checks the connections of all devices and tests that the cameras and sensors are working properly. It then provides an interface for users to input information about the crops they are growing, and retrieves and stores the input data from the agricultural database. This ensures that the system is ready to provide an optimal growing environment right from the start.

[0984] 2. Data Collection

[0985] Terminal (robot)

[0986] The device patrols the farmland, taking images of the crops with its camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors. This data is sent to a server in real time. Accurate data collected at this stage ensures the accuracy of subsequent analysis.

[0987] 3. Data Analysis

[0988] server

[0989] The server analyzes the received image data and sensor data using a generative artificial intelligence model. Image analysis determines the growth status of crops and checks for disease and pest damage based on leaf color, shape, and size. Environmental assessment evaluates whether the current growing environment is appropriate based on the acquired sensor data and determines whether any deficient environmental conditions need to be corrected. The analysis results are generated as a proposal, and the next action is decided.

[0990] 4. Providing the optimal environment

[0991] Terminal (robot)

[0992] The terminal receives instructions from the server and performs specific operations to optimize the growing environment for the crops, such as watering, fertilizing, and adjusting lighting as needed, ensuring that the crops are grown under optimal conditions.

[0993] 5. Operational suggestions

[0994] server

[0995] Based on the analysis results, the server suggests efficient operation methods to the user. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. For example, specific instructions such as "crops in a specific area require additional watering" are provided.

[0996] 6. Gathering Feedback

[0997] User

[0998] After carrying out actual work based on the suggestions, users provide feedback to the system on the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. This allows the system to continually evolve, enabling more effective agricultural management.

[0999] The above is a specific embodiment of the "Auto Farm Navigation" system. This system not only enables efficient and effective crop cultivation management, but also greatly contributes to the realization of sustainable agriculture. By understanding the specific operating procedures and system flow, users can make the most of this system.

[1000] The processing flow will be explained below.

[1001] Step 1:

[1002] server

[1003] When the system starts up, the server checks the connections of all devices, including cameras, sensors, and robots. It tests each device for normal operation and notifies users if any abnormalities are detected. Users enter information about the crops they are growing through the interface, and based on that, related data is retrieved and saved from the agricultural database.

[1004] Step 2:

[1005] Terminal (robot)

[1006] The terminal (robot) uses a camera to capture images of the growing conditions of the crops at each location. It also uses sensors to measure the soil humidity, temperature, and light intensity. The acquired data is sent to a server in real time.

[1007] Step 3:

[1008] server

[1009] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. It then analyzes the image data using a generative artificial intelligence model. Specifically, it evaluates the color, shape, and size of the leaves and determines the speed of the adult. It also analyzes environmental sensor data to evaluate whether humidity, temperature, and light intensity are within appropriate ranges. Based on the analysis results, it generates a proposal for the optimal growing environment.

[1010] Step 4:

[1011] server

[1012] Based on the generated suggestions, the server generates specific instructions for adjusting the environment, such as "add 500 ml of water to area A" or "apply fertilizer to area B." These instructions are sent to the terminal (robot).

[1013] Step 5:

[1014] Terminal (robot)

[1015] The terminal (robot) adjusts the environment based on instructions from the server. For example, it may water or fertilize the appropriate areas. It may also adjust the lighting. Once the adjustment is complete, it reports the results to the server.

[1016] Step 6:

[1017] server

[1018] The server receives the results of the environmental adjustments from the robot and checks whether the suggestions were implemented properly. Furthermore, it generates suggestions for return work to the user based on the crop growth status. Specifically, the suggestions include, "The harvest time is approaching in a specific zone," or "Signs of pests and diseases are visible and need to be addressed."

[1019] Step 7:

[1020] server

[1021] The system notifies the user of suggestions for efficient operation methods, and provides specific instructions on tasks and precautions the user should take based on the suggestions. For example, it may notify the user that "crops in a specific area require additional watering."

[1022] Step 8:

[1023] User

[1024] The user follows the suggestions from the server and performs actual operations, such as harvesting or pest control. The results of the work and any improvements needed on-site are then fed back to the system, providing reference data for future analyses and to improve the quality of suggestions.

[1025] Through these processing steps, the system of the present invention can streamline crop management and support sustainable agricultural operations.

[1026] Example 1

[1027] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1028] To achieve efficient and sustainable agricultural operations, it is necessary to monitor crop growth and environmental conditions in real time and provide an appropriate growing environment. However, conventional technologies mainly rely on manual monitoring and environmental adjustment, which is labor-intensive and time-consuming, making it difficult to constantly maintain an optimal environment. It is also difficult to respond quickly to changes in environmental conditions, which can affect crop quality and yield.

[1029] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1030] In this invention, the server includes means for photographing the growth of crops with a camera, means for measuring environmental conditions with a sensor, means for transmitting data from the camera and the sensor to an information processing device, means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal cultivation environment, means for adjusting the cultivation environment based on the generated proposal, means for notifying a user of an efficient operation method based on the analysis results, and means for receiving feedback from the user. This makes it possible to monitor the growth of crops in real time, quickly respond to changes in environmental conditions, and provide an optimal cultivation environment.

[1031] A "camera" is an image capturing device for capturing images of crop growth.

[1032] A "sensor" is a device used to measure environmental conditions such as soil humidity, temperature, and light intensity.

[1033] An "information processing device" is a computer system for receiving and processing data transmitted from cameras and sensors.

[1034] A "generative artificial intelligence model" is a machine learning model used to analyze collected data and evaluate the growth status of crops and environmental conditions.

[1035] "Analysis" is the process of using a generative artificial intelligence model to evaluate the content of the received data and determine the condition of the crops and any necessary environmental adjustments.

[1036] "Growing environment" refers to the physical conditions and environmental factors (humidity, temperature, light intensity, etc.) required for crop growth.

[1037] "Feedback" refers to information that is input by the user regarding the results of work performed and improvements to the system's proposals, and is used to improve the system's analysis and proposals from the next time onwards.

[1038] "User" refers to a person or organization that uses this system to grow and manage agricultural crops.

[1039] The system of the present invention is an automated agricultural management system for achieving efficient and sustainable agricultural operations. This system is composed of a camera, sensors, a generative AI model, an information processing device, a terminal, and a user.

[1040] Initial Setup

[1041] server

[1042] During initial setup, the server checks the connectivity of all devices and tests that the cameras and sensors are working properly. Specifically, it takes test images with the cameras and collects initial data (such as temperature and humidity) from the sensors. Next, it provides an interface for users to input information about the crops they are growing. Users enter data such as the type of crop, its growth stage, and specific growing conditions. This information is then retrieved from an agricultural database and stored on the server.

[1043] Data collection

[1044] Terminal (robot)

[1045] The device patrols the farmland, taking images of the crops with its camera. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. This data is sent to a server in real time. For example, the device could take images of the entire farmland at a specific time each day to collect environmental data.

[1046] Data analysis

[1047] server

[1048] The server analyzes the received image data and sensor data using a generative AI model. Image analysis determines the growth status of the crops and checks for disease and pest damage based on leaf color, shape, and size. It also performs environmental assessments based on sensor data to determine whether the current growing environment is appropriate. The generative AI model uses deep learning to improve data accuracy. For example, it can produce analysis results such as, "The leaves of the crops have turned yellow, so there is a high possibility that they are lacking nitrogen fertilizer."

[1049] Providing the optimal environment

[1050] Terminal (robot)

[1051] The device receives instructions from the server and performs operations to optimize the growing environment for crops. For example, if the data measured by the sensor indicates a lack of water, the device will automatically water the crops. It will also apply fertilizer and adjust the lighting as needed. This ensures that crops are grown under optimal conditions.

[1052] Operation suggestions

[1053] server

[1054] Based on the analysis results, the server suggests efficient operation methods to the user. For example, it notifies the user's device of specific instructions such as "crops in a specific area require additional watering." This allows the user to follow the instructions and improve efficiency.

[1055] Collecting feedback

[1056] User

[1057] After carrying out the work based on the suggestions, the user can then provide feedback to the system about the results and areas for improvement. For example, the user can enter specific information such as, "After watering the plant, the leaves suddenly became healthier." This feedback will be used as important data to improve the quality of future analyses and suggestions.

[1058] (Example)

[1059] For example:

[1060] Example prompt:

[1061] Crops are not growing well in the growing environment. Please analyze the data collected by the sensor and suggest the best solution.

[1062] Example result:

[1063] The server uses sensor data to determine if the soil moisture is too low and instructs the device, "Additional watering is required this morning." The device receives instructions from the server and supplies the appropriate amount of water to the specified location. The user then observes changes in growth and provides feedback on the effects.

[1064] This system not only enables efficient and effective crop management, but also enables sustainable agricultural operations.

[1065] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1066] Step 1:

[1067] The server checks the connection of all devices (cameras, sensors, terminals) and tests whether they are working properly. As input, it receives initial data from the cameras and sensors (e.g., test images, temperature, humidity, etc.), and as output, it sends a signal to each device to confirm that the device is working properly.

[1068] Step 2:

[1069] The user inputs information about the crops they are growing using an interface provided by the server. The input details, such as the type of crop, its growth stage, and cultivation conditions, are sent to the server, which then retrieves and stores this information from an agricultural database.

[1070] Step 3:

[1071] The terminal (robot) patrols the farmland, taking images of the crops with a camera. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. It collects crop images and environmental data as input, and sends this data to a server in real time as output.

[1072] Step 4:

[1073] The server receives image data and sensor data sent from the device. The data received as input is sent to a generative AI model, which analyzes it. The output is an evaluation result regarding the growth status of crops and environmental conditions.

[1074] Step 5:

[1075] The server generates optimal growing environment recommendations based on the data analyzed by the generative AI model. It uses the analysis results as input and generates specific growing environment recommendations (e.g., watering amounts, fertilization needs, lighting adjustments, etc.) as output.

[1076] Step 6:

[1077] The server sends the generated cultivation environment proposal to the terminal. It receives the cultivation environment proposal as input and sends instructions as output to the terminal. The terminal performs operations to optimize the crop cultivation environment based on these instructions. Specifically, it automatically waters the crops, applies fertilizer, and adjusts the lights.

[1078] Step 7:

[1079] The server notifies the user of efficient operation methods based on the analysis results. It takes the analysis results and generated suggestions as input and sends notifications about efficient operation methods to the user's device as output. For example, it may instruct the user that "crops in a specific area require additional watering."

[1080] Step 8:

[1081] The user performs the actual work based on the suggestions from the server. The user receives instructions from the server as input and sends the results of the work as feedback to the server as output. For example, the user can input specific information such as "After watering, the leaves of the plant suddenly became healthier."

[1082] Step 9:

[1083] The server receives feedback from users and uses it to improve the quality of the next analysis and proposal. Feedback information is taken in as input, and output is used to improve the generative AI model and increase the accuracy of analysis. This allows the system to continuously evolve, enabling more effective agricultural operations.

[1084] (Application example 1)

[1085] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1086] Modern agriculture requires real-time monitoring of crop growth conditions and the provision of an appropriate growing environment. However, current manual management methods make it difficult to quickly respond to small changes in the growing environment. Furthermore, a lack of technology for efficient agricultural management makes sustainable agricultural operations difficult. Furthermore, managing large areas of farmland requires a great deal of manpower and effort, resulting in high costs.

[1087] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1088] In this invention, the server includes means for photographing crop growth with a camera, means for measuring environmental conditions with a sensor, means for transmitting data from the camera and the sensor to the server, means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal cultivation environment, means for adjusting the cultivation environment based on the analysis results, means for notifying the user of efficient operation methods, means for receiving feedback, means for being mounted on the autonomous agricultural vehicle, and means for transmitting instructions to the autonomous agricultural vehicle. This enables crop growth to be managed in an optimal environment in real time, thereby realizing efficient agricultural management and sustainable agriculture.

[1089] The "camera" is a device that photographs the growth of crops and collects visual data.

[1090] A "sensor" is a device for measuring environmental conditions (such as humidity or temperature).

[1091] A "server" is a central processing unit that receives and analyzes data sent from cameras and sensors.

[1092] A "generative artificial intelligence model" is a machine learning model that analyzes collected data and generates suggestions for growing crops.

[1093] "Optimal growing environment proposals" are proposals based on the analysis results that show the best environmental settings and operation methods to promote crop growth.

[1094] "Means for adjusting the growing environment" refers to devices or methods for actually changing the environmental conditions of agricultural crops (such as the amount of light, water, and temperature) based on the analysis results.

[1095] The "means for notifying the user of an efficient operation method" refers to a method or device for conveying specific operation instructions to the user based on the analysis results.

[1096] "Means for receiving feedback" refers to a method or device for sending back to the system the results of operations performed by the user in accordance with instructions and suggestions for improvement.

[1097] An "autonomous agricultural vehicle" is a vehicle that operates autonomously according to the growth status of crops and environmental conditions, and manages the growing environment.

[1098] "Means for sending instructions to an autonomous agricultural vehicle" refers to a method or device for transmitting specific operational instructions from a server to an autonomous agricultural vehicle.

[1099] The present invention relates to a system for highly automating the cultivation and environmental management of agricultural crops in an autonomous agricultural vehicle. The system for carrying out the present invention is configured as follows.

[1100] 1. Initial Setup

[1101] server

[1102] The server checks the connection of all devices during the initial setup phase, tests whether the cameras and sensors are working properly, provides an interface for users to input information about the crops they are growing, and retrieves and stores the input data from an agricultural database.

[1103] 2. Data Collection

[1104] Autonomous agricultural vehicles

[1105] The autonomous agricultural vehicle travels around the field, taking images of the crops with a camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors, and transmits this data to a server in real time.

[1106] 3. Data Analysis

[1107] server

[1108] The server analyzes the received image data and sensor data using a generative artificial intelligence model. Image analysis determines the growth status of crops and checks for disease and pest damage based on leaf color, shape, and size. Environmental assessment evaluates whether the current growing environment is appropriate based on the acquired sensor data and determines whether any deficient environmental conditions need to be corrected. The analysis results are generated as a proposal, and the next action is decided.

[1109] 4. Providing the optimal environment

[1110] Autonomous agricultural vehicles

[1111] The autonomous agricultural vehicle receives instructions from the server and performs specific operations to optimize the growing environment for crops, such as watering, fertilizing, and adjusting lights as needed, ensuring that crops are grown under optimal growing conditions.

[1112] 5. Operational suggestions

[1113] server

[1114] Based on the analysis results, the server suggests efficient operation methods to the user. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. Specific instructions include information such as "crops in a specific area require additional watering."

[1115] 6. Gathering Feedback

[1116] User

[1117] After carrying out actual work based on the suggestions, users provide feedback to the system on the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. This allows the system to continually evolve, enabling more effective agricultural management.

[1118] Hardware and Software Used

[1119] Hardware: Cameras, humidity sensors, temperature sensors, autonomous agricultural vehicles

[1120] Software: Python, OpenCV (image processing library), requests (HTTP communication library), generative artificial intelligence model using deep learning

[1121] Example prompt

[1122] In a real application, the next action is determined based on the data analysis results and instructions are sent to the vehicle. For example, if data is collected at a humidity of 45% and a temperature of 24°C, the prompt sentence would be as follows:

[1123] "Data Collection: Environmental data was collected at 45% humidity and 24°C. Based on the analysis, area A1 requires additional watering. Instructions: Add 2 liters of water to area A1."

[1124] This will enable optimal environmental management of crops in real time, resulting in efficient agricultural operations.

[1125] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1126] Step 1:

[1127] As an initial setup, the server checks the connection of all devices and checks that the cameras and sensors are working properly. It also obtains information about the crops being grown based on user input, accesses an agricultural database, and stores the necessary data.

[1128] Input: User input of crop information, hardware operating status

[1129] Output: Device connection status report, saved crop data

[1130] Operation: The server checks the device connection and accesses data from the agricultural database

[1131] Step 2:

[1132] The autonomous agricultural vehicle patrols farmland, taking images of crops with a camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors, and transmits the acquired data to a server in real time.

[1133] Input: Farmland environmental conditions, camera and sensor information

[1134] Output: Image data captured, environmental data measured

[1135] Operation: Taking images and measuring data using an autonomous agricultural vehicle, and sending the data to a server

[1136] Step 3:

[1137] The server analyzes the received image data and sensor data using a generative artificial intelligence model. Image analysis determines the growth status of the crops and the presence of diseases and pests. Environmental assessment uses the sensor data to evaluate whether the growing environment is appropriate and determine whether any corrections are necessary.

[1138] Input: Image data, sensor data

[1139] Output: Analysis result report, environmental assessment report

[1140] Operation: Data analysis by server, evaluation of growth status and environment

[1141] Step 4:

[1142] Based on the analysis results, the server instructs the autonomous agricultural vehicle on specific operations to provide an optimal growing environment, such as watering, fertilizing, and adjusting lights as needed.

[1143] Input: Analysis results

[1144] Output: Operation instructions for the autonomous agricultural vehicle

[1145] Operation: Sending instructions from the server to the autonomous agricultural vehicle and executing specific operations

[1146] Step 5:

[1147] Based on the analysis results, the server suggests efficient operation methods to the user. The suggestions are sent to the user's device, and the user then performs the operation. For example, specific instructions such as "crops in a specific area require additional watering" are provided.

[1148] Input: Analysis results, operation suggestions

[1149] Output: Notification of suggested action to user

[1150] Action: Notification from the server to the user device, execution of user operation

[1151] Step 6:

[1152] After the user performs the actual work based on the suggestions, they provide feedback to the system regarding the results and improvements. This feedback is used to improve the accuracy of future analyses and suggestions.

[1153] Input: Results of user actions and improvements

[1154] Output: Feedback data sent to the server

[1155] How it works: User sends feedback, server receives data and improves analysis accuracy

[1156] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1157] The system of the present invention is designed to achieve efficient and sustainable agricultural management. This system is composed of cameras, sensors, generative AI models, servers, terminals (robots), emotion engines, and users. The combination of these components is explained in detail below.

[1158] 1. Initial Setup

[1159] server

[1160] During the initial setup, the server checks the connections of all devices and tests that the cameras, sensors, robots, and emotion engine are working properly. It also provides an interface for users to input information about the crops they are growing, and retrieves and stores the input data from the agricultural database.

[1161] 2. Data Collection

[1162] Terminal (robot)

[1163] The terminal (robot) patrols the farmland, taking pictures of the growing crops with a camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors. The acquired data is sent to a server in real time. In addition, an emotion engine analyzes the user's voice and facial expressions to obtain emotional data.

[1164] 3. Data Analysis

[1165] server

[1166] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. It then analyzes the image data using a generative artificial intelligence model. Specifically, it evaluates the color, shape, and size of the leaves to determine the rate of growth. It also analyzes environmental sensor data to determine whether humidity, temperature, and light intensity are within appropriate ranges. Based on the analysis results, it generates suggestions for the optimal growing environment. It also evaluates the user's stress level based on the user's emotional data obtained by the emotion engine.

[1167] 4. Providing the optimal environment

[1168] Terminal (robot)

[1169] The terminal (robot) receives instructions from the server and performs specific operations to optimize the growing environment for the crops. For example, it waters, fertilizes, adjusts lighting, etc. as needed, ensuring that the crops are grown under optimal conditions.

[1170] 5. Operational suggestions

[1171] server

[1172] The server then suggests efficient operation methods to the user based on the analysis results and the user's emotional data. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. For example, along with specific instructions such as "crops in a specific area require additional watering," the server may provide suggestions such as "your current stress level is high, so we recommend you take a moderate break."

[1173] 6. Gathering Feedback

[1174] User

[1175] After the user performs the actual work based on the suggestions, they provide feedback to the system about the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. At the same time, the emotion engine continuously monitors changes in the user's emotions, and this data is also sent to the server.

[1176] Through these steps, the Auto Farm Navigation system not only streamlines crop cultivation management and supports sustainable agricultural operations, but also reduces the mental burden on users. For example, when users harvest, the system provides appropriate guidance on harvesting areas and adjusts work times according to the user's stress level, making farming more efficient and less stressful.

[1177] The processing flow will be explained below.

[1178] Step 1:

[1179] server

[1180] When the system starts up, the server checks the connection of all devices. It tests the normal operation of the cameras, sensors, robots, and emotion engine, and notifies the user if any abnormalities are detected. Next, the user inputs information about the crops they are growing through the interface, which retrieves and stores related data from the agricultural database.

[1181] Step 2:

[1182] Terminal (robot)

[1183] The terminal (robot) patrols the farmland, taking pictures of the growth of each crop with a camera. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. This data is sent to a server in real time.

[1184] Step 3:

[1185] Terminal (emotion engine)

[1186] The emotion engine analyzes the user's voice and facial expressions to obtain emotional data, which is then immediately sent to the server.

[1187] Step 4:

[1188] server

[1189] The server receives image data and sensor data sent from the terminal (robot), as well as emotion data sent from the emotion engine, and stores them in a database. It then uses a generative AI model to analyze the image data. Specifically, it evaluates the color, shape, and size of the leaves to determine the rate of growth. It also analyzes environmental sensor data to determine whether humidity, temperature, and light intensity are within appropriate ranges. It uses the emotion data to evaluate the user's stress level, and based on all this data, it generates suggestions for the optimal growing environment and operation methods.

[1190] Step 5:

[1191] server

[1192] Based on the generated suggestions, the server generates specific instructions for adjusting the environment. For example, it creates specific instructions such as "Add 500 ml of water to area A" or "Apply fertilizer to area B" and sends them to the terminal (robot). It also generates suggestions based on emotion data, such as "Your current stress level is high, so we recommend you take a moderate break."

[1193] Step 6:

[1194] Terminal (robot)

[1195] The terminal (robot) adjusts the environment based on instructions from the server. For example, it may water or fertilize the appropriate areas. It may also adjust the lighting. Once the adjustment is complete, it reports the results to the server.

[1196] Step 7:

[1197] server

[1198] The server receives the results of the environmental adjustments from the robot and checks whether the suggestions were implemented properly. Based on the crop growth status, the server generates suggestions for further restoration work for the user. Specifically, it may suggest that "harvesting time is approaching in a specific zone." It also suggests ways to reduce stress levels based on emotion data.

[1199] Step 8:

[1200] server

[1201] The suggestions are sent to the user's device, allowing them to understand the necessary operations and timing for breaks. Specific instructions include "crops in a specific area need additional watering" and "your current stress level is high, so please take a moderate break."

[1202] Step 9:

[1203] User

[1204] The user follows the suggestions from the server and performs actual operations, such as harvesting or pest control. Afterwards, the results of the work and any improvements needed on-site are fed back to the system. The emotion engine continues to monitor the user's emotion data and also sends that data to the server.

[1205] Through these processing steps, the Auto Farm Navigation system not only streamlines crop cultivation management and supports sustainable agricultural operations, but also reduces the user's mental burden. For example, when harvesting, the system provides appropriate judgment of the harvesting area and adjusts work time according to the user's stress level, making agricultural work more efficient and less stressful.

[1206] Example 2

[1207] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1208] In modern agriculture, while there is a demand for improving production efficiency and ensuring the quality of agricultural products, there is also a need to reduce the burden of farm work and achieve sustainable operations. Conventional systems have the technology to collect and analyze environmental and growth data, but they lack mechanisms for improving work efficiency and reducing the burden by taking user emotional data into account. This can lead to the accumulation of fatigue and stress, which can lead to reduced productivity. Furthermore, proposals for appropriate growing environments and work methods are limited, resulting in a lack of flexible and comprehensive support for agricultural operations.

[1209] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1210] In this invention, the server includes a means for capturing images of crop growth with a camera, a means for measuring environmental conditions with a sensor, a means for transmitting data from the camera and the sensor to the server, a means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal growing environment, a means for adjusting the growing environment based on the analysis results, a means for informing the user of efficient operation methods, a means for receiving feedback, a means for acquiring and analyzing the user's emotional data with an emotion analysis engine, and a means for evaluating the user's stress level and proposing appropriate working hours based on the user's emotional data. This not only improves the efficiency of agricultural work, but also reduces user stress and provides an appropriate working environment. Furthermore, by comprehensively analyzing the acquired data, maintaining an optimal growing environment, and proposing agricultural work methods, sustainable agricultural operations can be realized.

[1211] A "camera" is a device used to photograph the growth of crops.

[1212] A "sensor" is a device for measuring environmental conditions (humidity, temperature, light intensity, etc.).

[1213] "Server" is a computer system that receives, stores, and analyzes data sent from devices, and generates and provides various proposals.

[1214] A "generative artificial intelligence model" is an AI algorithm that analyzes the transmitted data and generates suggestions for the optimal development environment.

[1215] "Adjusting the growing environment" refers to the operation of adjusting the environment necessary for crop growth based on the analysis results.

[1216] "Efficient operation methods" refer to suggestions and instructions that allow users to manage crops and adjust the environment more effectively.

[1217] "Feedback" refers to information returned to the system regarding the results of the work performed by the user and areas for improvement.

[1218] An "emotion analysis engine" is an algorithm that analyzes a user's voice and facial expressions to obtain and evaluate emotional data.

[1219] "Emotion data" is data that indicates the user's emotional state and stress level.

[1220] "Stress level" is an index that represents the degree of stress that the user is feeling.

[1221] The system of the present invention is designed to achieve efficient and sustainable agricultural management. This system is composed of a combination of multiple hardware and software components, including cameras, sensors, generative AI models, servers, terminals (robots), an emotion analysis engine, and users.

[1222] 1. Initial Setup

[1223] server

[1224] During initial setup, the server checks the connections of all devices and tests that the cameras, sensors, terminals (robots), and sentiment analysis engine are working properly. The server also provides an interface for users to input information about the crops they are growing, and retrieves and stores the input crop data from the agricultural database. For example, if a user is growing tomatoes, the server retrieves the optimal growing conditions for tomatoes from the agricultural database and reflects them in the system.

[1225] 2. Data Collection

[1226] Terminal (robot)

[1227] The terminal (robot) patrols the farmland, using a camera to take pictures of the growing state of the crops. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. This data is sent to a server in real time. The terminal (robot) also uses an emotion analysis engine to analyze the user's voice and facial expressions to obtain emotional data about the user. For example, while the user is harvesting, the robot measures the soil humidity and temperature and sends the data to the server.

[1228] 3. Data Analysis

[1229] server

[1230] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. It then uses a generative AI model to analyze the image data. Specifically, it evaluates the color, shape, and size of the leaves and determines the rate of growth. It also analyzes environmental sensor data to determine whether humidity, temperature, and light intensity are within appropriate ranges. It also evaluates the user's stress level based on the user's emotional data obtained by an emotion analysis engine.

[1231] Proposal generation

[1232] Based on the analysis results, the server generates recommendations for the optimal growing environment. For example, it generates specific instructions such as, "The current soil humidity is low, so additional watering is required." It also makes suggestions such as, "We recommend taking moderate breaks" if the user's stress level is high.

[1233] 4. Providing the optimal environment

[1234] Terminal (robot)

[1235] The terminal (robot) receives instructions from the server and performs specific operations to optimize the growing environment for the crops. For example, it performs watering, fertilization, and light adjustments. This ensures that the crops are grown under optimal conditions. If the robot detects a nutrient deficiency in the tomatoes, it can automatically supply the appropriate amount of fertilizer.

[1236] 5. Operational suggestions

[1237] server

[1238] The server then suggests efficient ways to operate the system based on the analysis results and the user's emotional data. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. For example, the server might suggest, "Your current stress level is high, so we recommend you take a moderate break."

[1239] 6. Gathering Feedback

[1240] User

[1241] The user performs the work based on the suggestions and provides feedback to the system with the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. The sentiment analysis engine also continuously monitors changes in the user's emotions, and this data is also sent to the server. If a user provides feedback such as "The additional fertilizer has helped the tomatoes grow better," this will be reflected in future fertilizer management.

[1242] Specific examples of prompts to input to generative AI models

[1243] 1. "Evaluate the current growth status of your tomato plants based on the color and shape of their leaves."

[1244] 2. "Based on recent sensor data, determine if soil moisture and temperature are adequate."

[1245] 3. "Please analyze the user's current stress level from their voice data."

[1246] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1247] Step 1: Performing initial setup

[1248] Server: Check device connection

[1249] Input: Connection status of the onboard camera, sensors, terminal (robot), and emotion analysis engine

[1250] Processing: The server checks whether each device is properly connected and tests whether it is working properly. For example, it checks whether a camera is connected to the network and can transmit images in real time. It also checks the operation status of sensors and robots.

[1251] Output: Device connection status and operation check results

[1252] Specific operations: Test whether the camera can capture images properly, whether the sensor can measure data accurately, and whether the robot can operate correctly. If there is an abnormality, the user will be notified.

[1253] Server: Provides the user interface

[1254] Input: Information about the crops the user grows

[1255] Processing: The server provides a web interface for inputting information about the crops to be grown. When a user inputs the tomato variety and growing conditions, the server retrieves the relevant data from an agricultural database and stores them.

[1256] Output: Crop information and growing conditions data

[1257] Specific operation: The user inputs tomato cultivation information through the interface, and the server stores it in the database.

[1258] Step 2: Data collection

[1259] Terminal (robot): Photographing crops

[1260] Input: Crop growth status

[1261] Processing: The terminal (robot) periodically patrols the farmland and uses a camera to take pictures of the growing conditions of the crops.

[1262] Output: Captured image data

[1263] What it does: The robot patrols farmland and uses cameras to collect detailed images, such as capturing leaf color, shape, and size.

[1264] Terminal (robot): Measurement of environmental data

[1265] Input: soil moisture, temperature, light intensity

[1266] Processing: The terminal (robot) uses sensors during its patrol to measure soil humidity, temperature, and light intensity.

[1267] Output: Measured environmental data

[1268] Specific operation: The robot uses a humidity sensor to measure the moisture content of the soil and a temperature sensor to measure the environmental temperature.

[1269] Terminal (robot): Acquire emotion data

[1270] Input: User's voice and facial expressions

[1271] Processing: The terminal (robot) uses an emotion analysis engine to analyze the user's voice and facial expressions and obtain emotional data.

[1272] Output: Obtained emotion data

[1273] Specific operation: While the user is working, the robot collects and analyzes voice and facial expression data.

[1274] Terminal (robot): Sending data

[1275] Input: photographed image data, measured environmental data, acquired emotion data

[1276] Processing: The terminal (robot) sends these data to the server in real time.

[1277] Output: Data sent to server completed

[1278] Specific operation: The terminal (robot) uses wireless communication to send all collected data to the server.

[1279] Step 3: Data analysis

[1280] Server: Receives and stores data

[1281] Input: Transmitted image data, environmental data, emotional data

[1282] Processing: The server receives the data sent from the terminal (robot) and stores it in a database.

[1283] Output: Data stored in the database

[1284] Specific operation: The server stores all received data in a database in the appropriate format.

[1285] Server: Image data analysis

[1286] Input: Saved image data

[1287] Processing: The server uses a generative artificial intelligence model to analyze the image data, assessing leaf color, shape, and size to determine growth rate.

[1288] Output: Growth assessment results based on image data

[1289] Specific operation: Image analysis AI analyzes the color and shape of tomato leaves and measures their growth status.

[1290] Server: Sensor data analysis

[1291] Input: Stored environmental data

[1292] Processing: The server analyzes the environmental sensor data sent and evaluates whether the humidity, temperature, and light level are within appropriate ranges.

[1293] Output: Environmental condition assessment results

[1294] Specific operation: The data analysis system analyzes humidity, temperature, and light intensity data to determine whether the growing conditions are appropriate.

[1295] Server: Proposal Generation

[1296] Input: Image data growth assessment results, environmental condition assessment results

[1297] Processing: The server generates a proposal for the optimal growing environment based on the results of analyzing image data and sensor data.

[1298] Output: Proposed development environment

[1299] Specific action: The server generates specific instructions such as "Current soil moisture is low and additional watering is required."

[1300] Server: Emotion data analysis

[1301] Input: Acquired emotion data

[1302] Processing: The server evaluates the user's stress level based on data from the emotion analysis engine.

[1303] Output: User's stress level assessment result

[1304] Specific operation: Emotion analysis AI analyzes the user's stress level and, if necessary, makes suggestions such as "We recommend you take a moderate break."

[1305] Step 4: Providing an optimal environment

[1306] Terminal (robot): Receives operation instructions

[1307] Input: Instructions from the server

[1308] Processing: The terminal (robot) receives the operation instructions sent from the server.

[1309] Output: Actions based on received instructions

[1310] Specific action: The robot receives the instruction to "water the tomato trellis."

[1311] Terminal (robot): Performing environmental adjustments

[1312] Input: Specific operation instructions from the server

[1313] Processing: The terminal (robot) adjusts the environment according to the received instructions.

[1314] Output: Applied environmental adjustments

[1315] Specific actions: The robot waters and fertilizes the plants, as well as adjusts the lights and controls the temperature.

[1316] Step 5: Suggested operation

[1317] Server: Proposal Notification

[1318] Input: Analysis results, emotion data

[1319] Processing: The server proposes efficient operation methods to the user based on the analysis results and the user's emotional data.

[1320] Output: User is notified of the suggestion

[1321] Specific operation: The server notifies the user via the terminal with specific suggestions such as "Tomatoes in a specific area need additional watering."

[1322] User: Operate based on instructions

[1323] Input: Operation instructions from the server

[1324] Processing: The user follows instructions from the server to manage crops and adjust the environment.

[1325] Output: The result of the operation performed

[1326] Specific operation: The user receives instructions from the server and manually waters the plants, adds fertilizer, adjusts the lights, etc.

[1327] Step 6: Gather feedback

[1328] User: Feedback on work results

[1329] Input: Result of the operation performed

[1330] Processing: The user performs actual work based on the suggestions and provides feedback to the system on the results and improvements.

[1331] Output: Feedback data to the system

[1332] Specific action: The user comments, "The tomatoes are growing better thanks to the additional fertilizer," and enters this into the system.

[1333] Server: Update emotion data

[1334] Input: Emotional data after the user's task

[1335] Processing: The sentiment analysis engine continuously monitors the user's emotional changes and sends the data to the server.

[1336] Output: Updated emotion data

[1337] Specific operation: Emotion analysis AI analyzes the user's emotional data and evaluates changes in stress levels after work.

[1338] (Application example 2)

[1339] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1340] Conventional agricultural management systems primarily monitor crop growth and environmental conditions, but do not address efficient operations in industrial manufacturing sites. Furthermore, there was a lack of a system that simultaneously optimized the quality control of the manufacturing process and the health of workers, creating a challenge to balance the efficiency of factory operations with worker health management.

[1341] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1342] In this invention, the server includes means for photographing crop growth with a camera, means for measuring environmental conditions with a sensor, means for transmitting data from the camera and the sensor to the server, means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal cultivation environment, means for adjusting the cultivation environment based on the analysis results, means for notifying the user of efficient operation methods, means for receiving feedback, and means for monitoring the production process and the health of workers and providing optimal operations. This makes it possible to achieve both quality control of the production process and health management of workers.

[1343] A "camera" is a device that photographs an object and acquires its image data.

[1344] "Crop" refers to cultivated plants, especially those grown in agriculture.

[1345] "Growth" refers to the process by which crops grow and develop over time.

[1346] A "sensor" is a device that detects a physical or environmental condition and outputs it as an electrical signal.

[1347] "Environmental conditions" refers to factors that affect crops and the working environment, such as temperature, humidity, and light intensity.

[1348] "Server" refers to a central computer system for receiving data, analyzing data, and sending instructions.

[1349] A "generative artificial intelligence model" is a machine learning model that analyzes data based on specific algorithms and generates new suggestions and predictions.

[1350] "Analysis" refers to the process of processing acquired data and images to extract specific information and patterns.

[1351] "Growing environment" refers to the environmental conditions that promote optimal growth and production of agricultural crops and manufacturing processes.

[1352] "Adjustment" refers to the act of changing the conditions of a machine or environment to an optimal state in order to achieve a specific purpose.

[1353] "How to use" refers to documentation or interfaces that provide procedures or techniques for users to use a system to perform specific tasks.

[1354] "Users" refer to people who interact with the system and act on its instructions and suggestions.

[1355] "Notification" refers to the act or function of informing a user of specific information.

[1356] "Feedback" refers to results and opinions collected from users, and is information that is used to improve the performance and functionality of the system.

[1357] "Manufacturing process" refers to the series of operations and steps that create a product using raw materials.

[1358] "Health status" refers to the state of a worker's physical and mental health, including an assessment of the worker's condition in the working environment.

[1359] "Optimization" refers to adjusting a system or environment to its best state in order to achieve a specific purpose.

[1360] The system of the present invention is designed to optimize the manufacturing process and worker health in a factory, supporting efficient operations. This system is composed of cameras, sensors, a generative AI model, a server, a terminal (robot), an emotion engine, and a user. The combination of these components is described in detail below.

[1361] 1. Initial Setup

[1362] server

[1363] The server first checks whether all devices (cameras, sensors, robots, and emotion engines) are operating normally. The camera used here is a Logitech C920 HD Pro Webcam, and the temperature and humidity sensor is a DHT11. Next, it inputs initial data about the factory's manufacturing process and stores it in a database. The server then uses a web framework such as Flask or Django to build an API and establish communication with the devices.

[1364] 2. Data Collection

[1365] Terminal (robot)

[1366] The terminal (robot) patrols the factory, checking product quality with a camera and measuring environmental conditions (temperature, humidity, etc.) with sensors. It is equipped with an emotion engine and acquires emotional data from the facial expressions and voices of workers. This data is sent to the server in real time.

[1367] 3. Data Analysis

[1368] server

[1369] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. A generative AI model using TensorFlow analyzes the image data and evaluates the quality of the product. It also analyzes environmental sensor data to determine whether temperature and humidity are within appropriate ranges. It also simultaneously evaluates the user's emotional data obtained by the emotion engine and measures the user's stress level.

[1370] 4. Providing the optimal environment

[1371] Terminal (robot)

[1372] Based on the analysis results, the server sends specific instructions to the terminal (robot), such as adjusting the lighting, adjusting the temperature and humidity, or suggesting that the worker take a break.

[1373] 5. Operational suggestions

[1374] server

[1375] The server then informs the user of the optimal operation method based on the analysis results, for example, providing specific instructions such as "Additional inspection of products in a specific area is required" along with suggestions such as "Your current stress level is high, so we recommend you take a moderate break."

[1376] 6. Gathering Feedback

[1377] User

[1378] The user performs actual tasks based on the suggestions and provides feedback on the results and improvements to the system. This feedback is used to improve the quality of future analyses and suggestions. At the same time, the emotion engine continuously monitors changes in the user's emotions, and this data is also sent to the server.

[1379] For example, a robot monitors the production line with a camera and measures environmental conditions in real time using a DHT11 sensor. It also captures the user's (worker's) face with a camera and analyzes the facial expression data using an emotion engine. The results are sent to a server, where a generative AI model predicts and suggests optimal operations and improvements. Specific instructions, such as "make the lighting a little brighter," "add more moisture to the product," or "encourage workers to take breaks," can be obtained.

[1380] Example prompt sentence:

[1381] "Detect scratches and defects on products from images"

[1382] "Please evaluate how these environmental conditions (humidity, temperature) affect the quality of the product."

[1383] "Assess workers' stress levels and suggest break needs."

[1384] This system simultaneously improves the efficiency of the manufacturing process and manages the health of workers.

[1385] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1386] Step 1:

[1387] The server checks whether all devices (cameras, sensors, robots, emotion engines) are working properly.

[1388] Input: Device connection status

[1389] Output: Device connection check result

[1390] How it works: The server pings each device and checks if there is a response. It uses Flask or Django to check the device connection status via an API.

[1391] Step 2:

[1392] The server inputs initial data about the factory's manufacturing process and stores it in a database.

[1393] Input: Initial data about the manufacturing process

[1394] Output: Initial data stored in the database

[1395] Operation: The server receives manufacturing process information entered through a user interface and stores it in a database, including information such as product specifications and process sequences.

[1396] Step 3:

[1397] The terminal (robot) patrols the factory, checking product quality with cameras and measuring environmental conditions with sensors.

[1398] Input: Camera footage, environmental data measured by sensors (temperature, humidity, etc.)

[1399] Output: Photographed image data, measured environmental data

[1400] Operation: The terminal (robot) moves autonomously within the factory, taking images of products using a camera and measuring environmental conditions such as temperature and humidity using sensors.

[1401] Step 4:

[1402] The terminal (robot) uses an emotion engine to obtain emotional data from the worker's facial expressions and voice.

[1403] Input: Worker's face video and voice

[1404] Output: Parsed emotion data

[1405] How it works: The device captures the worker's face with a camera and collects their voice with a microphone. It then uses an emotion engine to analyze this data and generate emotion data.

[1406] Step 5:

[1407] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database.

[1408] Input: Image data, sensor data

[1409] Output: Raw data stored in a database

[1410] Operation: The server receives data sent from the terminal using a communication protocol and stores it in a database.

[1411] Step 6:

[1412] The server analyzes image data using a generative artificial intelligence model to evaluate product quality, as well as environmental sensor data and emotion data.

[1413] Input: Image data, environmental sensor data, emotion data

[1414] Output: Analysis results (quality evaluation, environment evaluation, emotion evaluation)

[1415] How it works: The server uses TensorFlow to analyze image data and input prompts (e.g., "Detect scratches and defects on the product from the image"), as well as environmental data such as temperature and humidity (e.g., "Evaluate how these environmental conditions affect the product's quality") and emotional data (e.g., "Evaluate the worker's stress level and suggest the need for a break").

[1416] Step 7:

[1417] Based on the analysis results, the server sends specific operation instructions to the terminal (robot).

[1418] Input: Analysis results

[1419] Output: Instructions to the terminal

[1420] Operation: Based on the analysis results, the server generates specific operational instructions, such as "brighten the lights" or "adjust the temperature and humidity," and sends them to the terminal.

[1421] Step 8:

[1422] The terminal (robot) performs specific operations on-site based on operation instructions from the server.

[1423] Input: Operation instructions from the server

[1424] Output: The result of the operation performed

[1425] Operation: The terminal follows instructions from the server and performs operations such as "adjusting lighting," "increasing humidity," and "notifying workers to take a break."

[1426] Step 9:

[1427] Based on the analysis results, the server notifies the user of efficient operation methods.

[1428] Input: Analysis results

[1429] Output: A message to inform the user

[1430] How it works: The server analyzes the results and sends specific instructions to the user's device, such as "Products in a specific area require additional testing," and suggestions, such as "We recommend taking appropriate breaks."

[1431] Step 10:

[1432] The user performs actual work based on the suggestions and provides feedback on the results and improvements to the system.

[1433] Input: User feedback data

[1434] Output: accumulation and analysis of feedback data

[1435] How it works: The user follows the suggestions and reports the results to the system. This allows the system to improve the quality of its analysis and suggestions in the future. The emotion engine also continuously monitors the user's emotional changes and sends the data to the server.

[1436] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1437] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1438] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1439] [Fourth embodiment]

[1440] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1441] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1442] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1443] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1444] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1445] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1446] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1447] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1448] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1449] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1450] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1451] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1452] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1453] The system of the present invention is designed to realize efficient and sustainable agricultural management. This system consists of cameras, sensors, generative AI models, servers, terminals, and users.

[1454] 1. Initial Setup

[1455] server

[1456] During initial setup, the server checks the connections of all devices and tests that the cameras and sensors are working properly. It then provides an interface for users to input information about the crops they are growing, and retrieves and stores the input data from the agricultural database. This ensures that the system is ready to provide an optimal growing environment right from the start.

[1457] 2. Data Collection

[1458] Terminal (robot)

[1459] The device patrols the farmland, taking images of the crops with its camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors. This data is sent to a server in real time. Accurate data collected at this stage ensures the accuracy of subsequent analysis.

[1460] 3. Data Analysis

[1461] server

[1462] The server analyzes the received image data and sensor data using a generative artificial intelligence model. Image analysis determines the growth status of crops and checks for disease and pest damage based on leaf color, shape, and size. Environmental assessment evaluates whether the current growing environment is appropriate based on the acquired sensor data and determines whether any deficient environmental conditions need to be corrected. The analysis results are generated as a proposal, and the next action is decided.

[1463] 4. Providing the optimal environment

[1464] Terminal (robot)

[1465] The terminal receives instructions from the server and performs specific operations to optimize the growing environment for the crops, such as watering, fertilizing, and adjusting lighting as needed, ensuring that the crops are grown under optimal conditions.

[1466] 5. Operational suggestions

[1467] server

[1468] Based on the analysis results, the server suggests efficient operation methods to the user. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. For example, specific instructions such as "crops in a specific area require additional watering" are provided.

[1469] 6. Gathering Feedback

[1470] User

[1471] After carrying out actual work based on the suggestions, users provide feedback to the system on the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. This allows the system to continually evolve, enabling more effective agricultural management.

[1472] The above is a specific embodiment of the "Auto Farm Navigation" system. This system not only enables efficient and effective crop cultivation management, but also greatly contributes to the realization of sustainable agriculture. By understanding the specific operating procedures and system flow, users can make the most of this system.

[1473] The processing flow will be explained below.

[1474] Step 1:

[1475] server

[1476] When the system starts up, the server checks the connections of all devices, including cameras, sensors, and robots. It tests each device for normal operation and notifies users if any abnormalities are detected. Users enter information about the crops they are growing through the interface, and based on that, related data is retrieved and saved from the agricultural database.

[1477] Step 2:

[1478] Terminal (robot)

[1479] The terminal (robot) uses a camera to capture images of the growing conditions of the crops at each location. It also uses sensors to measure the soil humidity, temperature, and light intensity. The acquired data is sent to a server in real time.

[1480] Step 3:

[1481] server

[1482] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. It then analyzes the image data using a generative artificial intelligence model. Specifically, it evaluates the color, shape, and size of the leaves and determines the speed of the adult. It also analyzes environmental sensor data to evaluate whether humidity, temperature, and light intensity are within appropriate ranges. Based on the analysis results, it generates a proposal for the optimal growing environment.

[1483] Step 4:

[1484] server

[1485] Based on the generated suggestions, the server generates specific instructions for adjusting the environment, such as "add 500 ml of water to area A" or "apply fertilizer to area B." These instructions are sent to the terminal (robot).

[1486] Step 5:

[1487] Terminal (robot)

[1488] The terminal (robot) adjusts the environment based on instructions from the server. For example, it may water or fertilize the appropriate areas. It may also adjust the lighting. Once the adjustment is complete, it reports the results to the server.

[1489] Step 6:

[1490] server

[1491] The server receives the results of the environmental adjustments from the robot and checks whether the suggestions were implemented properly. Furthermore, it generates suggestions for return work to the user based on the crop growth status. Specifically, the suggestions include, "The harvest time is approaching in a specific zone," or "Signs of pests and diseases are visible and need to be addressed."

[1492] Step 7:

[1493] server

[1494] The system notifies the user of suggestions for efficient operation methods, and provides specific instructions on tasks and precautions the user should take based on the suggestions. For example, it may notify the user that "crops in a specific area require additional watering."

[1495] Step 8:

[1496] User

[1497] The user follows the suggestions from the server and performs actual operations, such as harvesting or pest control. The results of the work and any improvements needed on-site are then fed back to the system, providing reference data for future analyses and to improve the quality of suggestions.

[1498] Through these processing steps, the system of the present invention can streamline crop management and support sustainable agricultural operations.

[1499] Example 1

[1500] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1501] To achieve efficient and sustainable agricultural operations, it is necessary to monitor crop growth and environmental conditions in real time and provide an appropriate growing environment. However, conventional technologies mainly rely on manual monitoring and environmental adjustment, which is labor-intensive and time-consuming, making it difficult to constantly maintain an optimal environment. It is also difficult to respond quickly to changes in environmental conditions, which can affect crop quality and yield.

[1502] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1503] In this invention, the server includes means for photographing the growth of crops with a camera, means for measuring environmental conditions with a sensor, means for transmitting data from the camera and the sensor to an information processing device, means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal cultivation environment, means for adjusting the cultivation environment based on the generated proposal, means for notifying a user of an efficient operation method based on the analysis results, and means for receiving feedback from the user. This makes it possible to monitor the growth of crops in real time, quickly respond to changes in environmental conditions, and provide an optimal cultivation environment.

[1504] A "camera" is an image capturing device for capturing images of crop growth.

[1505] A "sensor" is a device used to measure environmental conditions such as soil humidity, temperature, and light intensity.

[1506] An "information processing device" is a computer system for receiving and processing data transmitted from cameras and sensors.

[1507] A "generative artificial intelligence model" is a machine learning model used to analyze collected data and evaluate the growth status of crops and environmental conditions.

[1508] "Analysis" is the process of using a generative artificial intelligence model to evaluate the content of the received data and determine the condition of the crops and any necessary environmental adjustments.

[1509] "Growing environment" refers to the physical conditions and environmental factors (humidity, temperature, light intensity, etc.) required for crop growth.

[1510] "Feedback" refers to information that is input by the user regarding the results of work performed and improvements to the system's proposals, and is used to improve the system's analysis and proposals from the next time onwards.

[1511] "User" refers to a person or organization that uses this system to grow and manage agricultural crops.

[1512] The system of the present invention is an automated agricultural management system for achieving efficient and sustainable agricultural operations. This system is composed of a camera, sensors, a generative AI model, an information processing device, a terminal, and a user.

[1513] Initial Setup

[1514] server

[1515] During initial setup, the server checks the connectivity of all devices and tests that the cameras and sensors are working properly. Specifically, it takes test images with the cameras and collects initial data (such as temperature and humidity) from the sensors. Next, it provides an interface for users to input information about the crops they are growing. Users enter data such as the type of crop, its growth stage, and specific growing conditions. This information is then retrieved from an agricultural database and stored on the server.

[1516] Data collection

[1517] Terminal (robot)

[1518] The device patrols the farmland, taking images of the crops with its camera. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. This data is sent to a server in real time. For example, the device could take images of the entire farmland at a specific time each day to collect environmental data.

[1519] Data analysis

[1520] server

[1521] The server analyzes the received image data and sensor data using a generative AI model. Image analysis determines the growth status of the crops and checks for disease and pest damage based on leaf color, shape, and size. It also performs environmental assessments based on sensor data to determine whether the current growing environment is appropriate. The generative AI model uses deep learning to improve data accuracy. For example, it can produce analysis results such as, "The leaves of the crops have turned yellow, so there is a high possibility that they are lacking nitrogen fertilizer."

[1522] Providing the optimal environment

[1523] Terminal (robot)

[1524] The device receives instructions from the server and performs operations to optimize the growing environment for crops. For example, if the data measured by the sensor indicates a lack of water, the device will automatically water the crops. It will also apply fertilizer and adjust the lighting as needed. This ensures that crops are grown under optimal conditions.

[1525] Operation suggestions

[1526] server

[1527] Based on the analysis results, the server suggests efficient operation methods to the user. For example, it notifies the user's device of specific instructions such as "crops in a specific area require additional watering." This allows the user to follow the instructions and improve efficiency.

[1528] Collecting feedback

[1529] User

[1530] After carrying out the work based on the suggestions, the user can then provide feedback to the system about the results and areas for improvement. For example, the user can enter specific information such as, "After watering the plant, the leaves suddenly became healthier." This feedback will be used as important data to improve the quality of future analyses and suggestions.

[1531] (Example)

[1532] For example:

[1533] Example prompt:

[1534] Crops are not growing well in the growing environment. Please analyze the data collected by the sensor and suggest the best solution.

[1535] Example result:

[1536] The server uses sensor data to determine if the soil moisture is too low and instructs the device, "Additional watering is required this morning." The device receives instructions from the server and supplies the appropriate amount of water to the specified location. The user then observes changes in growth and provides feedback on the effects.

[1537] This system not only enables efficient and effective crop management, but also enables sustainable agricultural operations.

[1538] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1539] Step 1:

[1540] The server checks the connection of all devices (cameras, sensors, terminals) and tests whether they are working properly. As input, it receives initial data from the cameras and sensors (e.g., test images, temperature, humidity, etc.), and as output, it sends a signal to each device to confirm that the device is working properly.

[1541] Step 2:

[1542] The user inputs information about the crops they are growing using an interface provided by the server. The input details, such as the type of crop, its growth stage, and cultivation conditions, are sent to the server, which then retrieves and stores this information from an agricultural database.

[1543] Step 3:

[1544] The terminal (robot) patrols the farmland, taking images of the crops with a camera. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. It collects crop images and environmental data as input, and sends this data to a server in real time as output.

[1545] Step 4:

[1546] The server receives image data and sensor data sent from the device. The data received as input is sent to a generative AI model, which analyzes it. The output is an evaluation result regarding the growth status of crops and environmental conditions.

[1547] Step 5:

[1548] The server generates optimal growing environment recommendations based on the data analyzed by the generative AI model. It uses the analysis results as input and generates specific growing environment recommendations (e.g., watering amounts, fertilization needs, lighting adjustments, etc.) as output.

[1549] Step 6:

[1550] The server sends the generated cultivation environment proposal to the terminal. It receives the cultivation environment proposal as input and sends instructions as output to the terminal. The terminal performs operations to optimize the crop cultivation environment based on these instructions. Specifically, it automatically waters the crops, applies fertilizer, and adjusts the lights.

[1551] Step 7:

[1552] The server notifies the user of efficient operation methods based on the analysis results. It takes the analysis results and generated suggestions as input and sends notifications about efficient operation methods to the user's device as output. For example, it may instruct the user that "crops in a specific area require additional watering."

[1553] Step 8:

[1554] The user performs the actual work based on the suggestions from the server. The user receives instructions from the server as input and sends the results of the work as feedback to the server as output. For example, the user can input specific information such as "After watering, the leaves of the plant suddenly became healthier."

[1555] Step 9:

[1556] The server receives feedback from users and uses it to improve the quality of the next analysis and proposal. Feedback information is taken in as input, and output is used to improve the generative AI model and increase the accuracy of analysis. This allows the system to continuously evolve, enabling more effective agricultural operations.

[1557] (Application example 1)

[1558] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1559] Modern agriculture requires real-time monitoring of crop growth conditions and the provision of an appropriate growing environment. However, current manual management methods make it difficult to quickly respond to small changes in the growing environment. Furthermore, a lack of technology for efficient agricultural management makes sustainable agricultural operations difficult. Furthermore, managing large areas of farmland requires a great deal of manpower and effort, resulting in high costs.

[1560] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1561] In this invention, the server includes means for photographing crop growth with a camera, means for measuring environmental conditions with a sensor, means for transmitting data from the camera and the sensor to the server, means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal cultivation environment, means for adjusting the cultivation environment based on the analysis results, means for notifying the user of efficient operation methods, means for receiving feedback, means for being mounted on the autonomous agricultural vehicle, and means for transmitting instructions to the autonomous agricultural vehicle. This enables crop growth to be managed in an optimal environment in real time, thereby realizing efficient agricultural management and sustainable agriculture.

[1562] The "camera" is a device that photographs the growth of crops and collects visual data.

[1563] A "sensor" is a device for measuring environmental conditions (such as humidity or temperature).

[1564] A "server" is a central processing unit that receives and analyzes data sent from cameras and sensors.

[1565] A "generative artificial intelligence model" is a machine learning model that analyzes collected data and generates suggestions for growing crops.

[1566] "Optimal growing environment proposals" are proposals based on the analysis results that show the best environmental settings and operation methods to promote crop growth.

[1567] "Means for adjusting the growing environment" refers to devices or methods for actually changing the environmental conditions of agricultural crops (such as the amount of light, water, and temperature) based on the analysis results.

[1568] The "means for notifying the user of an efficient operation method" refers to a method or device for conveying specific operation instructions to the user based on the analysis results.

[1569] "Means for receiving feedback" refers to a method or device for sending back to the system the results of operations performed by the user in accordance with instructions and suggestions for improvement.

[1570] An "autonomous agricultural vehicle" is a vehicle that operates autonomously according to the growth status of crops and environmental conditions, and manages the growing environment.

[1571] "Means for sending instructions to an autonomous agricultural vehicle" refers to a method or device for transmitting specific operational instructions from a server to an autonomous agricultural vehicle.

[1572] The present invention relates to a system for highly automating the cultivation and environmental management of agricultural crops in an autonomous agricultural vehicle. The system for carrying out the present invention is configured as follows.

[1573] 1. Initial Setup

[1574] server

[1575] The server checks the connection of all devices during the initial setup phase, tests whether the cameras and sensors are working properly, provides an interface for users to input information about the crops they are growing, and retrieves and stores the input data from an agricultural database.

[1576] 2. Data Collection

[1577] Autonomous agricultural vehicles

[1578] The autonomous agricultural vehicle travels around the field, taking images of the crops with a camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors, and transmits this data to a server in real time.

[1579] 3. Data Analysis

[1580] server

[1581] The server analyzes the received image data and sensor data using a generative artificial intelligence model. Image analysis determines the growth status of crops and checks for disease and pest damage based on leaf color, shape, and size. Environmental assessment evaluates whether the current growing environment is appropriate based on the acquired sensor data and determines whether any deficient environmental conditions need to be corrected. The analysis results are generated as a proposal, and the next action is decided.

[1582] 4. Providing the optimal environment

[1583] Autonomous agricultural vehicles

[1584] The autonomous agricultural vehicle receives instructions from the server and performs specific operations to optimize the growing environment for crops, such as watering, fertilizing, and adjusting lights as needed, ensuring that crops are grown under optimal growing conditions.

[1585] 5. Operational suggestions

[1586] server

[1587] Based on the analysis results, the server suggests efficient operation methods to the user. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. Specific instructions include information such as "crops in a specific area require additional watering."

[1588] 6. Gathering Feedback

[1589] User

[1590] After carrying out actual work based on the suggestions, users provide feedback to the system on the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. This allows the system to continually evolve, enabling more effective agricultural management.

[1591] Hardware and Software Used

[1592] Hardware: Cameras, humidity sensors, temperature sensors, autonomous agricultural vehicles

[1593] Software: Python, OpenCV (image processing library), requests (HTTP communication library), generative artificial intelligence model using deep learning

[1594] Example prompt

[1595] In a real application, the next action is determined based on the data analysis results and instructions are sent to the vehicle. For example, if data is collected at a humidity of 45% and a temperature of 24°C, the prompt sentence would be as follows:

[1596] "Data Collection: Environmental data was collected at 45% humidity and 24°C. Based on the analysis, area A1 requires additional watering. Instructions: Add 2 liters of water to area A1."

[1597] This will enable optimal environmental management of crops in real time, resulting in efficient agricultural operations.

[1598] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1599] Step 1:

[1600] As an initial setup, the server checks the connection of all devices and checks that the cameras and sensors are working properly. It also obtains information about the crops being grown based on user input, accesses an agricultural database, and stores the necessary data.

[1601] Input: User input of crop information, hardware operating status

[1602] Output: Device connection status report, saved crop data

[1603] Operation: The server checks the device connection and accesses data from the agricultural database

[1604] Step 2:

[1605] The autonomous agricultural vehicle patrols farmland, taking images of crops with a camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors, and transmits the acquired data to a server in real time.

[1606] Input: Farmland environmental conditions, camera and sensor information

[1607] Output: Image data captured, environmental data measured

[1608] Operation: Taking images and measuring data using an autonomous agricultural vehicle, and sending the data to a server

[1609] Step 3:

[1610] The server analyzes the received image data and sensor data using a generative artificial intelligence model. Image analysis determines the growth status of the crops and the presence of diseases and pests. Environmental assessment uses the sensor data to evaluate whether the growing environment is appropriate and determine whether any corrections are necessary.

[1611] Input: Image data, sensor data

[1612] Output: Analysis result report, environmental assessment report

[1613] Operation: Data analysis by server, evaluation of growth status and environment

[1614] Step 4:

[1615] Based on the analysis results, the server instructs the autonomous agricultural vehicle on specific operations to provide an optimal growing environment, such as watering, fertilizing, and adjusting lights as needed.

[1616] Input: Analysis results

[1617] Output: Operation instructions for the autonomous agricultural vehicle

[1618] Operation: Sending instructions from the server to the autonomous agricultural vehicle and executing specific operations

[1619] Step 5:

[1620] Based on the analysis results, the server suggests efficient operation methods to the user. The suggestions are sent to the user's device, and the user then performs the operation. For example, specific instructions such as "crops in a specific area require additional watering" are provided.

[1621] Input: Analysis results, operation suggestions

[1622] Output: Notification of suggested action to user

[1623] Action: Notification from the server to the user device, execution of user operation

[1624] Step 6:

[1625] After the user performs the actual work based on the suggestions, they provide feedback to the system regarding the results and improvements. This feedback is used to improve the accuracy of future analyses and suggestions.

[1626] Input: Results of user actions and improvements

[1627] Output: Feedback data sent to the server

[1628] How it works: User sends feedback, server receives data and improves analysis accuracy

[1629] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1630] The system of the present invention is designed to achieve efficient and sustainable agricultural management. This system is composed of cameras, sensors, generative AI models, servers, terminals (robots), emotion engines, and users. The combination of these components is explained in detail below.

[1631] 1. Initial Setup

[1632] server

[1633] During the initial setup, the server checks the connections of all devices and tests that the cameras, sensors, robots, and emotion engine are working properly. It also provides an interface for users to input information about the crops they are growing, and retrieves and stores the input data from the agricultural database.

[1634] 2. Data Collection

[1635] Terminal (robot)

[1636] The terminal (robot) patrols the farmland, taking pictures of the growing crops with a camera and measuring environmental conditions such as soil humidity, temperature, and light intensity with sensors. The acquired data is sent to a server in real time. In addition, an emotion engine analyzes the user's voice and facial expressions to obtain emotional data.

[1637] 3. Data Analysis

[1638] server

[1639] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. It then analyzes the image data using a generative artificial intelligence model. Specifically, it evaluates the color, shape, and size of the leaves to determine the rate of growth. It also analyzes environmental sensor data to determine whether humidity, temperature, and light intensity are within appropriate ranges. Based on the analysis results, it generates suggestions for the optimal growing environment. It also evaluates the user's stress level based on the user's emotional data obtained by the emotion engine.

[1640] 4. Providing the optimal environment

[1641] Terminal (robot)

[1642] The terminal (robot) receives instructions from the server and performs specific operations to optimize the growing environment for the crops. For example, it waters, fertilizes, adjusts lighting, etc. as needed, ensuring that the crops are grown under optimal conditions.

[1643] 5. Operational suggestions

[1644] server

[1645] The server then suggests efficient operation methods to the user based on the analysis results and the user's emotional data. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. For example, along with specific instructions such as "crops in a specific area require additional watering," the server may provide suggestions such as "your current stress level is high, so we recommend you take a moderate break."

[1646] 6. Gathering Feedback

[1647] User

[1648] After the user performs the actual work based on the suggestions, they provide feedback to the system about the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. At the same time, the emotion engine continuously monitors changes in the user's emotions, and this data is also sent to the server.

[1649] Through these steps, the Auto Farm Navigation system not only streamlines crop cultivation management and supports sustainable agricultural operations, but also reduces the mental burden on users. For example, when users harvest, the system provides appropriate guidance on harvesting areas and adjusts work times according to the user's stress level, making farming more efficient and less stressful.

[1650] The processing flow will be explained below.

[1651] Step 1:

[1652] server

[1653] When the system starts up, the server checks the connection of all devices. It tests the normal operation of the cameras, sensors, robots, and emotion engine, and notifies the user if any abnormalities are detected. Next, the user inputs information about the crops they are growing through the interface, which retrieves and stores related data from the agricultural database.

[1654] Step 2:

[1655] Terminal (robot)

[1656] The terminal (robot) patrols the farmland, taking pictures of the growth of each crop with a camera. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. This data is sent to a server in real time.

[1657] Step 3:

[1658] Terminal (emotion engine)

[1659] The emotion engine analyzes the user's voice and facial expressions to obtain emotional data, which is then immediately sent to the server.

[1660] Step 4:

[1661] server

[1662] The server receives image data and sensor data sent from the terminal (robot), as well as emotion data sent from the emotion engine, and stores them in a database. It then uses a generative AI model to analyze the image data. Specifically, it evaluates the color, shape, and size of the leaves to determine the rate of growth. It also analyzes environmental sensor data to determine whether humidity, temperature, and light intensity are within appropriate ranges. It uses the emotion data to evaluate the user's stress level, and based on all this data, it generates suggestions for the optimal growing environment and operation methods.

[1663] Step 5:

[1664] server

[1665] Based on the generated suggestions, the server generates specific instructions for adjusting the environment. For example, it creates specific instructions such as "Add 500 ml of water to area A" or "Apply fertilizer to area B" and sends them to the terminal (robot). It also generates suggestions based on emotion data, such as "Your current stress level is high, so we recommend you take a moderate break."

[1666] Step 6:

[1667] Terminal (robot)

[1668] The terminal (robot) adjusts the environment based on instructions from the server. For example, it may water or fertilize the appropriate areas. It may also adjust the lighting. Once the adjustment is complete, it reports the results to the server.

[1669] Step 7:

[1670] server

[1671] The server receives the results of the environmental adjustments from the robot and checks whether the suggestions were implemented properly. Based on the crop growth status, the server generates suggestions for further restoration work for the user. Specifically, it may suggest that "harvesting time is approaching in a specific zone." It also suggests ways to reduce stress levels based on emotion data.

[1672] Step 8:

[1673] server

[1674] The suggestions are sent to the user's device, allowing them to understand the necessary operations and timing for breaks. Specific instructions include "crops in a specific area need additional watering" and "your current stress level is high, so please take a moderate break."

[1675] Step 9:

[1676] User

[1677] The user follows the suggestions from the server and performs actual operations, such as harvesting or pest control. Afterwards, the results of the work and any improvements needed on-site are fed back to the system. The emotion engine continues to monitor the user's emotion data and also sends that data to the server.

[1678] Through these processing steps, the Auto Farm Navigation system not only streamlines crop cultivation management and supports sustainable agricultural operations, but also reduces the user's mental burden. For example, when harvesting, the system provides appropriate judgment of the harvesting area and adjusts work time according to the user's stress level, making agricultural work more efficient and less stressful.

[1679] Example 2

[1680] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1681] In modern agriculture, while there is a demand for improving production efficiency and ensuring the quality of agricultural products, there is also a need to reduce the burden of farm work and achieve sustainable operations. Conventional systems have the technology to collect and analyze environmental and growth data, but they lack mechanisms for improving work efficiency and reducing the burden by taking user emotional data into account. This can lead to the accumulation of fatigue and stress, which can lead to reduced productivity. Furthermore, proposals for appropriate growing environments and work methods are limited, resulting in a lack of flexible and comprehensive support for agricultural operations.

[1682] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1683] In this invention, the server includes a means for capturing images of crop growth with a camera, a means for measuring environmental conditions with a sensor, a means for transmitting data from the camera and the sensor to the server, a means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal growing environment, a means for adjusting the growing environment based on the analysis results, a means for informing the user of efficient operation methods, a means for receiving feedback, a means for acquiring and analyzing the user's emotional data with an emotion analysis engine, and a means for evaluating the user's stress level and proposing appropriate working hours based on the user's emotional data. This not only improves the efficiency of agricultural work, but also reduces user stress and provides an appropriate working environment. Furthermore, by comprehensively analyzing the acquired data, maintaining an optimal growing environment, and proposing agricultural work methods, sustainable agricultural operations can be realized.

[1684] A "camera" is a device used to photograph the growth of crops.

[1685] A "sensor" is a device for measuring environmental conditions (humidity, temperature, light intensity, etc.).

[1686] "Server" is a computer system that receives, stores, and analyzes data sent from devices, and generates and provides various proposals.

[1687] A "generative artificial intelligence model" is an AI algorithm that analyzes the transmitted data and generates suggestions for the optimal development environment.

[1688] "Adjusting the growing environment" refers to the operation of adjusting the environment necessary for crop growth based on the analysis results.

[1689] "Efficient operation methods" refer to suggestions and instructions that allow users to manage crops and adjust the environment more effectively.

[1690] "Feedback" refers to information returned to the system regarding the results of the work performed by the user and areas for improvement.

[1691] An "emotion analysis engine" is an algorithm that analyzes a user's voice and facial expressions to obtain and evaluate emotional data.

[1692] "Emotion data" is data that indicates the user's emotional state and stress level.

[1693] "Stress level" is an index that represents the degree of stress that the user is feeling.

[1694] The system of the present invention is designed to achieve efficient and sustainable agricultural management. This system is composed of a combination of multiple hardware and software components, including cameras, sensors, generative AI models, servers, terminals (robots), an emotion analysis engine, and users.

[1695] 1. Initial Setup

[1696] server

[1697] During initial setup, the server checks the connections of all devices and tests that the cameras, sensors, terminals (robots), and sentiment analysis engine are working properly. The server also provides an interface for users to input information about the crops they are growing, and retrieves and stores the input crop data from the agricultural database. For example, if a user is growing tomatoes, the server retrieves the optimal growing conditions for tomatoes from the agricultural database and reflects them in the system.

[1698] 2. Data Collection

[1699] Terminal (robot)

[1700] The terminal (robot) patrols the farmland, using a camera to take pictures of the growing state of the crops. It also uses sensors to measure environmental conditions such as soil humidity, temperature, and light intensity. This data is sent to a server in real time. The terminal (robot) also uses an emotion analysis engine to analyze the user's voice and facial expressions to obtain emotional data about the user. For example, while the user is harvesting, the robot measures the soil humidity and temperature and sends the data to the server.

[1701] 3. Data Analysis

[1702] server

[1703] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. It then uses a generative AI model to analyze the image data. Specifically, it evaluates the color, shape, and size of the leaves and determines the rate of growth. It also analyzes environmental sensor data to determine whether humidity, temperature, and light intensity are within appropriate ranges. It also evaluates the user's stress level based on the user's emotional data obtained by an emotion analysis engine.

[1704] Proposal generation

[1705] Based on the analysis results, the server generates recommendations for the optimal growing environment. For example, it generates specific instructions such as, "The current soil humidity is low, so additional watering is required." It also makes suggestions such as, "We recommend taking moderate breaks" if the user's stress level is high.

[1706] 4. Providing the optimal environment

[1707] Terminal (robot)

[1708] The terminal (robot) receives instructions from the server and performs specific operations to optimize the growing environment for the crops. For example, it performs watering, fertilization, and light adjustments. This ensures that the crops are grown under optimal conditions. If the robot detects a nutrient deficiency in the tomatoes, it can automatically supply the appropriate amount of fertilizer.

[1709] 5. Operational suggestions

[1710] server

[1711] The server then suggests efficient ways to operate the system based on the analysis results and the user's emotional data. The suggestions are sent to the user's device, and the user can improve work efficiency by following the instructions. For example, the server might suggest, "Your current stress level is high, so we recommend you take a moderate break."

[1712] 6. Gathering Feedback

[1713] User

[1714] The user performs the work based on the suggestions and provides feedback to the system with the results and areas for improvement. This feedback is used as important data to improve the quality of future analyses and suggestions. The sentiment analysis engine also continuously monitors changes in the user's emotions, and this data is also sent to the server. If a user provides feedback such as "The additional fertilizer has helped the tomatoes grow better," this will be reflected in future fertilizer management.

[1715] Specific examples of prompts to input to generative AI models

[1716] 1. "Evaluate the current growth status of your tomato plants based on the color and shape of their leaves."

[1717] 2. "Based on recent sensor data, determine if soil moisture and temperature are adequate."

[1718] 3. "Please analyze the user's current stress level from their voice data."

[1719] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1720] Step 1: Performing initial setup

[1721] Server: Check device connection

[1722] Input: Connection status of the onboard camera, sensors, terminal (robot), and emotion analysis engine

[1723] Processing: The server checks whether each device is properly connected and tests whether it is working properly. For example, it checks whether a camera is connected to the network and can transmit images in real time. It also checks the operation status of sensors and robots.

[1724] Output: Device connection status and operation check results

[1725] Specific operations: Test whether the camera can capture images properly, whether the sensor can measure data accurately, and whether the robot can operate correctly. If there is an abnormality, the user will be notified.

[1726] Server: Provides the user interface

[1727] Input: Information about the crops the user grows

[1728] Processing: The server provides a web interface for inputting information about the crops to be grown. When a user inputs the tomato variety and growing conditions, the server retrieves the relevant data from an agricultural database and stores them.

[1729] Output: Crop information and growing conditions data

[1730] Specific operation: The user inputs tomato cultivation information through the interface, and the server stores it in the database.

[1731] Step 2: Data collection

[1732] Terminal (robot): Photographing crops

[1733] Input: Crop growth status

[1734] Processing: The terminal (robot) periodically patrols the farmland and uses a camera to take pictures of the growing conditions of the crops.

[1735] Output: Captured image data

[1736] What it does: The robot patrols farmland and uses cameras to collect detailed images, such as capturing leaf color, shape, and size.

[1737] Terminal (robot): Measurement of environmental data

[1738] Input: soil moisture, temperature, light intensity

[1739] Processing: The terminal (robot) uses sensors during its patrol to measure soil humidity, temperature, and light intensity.

[1740] Output: Measured environmental data

[1741] Specific operation: The robot uses a humidity sensor to measure the moisture content of the soil and a temperature sensor to measure the environmental temperature.

[1742] Terminal (robot): Acquire emotion data

[1743] Input: User's voice and facial expressions

[1744] Processing: The terminal (robot) uses an emotion analysis engine to analyze the user's voice and facial expressions and obtain emotional data.

[1745] Output: Obtained emotion data

[1746] Specific operation: While the user is working, the robot collects and analyzes voice and facial expression data.

[1747] Terminal (robot): Sending data

[1748] Input: photographed image data, measured environmental data, acquired emotion data

[1749] Processing: The terminal (robot) sends these data to the server in real time.

[1750] Output: Data sent to server completed

[1751] Specific operation: The terminal (robot) uses wireless communication to send all collected data to the server.

[1752] Step 3: Data analysis

[1753] Server: Receives and stores data

[1754] Input: Transmitted image data, environmental data, emotional data

[1755] Processing: The server receives the data sent from the terminal (robot) and stores it in a database.

[1756] Output: Data stored in the database

[1757] Specific operation: The server stores all received data in a database in the appropriate format.

[1758] Server: Image data analysis

[1759] Input: Saved image data

[1760] Processing: The server uses a generative artificial intelligence model to analyze the image data, assessing leaf color, shape, and size to determine growth rate.

[1761] Output: Growth assessment results based on image data

[1762] Specific operation: Image analysis AI analyzes the color and shape of tomato leaves and measures their growth status.

[1763] Server: Sensor data analysis

[1764] Input: Stored environmental data

[1765] Processing: The server analyzes the environmental sensor data sent and evaluates whether the humidity, temperature, and light level are within appropriate ranges.

[1766] Output: Environmental condition assessment results

[1767] Specific operation: The data analysis system analyzes humidity, temperature, and light intensity data to determine whether the growing conditions are appropriate.

[1768] Server: Proposal Generation

[1769] Input: Image data growth assessment results, environmental condition assessment results

[1770] Processing: The server generates a proposal for the optimal growing environment based on the results of analyzing image data and sensor data.

[1771] Output: Proposed development environment

[1772] Specific action: The server generates specific instructions such as "Current soil moisture is low and additional watering is required."

[1773] Server: Emotion data analysis

[1774] Input: Acquired emotion data

[1775] Processing: The server evaluates the user's stress level based on data from the emotion analysis engine.

[1776] Output: User's stress level assessment result

[1777] Specific operation: Emotion analysis AI analyzes the user's stress level and, if necessary, makes suggestions such as "We recommend you take a moderate break."

[1778] Step 4: Providing an optimal environment

[1779] Terminal (robot): Receives operation instructions

[1780] Input: Instructions from the server

[1781] Processing: The terminal (robot) receives the operation instructions sent from the server.

[1782] Output: Actions based on received instructions

[1783] Specific action: The robot receives the instruction to "water the tomato trellis."

[1784] Terminal (robot): Performing environmental adjustments

[1785] Input: Specific operation instructions from the server

[1786] Processing: The terminal (robot) adjusts the environment according to the received instructions.

[1787] Output: Applied environmental adjustments

[1788] Specific actions: The robot waters and fertilizes the plants, as well as adjusts the lights and controls the temperature.

[1789] Step 5: Suggested operation

[1790] Server: Proposal Notification

[1791] Input: Analysis results, emotion data

[1792] Processing: The server proposes efficient operation methods to the user based on the analysis results and the user's emotional data.

[1793] Output: User is notified of the suggestion

[1794] Specific operation: The server notifies the user via the terminal with specific suggestions such as "Tomatoes in a specific area need additional watering."

[1795] User: Operate based on instructions

[1796] Input: Operation instructions from the server

[1797] Processing: The user follows instructions from the server to manage crops and adjust the environment.

[1798] Output: The result of the operation performed

[1799] Specific operation: The user receives instructions from the server and manually waters the plants, adds fertilizer, adjusts the lights, etc.

[1800] Step 6: Gather feedback

[1801] User: Feedback on work results

[1802] Input: Result of the operation performed

[1803] Processing: The user performs actual work based on the suggestions and provides feedback to the system on the results and improvements.

[1804] Output: Feedback data to the system

[1805] Specific action: The user comments, "The tomatoes are growing better thanks to the additional fertilizer," and enters this into the system.

[1806] Server: Update emotion data

[1807] Input: Emotional data after the user's task

[1808] Processing: The sentiment analysis engine continuously monitors the user's emotional changes and sends the data to the server.

[1809] Output: Updated emotion data

[1810] Specific operation: Emotion analysis AI analyzes the user's emotional data and evaluates changes in stress levels after work.

[1811] (Application example 2)

[1812] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1813] Conventional agricultural management systems primarily monitor crop growth and environmental conditions, but do not address efficient operations in industrial manufacturing sites. Furthermore, there was a lack of a system that simultaneously optimized the quality control of the manufacturing process and the health of workers, creating a challenge to balance the efficiency of factory operations with worker health management.

[1814] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1815] In this invention, the server includes means for photographing crop growth with a camera, means for measuring environmental conditions with a sensor, means for transmitting data from the camera and the sensor to the server, means for analyzing the data with a generative artificial intelligence model and generating a proposal for an optimal cultivation environment, means for adjusting the cultivation environment based on the analysis results, means for notifying the user of efficient operation methods, means for receiving feedback, and means for monitoring the production process and the health of workers and providing optimal operations. This makes it possible to achieve both quality control of the production process and health management of workers.

[1816] A "camera" is a device that photographs an object and acquires its image data.

[1817] "Crop" refers to cultivated plants, especially those grown in agriculture.

[1818] "Growth" refers to the process by which crops grow and develop over time.

[1819] A "sensor" is a device that detects a physical or environmental condition and outputs it as an electrical signal.

[1820] "Environmental conditions" refers to factors that affect crops and the working environment, such as temperature, humidity, and light intensity.

[1821] "Server" refers to a central computer system for receiving data, analyzing data, and sending instructions.

[1822] A "generative artificial intelligence model" is a machine learning model that analyzes data based on specific algorithms and generates new suggestions and predictions.

[1823] "Analysis" refers to the process of processing acquired data and images to extract specific information and patterns.

[1824] "Growing environment" refers to the environmental conditions that promote optimal growth and production of agricultural crops and manufacturing processes.

[1825] "Adjustment" refers to the act of changing the conditions of a machine or environment to an optimal state in order to achieve a specific purpose.

[1826] "How to use" refers to documentation or interfaces that provide procedures or techniques for users to use a system to perform specific tasks.

[1827] "Users" refer to people who interact with the system and act on its instructions and suggestions.

[1828] "Notification" refers to the act or function of informing a user of specific information.

[1829] "Feedback" refers to results and opinions collected from users, and is information that is used to improve the performance and functionality of the system.

[1830] "Manufacturing process" refers to the series of operations and steps that create a product using raw materials.

[1831] "Health status" refers to the state of a worker's physical and mental health, including an assessment of the worker's condition in the working environment.

[1832] "Optimization" refers to adjusting a system or environment to its best state in order to achieve a specific purpose.

[1833] The system of the present invention is designed to optimize the manufacturing process and worker health in a factory, supporting efficient operations. This system is composed of cameras, sensors, a generative AI model, a server, a terminal (robot), an emotion engine, and a user. The combination of these components is described in detail below.

[1834] 1. Initial Setup

[1835] server

[1836] The server first checks whether all devices (cameras, sensors, robots, and emotion engines) are operating normally. The camera used here is a Logitech C920 HD Pro Webcam, and the temperature and humidity sensor is a DHT11. Next, it inputs initial data about the factory's manufacturing process and stores it in a database. The server then uses a web framework such as Flask or Django to build an API and establish communication with the devices.

[1837] 2. Data Collection

[1838] Terminal (robot)

[1839] The terminal (robot) patrols the factory, checking product quality with a camera and measuring environmental conditions (temperature, humidity, etc.) with sensors. It is equipped with an emotion engine and acquires emotional data from the facial expressions and voices of workers. This data is sent to the server in real time.

[1840] 3. Data Analysis

[1841] server

[1842] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database. A generative AI model using TensorFlow analyzes the image data and evaluates the quality of the product. It also analyzes environmental sensor data to determine whether temperature and humidity are within appropriate ranges. It also simultaneously evaluates the user's emotional data obtained by the emotion engine and measures the user's stress level.

[1843] 4. Providing the optimal environment

[1844] Terminal (robot)

[1845] Based on the analysis results, the server sends specific instructions to the terminal (robot), such as adjusting the lighting, adjusting the temperature and humidity, or suggesting that the worker take a break.

[1846] 5. Operational suggestions

[1847] server

[1848] The server then informs the user of the optimal operation method based on the analysis results, for example, providing specific instructions such as "Additional inspection of products in a specific area is required" along with suggestions such as "Your current stress level is high, so we recommend you take a moderate break."

[1849] 6. Gathering Feedback

[1850] User

[1851] The user performs actual tasks based on the suggestions and provides feedback on the results and improvements to the system. This feedback is used to improve the quality of future analyses and suggestions. At the same time, the emotion engine continuously monitors changes in the user's emotions, and this data is also sent to the server.

[1852] For example, a robot monitors the production line with a camera and measures environmental conditions in real time using a DHT11 sensor. It also captures the user's (worker's) face with a camera and analyzes the facial expression data using an emotion engine. The results are sent to a server, where a generative AI model predicts and suggests optimal operations and improvements. Specific instructions, such as "make the lighting a little brighter," "add more moisture to the product," or "encourage workers to take breaks," can be obtained.

[1853] Example prompt sentence:

[1854] "Detect scratches and defects on products from images"

[1855] "Please evaluate how these environmental conditions (humidity, temperature) affect the quality of the product."

[1856] "Assess workers' stress levels and suggest break needs."

[1857] This system simultaneously improves the efficiency of the manufacturing process and manages the health of workers.

[1858] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1859] Step 1:

[1860] The server checks whether all devices (cameras, sensors, robots, emotion engines) are working properly.

[1861] Input: Device connection status

[1862] Output: Device connection check result

[1863] How it works: The server pings each device and checks if there is a response. It uses Flask or Django to check the device connection status via an API.

[1864] Step 2:

[1865] The server inputs initial data about the factory's manufacturing process and stores it in a database.

[1866] Input: Initial data about the manufacturing process

[1867] Output: Initial data stored in the database

[1868] Operation: The server receives manufacturing process information entered through a user interface and stores it in a database, including information such as product specifications and process sequences.

[1869] Step 3:

[1870] The terminal (robot) patrols the factory, checking product quality with cameras and measuring environmental conditions with sensors.

[1871] Input: Camera footage, environmental data measured by sensors (temperature, humidity, etc.)

[1872] Output: Photographed image data, measured environmental data

[1873] Operation: The terminal (robot) moves autonomously within the factory, taking images of products using a camera and measuring environmental conditions such as temperature and humidity using sensors.

[1874] Step 4:

[1875] The terminal (robot) uses an emotion engine to obtain emotional data from the worker's facial expressions and voice.

[1876] Input: Worker's face video and voice

[1877] Output: Parsed emotion data

[1878] How it works: The device captures the worker's face with a camera and collects their voice with a microphone. It then uses an emotion engine to analyze this data and generate emotion data.

[1879] Step 5:

[1880] The server receives image data and sensor data sent from the terminal (robot) and stores it in a database.

[1881] Input: Image data, sensor data

[1882] Output: Raw data stored in a database

[1883] Operation: The server receives data sent from the terminal using a communication protocol and stores it in a database.

[1884] Step 6:

[1885] The server analyzes image data using a generative artificial intelligence model to evaluate product quality, as well as environmental sensor data and emotion data.

[1886] Input: Image data, environmental sensor data, emotion data

[1887] Output: Analysis results (quality evaluation, environment evaluation, emotion evaluation)

[1888] How it works: The server uses TensorFlow to analyze image data and input prompts (e.g., "Detect scratches and defects on the product from the image"), as well as environmental data such as temperature and humidity (e.g., "Evaluate how these environmental conditions affect the product's quality") and emotional data (e.g., "Evaluate the worker's stress level and suggest the need for a break").

[1889] Step 7:

[1890] Based on the analysis results, the server sends specific operation instructions to the terminal (robot).

[1891] Input: Analysis results

[1892] Output: Instructions to the terminal

[1893] Operation: Based on the analysis results, the server generates specific operational instructions, such as "brighten the lights" or "adjust the temperature and humidity," and sends them to the terminal.

[1894] Step 8:

[1895] The terminal (robot) performs specific operations on-site based on operation instructions from the server.

[1896] Input: Operation instructions from the server

[1897] Output: The result of the operation performed

[1898] Operation: The terminal follows instructions from the server and performs operations such as "adjusting lighting," "increasing humidity," and "notifying workers to take a break."

[1899] Step 9:

[1900] Based on the analysis results, the server notifies the user of efficient operation methods.

[1901] Input: Analysis results

[1902] Output: A message to inform the user

[1903] How it works: The server analyzes the results and sends specific instructions to the user's device, such as "Products in a specific area require additional testing," and suggestions, such as "We recommend taking appropriate breaks."

[1904] Step 10:

[1905] The user performs actual work based on the suggestions and provides feedback on the results and improvements to the system.

[1906] Input: User feedback data

[1907] Output: accumulation and analysis of feedback data

[1908] How it works: The user follows the suggestions and reports the results to the system. This allows the system to improve the quality of its analysis and suggestions in the future. The emotion engine also continuously monitors the user's emotional changes and sends the data to the server.

[1909] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1910] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1911] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1912] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1913] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1914] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1915] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1916] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1917] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1918] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1919] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1920] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1921] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 ...

Claims

1. A means of photographing the growth of crops with a camera; means for measuring environmental conditions with sensors; means for transmitting data from said camera and sensor to a server; A means for analyzing the data using a generative artificial intelligence model and generating a proposal for an optimal development environment; A means for adjusting the growing environment based on the analysis results; A means for informing the user of an efficient operation method; a means of receiving feedback; and A system including:

2. The system of claim 1 , wherein the generative artificial intelligence model analyzes image data using deep learning.

3. The system according to claim 1, wherein specific environmental adjustment instructions are generated for a specific area based on the analysis results.

Images