System

Abstract

A system is provided.SOLUTION: A system including means for acquiring image data of a plant or soil using a camera mounted on a smart device, means for communicating with a server including artificial intelligence for analyzing the acquired image data and receiving an analysis result, means for feeding back a growth state of the plant or presence or absence of a pest to a user based on the received analysis result, and software means for providing a growth prediction of the plant and an optimal coping method.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention aims to provide an environment in which individuals can easily start a home vegetable garden, thereby popularizing agriculture and increasing the number of people engaged in agriculture. Traditional agriculture requires high capital investment and technical knowledge, making it difficult for newcomers to enter the industry. Furthermore, the image of agriculture is fixed, making it difficult for young people and beginners to actively participate. Furthermore, appropriate responses to climate change and natural disasters are required, making it important to select optimal crops and predict their growth, but these are difficult for individuals to accomplish. [Means for solving the problem]

[0005] This invention provides a system that uses a camera mounted on a smart device to acquire image data of plants and soil, communicates with a server equipped with artificial intelligence that analyzes the acquired image data, and receives the analysis results. Based on the received analysis results, the system provides feedback to the user on the plant's growth status and the presence of pests, and provides plant growth predictions and optimal countermeasures through software means. The system also includes a sensor that collects data on the plant's cultivation environment, and transmits the collected data to the server, enabling detailed analysis. The system also promotes user engagement by providing plant growth predictions and cultivation missions in a game-like manner. This allows individuals to easily utilize advanced agricultural techniques, promoting the spread of agriculture and sustainable crop cultivation.

[0006] A "smart device" is an electronic device with advanced functions that include internet connectivity and sensors, and responds to user operations.

[0007] A "camera" is a device that detects light and converts it into image data, and is used in smart devices to obtain real-time visual information.

[0008] "Image data" is visual information captured using a photographing device such as a camera, expressed in digital form.

[0009] An "artificial intelligence server" is a networked computer system that has programs that automatically perform specific tasks using technologies such as machine learning and deep learning.

[0010] "Analysis results" refer to the information and insights obtained when artificial intelligence processes input data, and serve as guidelines for decisions and actions based on the user's goals.

[0011] "Feedback" is the process by which a system notifies a user of analysis results or other information, providing the user with information to act upon.

[0012] "Software means" is a set of instructions written as a computer program and executed to implement a particular function.

[0013] A "sensor" is a device that detects physical or chemical changes and converts them into electrical signals, and is used to collect environmental data.

[0014] "Cultivation environment data" refers to data that includes information related to plant cultivation, such as temperature, humidity, and pH concentration.

[0015] "Growth prediction" is the process of estimating future growth based on the current state of a plant and environmental data.

[0016] The "best solution" is the most effective way to promote plant growth and solve the problem.

[0017] "Game-like" is an experience that incorporates elements that allow users to complete tasks while having fun.

[0018] A "cultivation mission" is a task that includes specific farming tasks and goals that the user must accomplish. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention is a system that uses a camera mounted on a smart device to acquire image data of plants and soil, communicates with a server equipped with artificial intelligence that analyzes the image data, and receives the analysis results. Specific embodiments of this system and its program processing are described below.

[0041] System Overview

[0042] The system is based on the premise that users wear a smart device (e.g., smart glasses or a hands-free phone) and use it to manage their home garden. A camera installed in the smart device captures image data of the soil and plants, which it then sends to a server for analysis. The server analyzes the image data, determines the growth status and any problems, and provides feedback. In addition, sensors are used to capture cultivation environment data (temperature, humidity, pH level, etc.), which is also analyzed by the server, enabling even more detailed feedback to be provided.

[0043] Program processing explanation

[0044] Acquisition and transmission of image data

[0045] The device uses the smart device's camera to capture real-time image data of soil and plants. For example, when a user takes a photo of a tomato leaf, the device captures the image data.

[0046] The device then sends the acquired image data to the server. This data includes not only the image itself but also information about the environment at the time of shooting (time, location, etc.).

[0047] Image analysis and feedback

[0048] The server receives the image data and analyzes it using an AI model installed inside it. The image analysis process detects the plant's growth status, the presence of pests, and signs of disease.

[0049] The server receives the analysis results and generates a report on the current status and the best course of action based on the results. For example, it may provide specific advice such as, "There are pests on the tomato leaves, so they need to be removed immediately."

[0050] The device receives the analysis results sent from the server and notifies the user in a variety of ways, including voice, screen display, and vibration, in the form that is easiest for the user to understand.

[0051] Environmental data acquisition and analysis

[0052] The device periodically acquires environmental data such as temperature, humidity, and pH level using soil sensors and external sensors, and transmits this data to the server in real time.

[0053] The server then analyzes the received environmental data to predict crop growth and determine optimal treatment options. This analysis takes into account the seasons, weather, and soil characteristics, supporting long-term cultivation planning.

[0054] User-friendly game-like app

[0055] The application installed on the device is designed to make managing the vegetable garden fun for users. For example, the app encourages user involvement by presenting challenges such as "This week's mission: Increase your tomato harvest!"

[0056] The app evaluates the user's progress and achievement based on their behavioral history and cultivation data. For example, it may motivate them by saying, "You've exceeded your tomato harvest goal. Now, try growing eggplants!"

[0057] Specific examples

[0058] 1. A user wears smart glasses and takes an image of a tomato leaf.

[0059] 2. The device sends image data and environmental data to the server.

[0060] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[0061] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[0062] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[0063] 6. The user follows the app's instructions to remove the pests and carry out any subsequent maintenance work.

[0064] This system allows users to effectively manage their home gardens without requiring advanced technology or large amounts of capital, which is expected to have social benefits such as promoting agriculture, revitalizing local communities, and increasing food supplies.

[0065] The processing flow will be explained below.

[0066] Step 1:

[0067] The user puts on the smart device and starts the application. The smart device automatically initializes the camera and sensors and is ready for use.

[0068] Step 2:

[0069] The device uses the smart device's camera to take real-time images of plants and soil, for example, a user captures an image of a tomato leaf.

[0070] Step 3:

[0071] The device sends the captured image data to the server. This data includes not only the image data but also environmental data (such as time and location) at the time of capture.

[0072] Step 4:

[0073] The server inputs the received image data into an AI model and begins analysis, which detects the plant's growth status, the presence or absence of pests, signs of disease, and more.

[0074] Step 5:

[0075] The server generates the analysis results and sends them to the device. These results include the current state of the plants and the optimal course of action. For example, the results may include a message such as, "Pests have been found on tomato leaves. Please take measures to remove them."

[0076] Step 6:

[0077] The device will then notify the user of the analysis results received from the server. Notification methods vary, including voice, screen display, and vibration, and are provided in the form that is easiest for the user to understand.

[0078] Step 7:

[0079] The device uses soil sensors and external sensors to collect environmental data such as temperature, humidity, and pH level, which is then sent to a server in real time.

[0080] Step 8:

[0081] The server receives environmental data and then analyzes it to provide detailed plant growth predictions and optimal countermeasures. The analysis results also include information on long-term cultivation plans.

[0082] Step 9:

[0083] The device will notify the user of growth forecasts and advice on how to deal with the situation. For example, it will give advice such as, "There will be a lot of rain this week, so please refrain from watering."

[0084] Step 10:

[0085] The application installed on the device provides users with game-like missions, such as "This week's mission: Increase your tomato harvest!"

[0086] Step 11:

[0087] The application guides the user to perform specific farming tasks, such as removing pests from tomato leaves and then adding the appropriate fertilizer.

[0088] Step 12:

[0089] The terminals periodically collect data and send it to a server, which then monitors agricultural data for the entire region, enabling the analysis of performance and problems across the region and the proposal of improvement measures.

[0090] These are the specific processing steps of the system, which enables users to effectively manage their home gardens without requiring advanced technology or large amounts of capital.

[0091] Example 1

[0092] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] Conventional home garden management systems require users to visually check the condition of plants and soil and take manual action, which requires advanced technology and specialized knowledge. Furthermore, it is difficult to quickly respond to changes in the cultivation environment, resulting in problems such as slow plant growth and increased damage from diseases and pests. Furthermore, it is difficult for users to maintain an interest in garden management, which often results in inappropriate management.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0095] In this invention, the server includes: means for acquiring image data of plants and soil using a camera mounted on a smart device; means for transmitting the acquired image data together with environmental information at the time of capture to the server; means for analyzing the received image data using an artificial intelligence model; means for providing feedback to the user on the plant's growth status, the presence of pests, and signs of disease based on the analysis results; means for generating and notifying the user of reports on optimal treatment methods and the plant's condition; means for acquiring temperature, humidity, and pH data using an environmental sensor and transmitting the data to the server; and software means for analyzing the received environmental data and providing plant growth predictions and optimal cultivation methods. This allows users to efficiently and effectively manage their home gardens without specialized knowledge. Furthermore, the software means for providing a game-like cultivation management experience helps maintain user involvement.

[0096] A "smart device" is an electronic device that can be worn or carried by a user and is equipped with a camera and sensors.

[0097] A "camera" is a photographing device for capturing images and videos, and in the present invention is installed in a smart device.

[0098] "Image data" refers to digitized visual information captured by a camera, and is data used to evaluate the condition of plants and soil through analysis.

[0099] "Environment information" is context data such as the time and location at the time of shooting, and is transmitted to the server together with the image data.

[0100] A "server" is a computer system that receives, analyzes, stores, and transmits data over a network.

[0101] An "artificial intelligence model" refers to an algorithm that has been trained to perform analysis and predictions based on input data.

[0102] "Feedback" refers to providing information or advice generated based on the analysis results to the user.

[0103] A "sensor" is a device that detects physical environmental information (e.g., temperature, humidity, pH concentration, etc.) and outputs it as digital data.

[0104] "Growth prediction" refers to predicting the future growth state of a plant based on past and current data.

[0105] "Cultivation methods" refer to the procedures and treatments required for plants to grow healthily in optimal conditions.

[0106] A "report" is a document or digital file that summarizes the analysis results and feedback information.

[0107] "User" refers to an individual who uses this system to manage a home garden.

[0108] The present invention is a system that acquires image data of plants and soil using a camera mounted on a smart device, communicates with a server equipped with artificial intelligence that analyzes the image data, and receives the analysis results. Specific embodiments of the present invention will be described below.

[0109] System Overview

[0110] The system aims to enable users to manage their home vegetable gardens using a smart device (e.g., smart glasses or a hands-free phone). The smart device's built-in camera captures image data of the soil and plants, which is then sent to a server for analysis. The server analyzes the image data, determines the plant's growth status and any problems, and provides feedback. Environmental sensors are also used to capture cultivation environment data (temperature, humidity, pH concentration, etc.), which is also analyzed by the server to provide more detailed feedback.

[0111] Hardware and software used

[0112] Smart device: Has the ability to connect a camera to acquire image data and environmental sensors.

[0113] Server: Equipped with artificial intelligence models (e.g., TensorFlow or PyTorch) for analyzing data.

[0114] Environmental sensors: DHT22 (temperature and humidity sensor), pH meter, etc. are used.

[0115] Image data acquisition and analysis

[0116] The device uses the smart device's camera to acquire real-time image data of plants and soil. For example, when a user takes a photo of a tomato leaf, the camera captures the image in high resolution. The acquired image data is then sent to the server along with environmental information at the time of the photo (e.g., photo time, location, etc.).

[0117] The server then analyzes the received image data using an artificial intelligence model. During the analysis process, features within the image are extracted to detect the plant's growth status, the presence of pests, signs of disease, and so on.

[0118] Analysis result feedback

[0119] Based on the analysis results, the server generates a report on the optimal method of dealing with the problem and the condition of the plant. For example, if there are pests on tomato leaves, it generates specific advice such as, "There are pests on the tomato leaves. Please remove them immediately."

[0120] The analysis results are sent to the device and provided to the user in a variety of ways, including voice notification, screen display, and vibration alert, allowing the user to take appropriate action quickly.

[0121] Environmental data acquisition and analysis

[0122] The device periodically collects environmental data such as temperature, humidity, and pH level using soil sensors and external sensors, and transmits this data to the server in real time.

[0123] The server analyzes the received environmental data and proposes optimal cultivation methods and forecasts for plant growth. This analysis takes into account the season, weather, and soil characteristics, and also provides support for long-term cultivation planning.

[0124] Game-like application

[0125] The application installed on the device is designed to make managing a home vegetable garden fun for users. For example, it encourages user involvement by presenting challenges such as, "This week's mission: Increase your tomato harvest!" The app also evaluates the user's progress and achievement based on their behavioral history and cultivation data, and motivates them by saying, "Your tomato harvest exceeded your goal. Let's try growing eggplants next!"

[0126] Examples of specific examples and prompts

[0127] 1. A user wears smart glasses and takes an image of a tomato leaf.

[0128] 2. The device sends image data and environmental data to the server.

[0129] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[0130] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[0131] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[0132] 6. The user follows the app's instructions to remove the pests and carry out any subsequent maintenance work.

[0133] Prompt Sentence Examples

[0134] "Take an image of a tomato leaf and analyze it to detect whether there are any pests on the leaf."

[0135] "Please suggest specific measures to take if there are pests on tomato leaves."

[0136] "Predict plant growth based on temperature, humidity, and pH data."

[0137] In this way, the present invention allows users to efficiently and effectively manage their home gardens without requiring advanced skills or specialized knowledge. Furthermore, the software means that allows users to manage cultivation in a game-like manner can keep users engaged.

[0138] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0139] Step 1: Acquiring image data

[0140] The device uses the smart device's camera to capture image data of plants and soil. When a user wears smart glasses and takes a picture of a tomato leaf, the camera captures the image in high resolution. The input is the video from the camera attached to the smart glasses. The output is the captured image data.

[0141] Step 2: Sending image data

[0142] The device sends the image data it has acquired along with environmental information at the time of shooting (such as the time of shooting and location information) to the server. The input is the image data and environmental information acquired in step 1. Specifically, the data is transferred securely using the HTTPS protocol. The output is the image data and environmental information received by the server.

[0143] Step 3: Analyzing the image data

[0144] The image data received by the server is analyzed using an internal artificial intelligence model (e.g., using TensorFlow or PyTorch). The input is the image data and environmental information sent in step 2. Features within the image are extracted to detect the plant's growth status, the presence or absence of pests, signs of disease, etc. The output is the analysis results.

[0145] Step 4: Generate analysis results

[0146] The server generates feedback (report) to provide to the user based on the analysis results. The input is the analysis results from step 3. Specifically, it generates advice such as "There are pests on the tomato leaves. Please remove them immediately" based on the plant's growth status and any problems. The output is the generated feedback report.

[0147] Step 5: Notification of analysis results

[0148] The device receives the analysis results sent from the server and notifies the user. The input is the feedback report generated in step 4. Notification methods include voice notification, screen display, and vibration alert. The output is the notification information received by the user.

[0149] Step 6: Get environment data

[0150] The terminal uses soil sensors and external sensors to acquire environmental data such as temperature, humidity, and pH concentration. The input is real-time environmental data acquired from the sensors. The output is the acquired environmental data.

[0151] Step 7: Sending environment data

[0152] The terminal transmits the environmental data acquired to the server. The input is the environmental data acquired in step 6. Specifically, the MQTT protocol is used to efficiently transfer the data. The output is the environmental data received by the server.

[0153] Step 8: Analyze environmental data

[0154] The server analyzes the received environmental data and makes plant growth predictions and proposes optimal cultivation methods. The input is the environmental data sent in step 7. Regression analysis and machine learning algorithms are used to analyze data trends and predict future growth. The output is the analysis results and proposals.

[0155] Step 9: User-responsive application behavior

[0156] The application installed on the device presents the user with home garden management tasks (e.g., "This week's mission: Increase your tomato harvest!") and evaluates their progress and achievement. The inputs are the analysis results from Steps 4 and 8, as well as the user's behavioral history and cultivation data. The output is the task presented to the user and its evaluation results.

[0157] Through this series of processing steps, users can efficiently and effectively manage their home gardens without needing advanced skills or specialized knowledge.

[0158] (Application example 1)

[0159] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0160] Conventional home garden management systems have the drawback of requiring users to visit the site in person to check the status of the plants, which is time-consuming and labor-intensive. Furthermore, accurate assessment of plant growth and the presence of pests requires specialized knowledge, making it difficult for beginners. Furthermore, when learning and involvement are required, real-world experiments are required, increasing the risk of failure.

[0161] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0162] In this invention, the server includes means for acquiring image data of plants and soil using a camera mounted on a smart device, means for communicating with the server equipped with artificial intelligence that analyzes the acquired image data and receiving the analysis results, means for providing feedback to the user on the plant growth status and the presence or absence of pests based on the received analysis results, software means for predicting plant growth and providing optimal countermeasures, and software means for allowing the user to manage and learn about their home garden in a virtual environment. This allows the user to effectively manage and learn about their home garden in a virtual environment, saving time and effort and reducing the risk of failure.

[0163] A "smart device" is an electronic device equipped with a camera, sensor, and communication function, and capable of acquiring and transmitting image data and environmental data.

[0164] "Camera" means a photographic device that can capture image data and store or transmit it as electronic data.

[0165] "Plants" are green plants grown in home gardens and agricultural activities.

[0166] "Soil" is the natural material containing organic matter and minerals that serves as a substrate for plant growth.

[0167] "Image data" refers to visual information captured by a camera and stored electronically.

[0168] "Artificial intelligence" is a technology that enables computer systems to analyze data and assist in problem-solving and decision-making.

[0169] A "server" is a computer system that processes data over a network and provides analytical results to other devices.

[0170] "Communication" is the process and means for sending and receiving data.

[0171] "Analysis results" are the detection results and judgment results of data analyzed by artificial intelligence.

[0172] "Feedback" is the process of providing analysis results and recommendations to users.

[0173] The "growth status of the plant" refers to the health and developmental progress of the plant during its cultivation.

[0174] "Pests" are insects or other harmful organisms that cause damage to plants.

[0175] "Best practices" are specific actions or processes recommended to promote plant growth and solve the problem.

[0176] "Software means" means means for executing computer programs and providing specific functions or services.

[0177] A "virtual environment" is a fictitious space or scenario generated by a computer system.

[0178] A "home garden" is a small-scale farm where individuals cultivate food within their homes or in their local neighborhoods.

[0179] "Management" is the process of monitoring and caring for plants to promote their healthy growth.

[0180] "Learning" is the activity of a user acquiring and understanding information in order to improve their knowledge or skills.

[0181] System Overview

[0182] The system of the present invention consists of a smart device, a server, and a user. The user uses a smart device such as smart glasses or a head-mounted display to acquire image data of plants and soil and transmits the data to the server. The server analyzes the acquired data, determines the plant's growth status and the presence of pests, and provides feedback. The smart device also acquires environmental data (temperature, humidity, pH concentration, etc.) and transmits this to the server, enabling more detailed analysis and feedback.

[0183] Image data acquisition and analysis process

[0184] Image data of plants and soil is acquired using a camera mounted on a smart device. The user wears smart glasses and takes images of plants in a virtual environment. Image data and environmental data are then captured in real time and sent to a server.

[0185] The server uses its built-in artificial intelligence model to analyze the received image data. The image analysis process detects the plant's growth status, the presence of pests, signs of disease, etc. It also analyzes environmental data (temperature, humidity, pH level, etc.) and generates the optimal countermeasures based on this.

[0186] Providing feedback

[0187] The analysis results are sent from the server to the smart device and notified to the user. This notification is given in the form of voice, screen display, vibration, etc., and is provided in the most understandable way for the user. For example, specific advice such as "There are pests on the tomato leaves. Please remove them" is provided.

[0188] Virtual plant management

[0189] Furthermore, the system allows users to manage and learn about their home gardens in a virtual environment. For example, when a user wears smart glasses and checks the quality of tomato leaves in a virtual sunroom, the user captures an image, and the server analyzes it to determine whether there are any pests and provides feedback.

[0190] Hardware and software used

[0191] Hardware: smart glasses, head-mounted displays, cameras, sensors

[0192] Software: Python, OpenCV, Requests, Artificial Intelligence Models

[0193] Specific examples

[0194] Example: A user uses smart glasses to take a photo of the leaves of a tomato plant they are growing in a virtual sunroom, and the system analyzes whether or not there are any pests and notifies the user.

[0195] Example prompt: Please provide a description of an application that takes pictures of virtual plants, analyzes their growth status, and provides appropriate measures. Required information includes image data of the virtual plants, and environmental data such as temperature, humidity, and pH level.

[0196] This system allows users to effectively manage and learn about home gardens in a virtual environment without visiting the site, saving time and effort. Even beginners can learn proper cultivation methods without requiring specialized knowledge.

[0197] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0198] Step 1:

[0199] The user photographs the plants they are growing in the virtual sunroom using smart glasses or a head-mounted display. The input is image data of the plants and soil captured by the smart device's camera. For data processing, the smart device captures the image data in real time and adds environmental data (time, location, etc.). The output is the captured image data and accompanying environmental data.

[0200] Step 2:

[0201] The terminal transmits the acquired image data and environmental data to the server via the Internet. The input is the image data and environmental data captured in the previous step. As a data calculation, the terminal converts these data into an appropriate format and transmits it. The output is the data transmitted to the server.

[0202] Step 3:

[0203] The server analyzes the received image data. At this stage, the input is the image data and environmental data sent from the device. The server uses a generative AI model to analyze the plant's growth status, the presence or absence of pests, signs of disease, etc. For data calculation, image analysis algorithms are used to extract various information and identify problems. The output is feedback data containing the analysis results.

[0204] Step 4:

[0205] The server generates feedback for the user based on the analysis results. The input is various information obtained through image analysis (e.g., plant health, presence of pests, environmental conditions). For data processing, the server compiles this information and creates a report in an easy-to-understand format for the user. The output is a feedback message sent to the user.

[0206] Step 5:

[0207] The terminal notifies the user of the feedback received from the server. At this stage, the input is the feedback message sent from the server. As a data computation, the terminal presents the feedback message to the user in the form of text, sound, vibration, etc. The output is the user receiving the feedback information.

[0208] Step 6:

[0209] The user takes specific action based on the feedback. The input is the feedback message notified by the terminal. As data processing, the user adjusts the plant management method based on the feedback. The output is that appropriate management is carried out and the health of the plant is maintained or improved.

[0210] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0211] The present invention is a home garden management system that combines an emotion engine that recognizes the user's emotions. This system uses cameras and sensors installed in smart devices to collect plant and soil data, and links with a server that analyzes the data to provide the analysis results as feedback to the user. It also recognizes the user's emotional state and provides feedback and advice based on that emotion. A specific embodiment of this system and its program processing are described below.

[0212] System Overview

[0213] The system is based on the premise that users wear a smart device (e.g., smart glasses or a hands-free phone) and use it to manage their home garden. A camera installed in the smart device captures image data of the soil and plants, which it then sends to a server for analysis. The server analyzes the image data, determines the growth status and any problems, and provides feedback. In addition, sensors are used to capture cultivation environment data (temperature, humidity, pH level, etc.), which is also analyzed by the server, enabling even more detailed feedback to be provided.

[0214] Furthermore, by incorporating an emotion engine, the system can recognize the user's emotional state and provide feedback and advice based on that emotion, allowing the user to perform farm work more appropriately and with greater satisfaction.

[0215] Program processing explanation

[0216] Acquisition and transmission of image data

[0217] The terminal uses the smart device's camera to capture real-time image data of soil and plants. For example, a user captures an image of a tomato leaf.

[0218] The device then sends the acquired image data to the server. This data includes not only the image itself but also information about the environment at the time of shooting (time, location, etc.).

[0219] Image analysis and feedback

[0220] The server inputs the received image data into an AI model and begins analysis, which detects the plant's growth status, the presence or absence of pests, signs of disease, and more.

[0221] The server receives the analysis results and generates a report on the current status and the best course of action based on the results. For example, it may provide specific advice such as, "There are pests on the tomato leaves, so they need to be removed immediately."

[0222] The device receives the analysis results sent from the server and notifies the user in a variety of ways, including voice, screen display, and vibration, in the form that is easiest for the user to understand.

[0223] Environmental data acquisition and analysis

[0224] The device periodically acquires environmental data such as temperature, humidity, and pH level using soil sensors and external sensors, and transmits this data to the server in real time.

[0225] The server then analyzes the received environmental data to predict crop growth and determine optimal treatment options. This analysis takes into account the seasons, weather, and soil characteristics, supporting long-term cultivation planning.

[0226] User Emotion Recognition and Feedback

[0227] The device uses its built-in emotion engine to analyze the user's facial expressions and voice to recognize their emotional state. For example, if the user is feeling frustrated while working, the device can detect that emotion.

[0228] The server generates feedback and advice based on the emotion analysis results sent from the emotion engine. For example, if the user is feeling stressed, the server may provide advice such as "It would be good to take a short break."

[0229] The device notifies the user based on their emotions, allowing the user to respond optimally according to their emotional state.

[0230] Specific examples

[0231] 1. A user wears smart glasses and takes an image of a tomato leaf.

[0232] 2. The device sends image data and environmental data to the server.

[0233] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[0234] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[0235] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[0236] 6. The device uses the emotion engine to analyze the user's emotional state and detects that the user is dissatisfied.

[0237] 7. Based on the emotion analysis results, the server generates advice such as "It might be a good idea to take a short break" and sends it to the device.

[0238] 8. The device provides emotion-based advice to the user.

[0239] 9. The user follows the app's instructions to remove the pests and carry out any subsequent maintenance work.

[0240] This system allows users to effectively manage their home gardens without requiring advanced technology or large amounts of capital, and it also improves work efficiency and satisfaction through feedback based on users' emotions.

[0241] The processing flow will be explained below.

[0242] Step 1:

[0243] The user puts on the smart device and starts the application. The smart device automatically initializes the camera and sensors and is ready for use.

[0244] Step 2:

[0245] The device uses the smart device's camera to take real-time images of plants and soil, for example, a user captures an image of a tomato leaf.

[0246] Step 3:

[0247] The device sends the captured image data to the server. This data includes not only the image data but also environmental information (such as time and location) at the time of capture.

[0248] Step 4:

[0249] The server inputs the received image data into an AI model and begins analysis, which detects the plant's growth status, the presence or absence of pests, signs of disease, and more.

[0250] Step 5:

[0251] The server generates the analysis results and sends them to the device. These results include the current state of the plants and the optimal course of action. For example, the results may include a message such as, "Pests have been found on tomato leaves. Please take measures to remove them."

[0252] Step 6:

[0253] The device will then notify the user of the analysis results received from the server. Notification methods vary, including voice, screen display, and vibration, and are provided in the form that is easiest for the user to understand.

[0254] Step 7:

[0255] The device uses soil sensors and external sensors to collect environmental data such as temperature, humidity, and pH level, which is then sent to a server in real time.

[0256] Step 8:

[0257] The server then analyzes the received environmental data to provide detailed plant growth predictions and optimal countermeasures, including information on long-term cultivation plans.

[0258] Step 9:

[0259] The device will notify the user of growth forecasts and advice on how to deal with the situation. For example, it will give advice such as, "There will be a lot of rain this week, so please refrain from watering."

[0260] Step 10:

[0261] The application installed on the device provides users with game-like missions, such as "This week's mission: Increase your tomato harvest!"

[0262] Step 11:

[0263] The application guides the user to perform specific farming tasks, such as removing pests from tomato leaves and then adding the appropriate fertilizer.

[0264] Step 12:

[0265] The device uses an emotion engine to analyze the user's facial expressions and voice to recognize their emotional state. For example, if the user is feeling frustrated while working, that emotion can be detected.

[0266] Step 13:

[0267] The server generates feedback and advice based on the user's emotions based on the emotion analysis results sent from the emotion engine. For example, if the user is feeling stressed, the server generates advice such as "It would be good to take a short break."

[0268] Step 14:

[0269] The device will notify the user based on their emotions. For example, the device will tell the user, "It would be good to take a short break," by displaying a message on the screen or by voice.

[0270] Step 15:

[0271] The terminals periodically collect data and send it to a server, which then monitors agricultural data for the entire region, enabling the analysis of performance and problems across the region and the proposal of improvement measures.

[0272] Example 2

[0273] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0274] Conventional home garden management systems only collect and analyze plant growth and environmental data, but are indifferent to the user's emotional state. As a result, they do not take into account the user's emotions or stress levels and are unable to provide appropriate feedback or advice, resulting in problems that reduce user satisfaction and work efficiency. Furthermore, conventional systems often lack real-time data analysis and feedback, making it difficult to quickly address plant problems.

[0275] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0276] In this invention, the server includes means for acquiring image data of plants and soil using a camera mounted on the smart device, means for communicating with the server equipped with artificial intelligence that analyzes the acquired image data and receiving the analysis results, means for incorporating an emotion engine that recognizes the user's emotional state and provides feedback and advice based on the recognition results, means for providing feedback to the user on the plant's growth status and the presence or absence of pests based on the image data and environmental data, and software means for providing plant growth predictions and optimal countermeasures. This allows the user to not only respond appropriately to the plant's growth status and cultivation environment, but also receive feedback and advice based on their own emotional state, enabling high satisfaction and efficient home vegetable garden management.

[0277] A "smart device" is an electronic device equipped with a camera and sensors that can be operated directly or worn by the user.

[0278] A "camera" is an optical device for acquiring image data.

[0279] "Image data" is digital data representing visual information captured by a camera.

[0280] "Artificial intelligence" refers to software technology that automatically analyzes data and makes decisions.

[0281] A "server" is a computer system used to analyze data and provide information.

[0282] "Analysis results" refers to the information obtained after artificial intelligence analyzes image data and environmental data.

[0283] "Emotional state" refers to the user's psychological and emotional state, and is recognized from data such as facial expressions and voice.

[0284] The "emotion engine" is a system that analyzes the user's emotional state and reflects the results in feedback and advice.

[0285] "Feedback" refers to information that conveys analysis results and advice to the user.

[0286] "Environmental data" refers to data on the plant's growing environment, such as temperature, humidity, and pH level.

[0287] A "sensor" is a device for collecting environmental data.

[0288] "Growth prediction" refers to information that predicts the future growth state of a plant.

[0289] The "optimal countermeasure" is a countermeasure proposed based on the analysis results and growth forecasts.

[0290] "Software means" refers to a computer program for realizing a specific function.

[0291] MODE FOR CARRYING OUT THE INVENTION

[0292] This invention is a home vegetable garden management system that combines an emotion engine that recognizes the user's emotions. This system uses cameras and sensors installed in smart devices to collect plant and soil data, and connects to a server that analyzes the data to provide the analysis results as feedback to the user. It also recognizes the user's emotional state and provides feedback and advice according to that emotion.

[0293] The system is based on the premise that users wear a smart device (e.g., smart glasses or a hands-free phone) and use it to manage their home garden. The camera on the smart device captures image data of the soil and plants and sends the data to a server for analysis.

[0294] The server uses a generative AI model (e.g., TensorFlow) to analyze the received image data. This model is trained to detect plant growth status, the presence of pests, signs of disease, etc. Based on the analysis results, it generates a detailed report on the current status and optimal measures. The analysis results are fed back to the smart device user in real time.

[0295] The device then periodically collects cultivation environment data (temperature, humidity, pH level, etc.) using sensors. This data is also sent to the server in real time and used for environmental data analysis. The server then uses the environmental data to make growth predictions and provide feedback on long-term cultivation plans and specific countermeasures.

[0296] Furthermore, the system is equipped with a user emotion engine. The device detects the user's facial expressions and voice and analyzes their emotional state. For example, if the user says "I'm tired," that information is sent to the server. The server generates optimal advice for the user based on the emotion analysis results. For example, if the user is feeling stressed, feedback such as "It would be good to take a short break" is generated and notified to the user via the device.

[0297] Specific examples

[0298] For example, you can use the system by following these steps:

[0299] 1. A user wears smart glasses and takes an image of a tomato leaf.

[0300] 2. The device sends image data and environmental data to the server.

[0301] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[0302] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[0303] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[0304] 6. The device uses the emotion engine to analyze the user's emotional state and detects that the user is dissatisfied.

[0305] 7. Based on the emotion analysis results, the server generates advice such as "It might be a good idea to take a short break" and sends it to the device.

[0306] 8. The device provides emotion-based advice to the user.

[0307] Prompt Sentence Examples

[0308] "Use this home garden management system to check whether there are any pests on your tomato leaves. Also, analyze the user's emotional state to provide appropriate feedback."

[0309] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0310] Step 1:

[0311] The terminal acquires soil and plant image data in real time using the smart device's camera. The input is image data captured through the smart device's camera. The output is high-resolution image data stored on the terminal and environmental information at the time of capture (e.g., GPS location information, timestamp). In concrete terms, a user uses smart glasses to capture an image of a tomato leaf.

[0312] Step 2:

[0313] The device sends the acquired image data to the server. The input is the image data and environmental information captured in step 1. The output is the data sent to the server. Specifically, the device transfers data to the server via a REST API using asynchronous communication.

[0314] Step 3:

[0315] The image data received by the server is input into a generative AI model for analysis. The input is the image data and environmental information sent to the server. The output is the analysis results, such as the plant's growth status, the presence or absence of pests, and signs of disease. Specifically, the server performs image analysis using a generative AI model such as TensorFlow and stores the results in a database.

[0316] Step 4:

[0317] The server generates feedback based on the analysis results. The input is the analysis result from step 3. The output is feedback information to the user. Specifically, the server generates specific advice such as "There are pests on the tomato leaves, so please remove them immediately."

[0318] Step 5:

[0319] The device notifies the user of the analysis results sent from the server. The input is the feedback information sent from the server. The output is the notification received by the user. Specifically, the device notifies the user in multiple ways, such as by voice, on-screen display, or vibration.

[0320] Step 6:

[0321] The device periodically obtains environmental data (temperature, humidity, pH concentration, etc.) using soil sensors and external sensors. The input is real-time data obtained from the sensors. The output is environmental data that is temporarily stored on the device. Specifically, the device obtains temperature and humidity data from the sensors every 30 minutes and stores it in the cache.

[0322] Step 7:

[0323] The terminal sends the acquired environmental data to the server. The input is the environmental data acquired in step 6. The output is the environmental data sent to the server. Specifically, the data is transferred to the server using real-time streaming technology (e.g., Kafka).

[0324] Step 8:

[0325] The server analyzes the received environmental data. The input is the environmental data sent to the server. The output is the environmental analysis results, which include growth predictions and countermeasures. Specifically, the server uses the environmental data to run growth prediction algorithms and simulations, and stores the results in a database.

[0326] Step 9:

[0327] The server generates feedback based on the results of the environmental analysis. The input is the environmental analysis result from step 8. The output is feedback information based on the environment to the user. Specifically, the server generates advice such as "The temperature is too high, so it would be a good idea to use the shade during the day" and sends it to the device.

[0328] Step 10:

[0329] The device uses an emotion engine to analyze the user's facial expressions and voice and recognize the user's emotional state. The input is the user's facial expression and voice data. The output is the recognized emotional state data. Specifically, the device's emotion engine detects a user statement such as "I'm tired" and sends the emotional data to the server.

[0330] Step 11:

[0331] The server generates feedback based on the emotion analysis results. The input is the emotional state data obtained in step 10. The output is feedback information based on the user's emotions. Specifically, the server generates advice such as "It might be a good idea to take a short break" and sends it to the device.

[0332] Step 12:

[0333] The device notifies the user of emotion-based feedback. The input is emotion-based feedback information sent from the server. The output is the notification received by the user. Specifically, the device notifies the user by voice, saying, "It might be a good idea to take a short break."

[0334] (Application example 2)

[0335] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0336] Conventional home garden management systems and factory quality control systems have limited means for appropriately monitoring the condition of plants and products. Furthermore, no systems existed that took into account the emotional state of workers, creating challenges in terms of work efficiency and satisfaction. This created a need for a system that could accurately grasp the growth status of plants and provide feedback based on workers' emotions.

[0337] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0338] In this invention, the server includes means for acquiring image data of plants and soil using a camera mounted on the smart device, means for communicating with the server equipped with artificial intelligence that analyzes the acquired image data and receiving the analysis results, means for providing feedback to the user on the plant growth status and the presence or absence of pests based on the received analysis results, software means for providing plant growth predictions and optimal countermeasures, and means for analyzing the emotional state of the worker using an emotion engine mounted on the smart device and providing feedback and advice based on the emotion. This makes it possible to analyze the state of the plants with high accuracy and provide feedback that takes the worker's emotion into consideration, thereby improving work efficiency and satisfaction.

[0339] A "smart device" is an advanced information terminal equipped with a camera, sensors, etc., that enables data acquisition and communication.

[0340] A "camera" is a device that captures optical images as electronic data.

[0341] "Soil" refers to the ground that serves as the base for plant growth, and is generally a natural substance that contains nutrients and moisture.

[0342] "Image data" is an electronic representation of visual information captured by a camera.

[0343] "Artificial intelligence" refers to computer systems that simulate human intelligence and have the ability to automatically analyze and solve specific problems.

[0344] A "server" is a computer system that stores and processes data over a network.

[0345] "User" refers to a person who uses this system and manages or works on plants.

[0346] "Feedback" is the process of notifying the user of analysis results and advice.

[0347] "Software means" is a computer program designed to perform a specific function.

[0348] An "emotion engine" is a technology that analyzes and judges a user's emotional state from facial expressions, voice, etc.

[0349] "Workers" refers to those involved in quality control and product manufacturing within the factory.

[0350] An "emotional state" refers to a temporary psychological state that an individual has, such as stress or satisfaction.

[0351] "Feedback and advice" refers to specific instructions or suggestions provided to a user or worker.

[0352] "Plant growth prediction" is the process of predicting the future growth state of a plant based on collected data.

[0353] The "best solution" is the most effective action or measure proposed to solve the current problem.

[0354] MODE FOR CARRYING OUT THE INVENTION

[0355] This invention relates to a home vegetable garden management system using smart devices and a server, and a quality control system using factory robots. This system uses cameras and sensors to acquire plant and product data, analyzes the data, and provides feedback to users. It also analyzes the emotional state of workers and provides advice based on that data, improving work efficiency and satisfaction.

[0356] System configuration

[0357] This system mainly consists of the following components:

[0358] Smart Devices

[0359] The smart glass is equipped with a camera, microphone, and temperature and humidity sensors, which allow it to capture plant and product data, as well as environmental data and the emotional state of workers.

[0360] server

[0361] The server has advanced analytical capabilities and analyzes the acquired data using artificial intelligence models (TensorFlow, OpenCV, etc.), and feeds back the results of image analysis, environmental data analysis, and emotion analysis to the smart device.

[0362] Software Means

[0363] Dedicated software for analysis and feedback is installed, which manages the entire process of receiving, analyzing, and providing feedback on data.

[0364] Specific Examples

[0365] A specific example will be described below.

[0366] Image data acquisition and analysis

[0367] A camera installed in the smart glass captures image data of plants (for example, tomato leaves in a home garden) or products (products in a factory). The captured data is sent in real time to a server, which then inputs it into an artificial intelligence model and begins analysis. The analysis results include the plant's growth status, the presence or absence of pests, or abnormalities in the product's quality. Feedback is sent from the server to the smart device, and the user receives specific advice.

[0368] Environmental data acquisition and analysis

[0369] Temperature and humidity sensors installed in the smart glass periodically collect data on the plant cultivation environment and the factory environment. This data is sent to a server, which analyzes it according to the season, weather, and the characteristics of the working environment. As a result, plant growth forecasts and optimal treatment methods are provided to the user.

[0370] Emotional state analysis and feedback

[0371] The smart device's built-in emotion engine captures the worker's facial expressions and voice data. This data is sent to a server in real time, and the server uses an emotion analysis model to analyze the worker's emotional state. For example, if a worker is feeling stressed, the system will detect that emotion and provide advice such as "take a short break."

[0372] Prompt Sentence Examples

[0373] Below are examples of specific prompt statements that are executed by this system.

[0374] Text format

[0375] Data Acquisition

[0376] 1. Taking images of plants and products with smart glass.

[0377] 2. Obtain real-time temperature and humidity data from environmental sensors.

[0378] 3. Photograph the worker's facial expressions and analyze their emotions.

[0379] Data analysis

[0380] 1. Image data is sent to a server and analyzed using an artificial intelligence model.

[0381] 2. Environmental data is sent to a server and analyzed according to seasons and weather conditions.

[0382] 3. The emotion data is sent to the server and analyzed using the emotion analysis model.

[0383] Providing feedback

[0384] 1. Provide feedback to the user based on the analysis results.

[0385] 2. Provide appropriate advice based on the worker's emotional state.

[0386] This system analyzes the condition of plants and products with high accuracy and provides feedback that takes into account the emotions of workers, thereby improving work efficiency and satisfaction.

[0387] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0388] Step 1:

[0389] Capture visual images using optical cameras and other devices from smart glasses

[0390] The camera installed in the device (smart glass) captures image data of plants and products, while environmental data such as temperature and humidity are simultaneously collected by sensors.

[0391] Input: Plant images, environmental sensor data (temperature, humidity)

[0392] Output: Captured image data and sensor data

[0393] Step 2:

[0394] Captured image data and sensor data are sent to a cloud server

[0395] The device transmits the acquired image data and environmental data to the server in real time.

[0396] Input: Image data and sensor data

[0397] Output: Data sent to the server

[0398] Step 3:

[0399] Analyze data using cloud-based AI models

[0400] The server inputs the received image data into an artificial intelligence model (e.g., TensorFlow) to analyze the plant's health and product quality abnormalities. Environmental data is also analyzed in a similar manner.

[0401] Input: Data received by the server

[0402] Output: Detected growth status, presence or absence of pests, or quality abnormalities

[0403] Step 4:

[0404] Create work guidelines and recommendations based on the analysis results

[0405] Based on the analysis results, the server generates feedback including plant growth predictions and optimal countermeasures, as well as suggestions for improving quality at the factory.

[0406] Input: Detection results

[0407] Output: Feedback and suggestions

[0408] Step 5:

[0409] Analyze worker emotions using captured facial expressions and voice

[0410] The terminal captures the worker's facial expression and voice data and sends it to the emotion engine, where the server analyzes the worker's emotional state using an emotion analysis model.

[0411] Input: facial expression and voice data

[0412] Output: Emotional status (stress, satisfaction, etc.)

[0413] Step 6:

[0414] Generate feedback and recommendations based on sentiment data

[0415] Based on the emotional data, the server generates feedback appropriate to the worker's psychological state and suggested actions, such as "take a break."

[0416] Input: Emotional status

[0417] Output: Emotion-based advice and recommendations

[0418] Step 7:

[0419] Send all generated feedback to the device

[0420] The server sends the final feedback, growth forecast, countermeasures, and advice to the worker to the terminal and notifies the user.

[0421] Input: Generated feedback data

[0422] Output: Notification information to the device

[0423] Step 8:

[0424] Users act on feedback

[0425] The user takes necessary measures and actions based on the notifications and advice provided by the device.

[0426] Input: Notification information from the device

[0427] Output: The measures or actions taken

[0428] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0429] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0430] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0431] [Second embodiment]

[0432] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0433] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0434] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0435] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0436] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0437] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0438] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0439] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0440] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0441] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0442] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0443] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0444] The present invention is a system that uses a camera mounted on a smart device to acquire image data of plants and soil, communicates with a server equipped with artificial intelligence that analyzes the image data, and receives the analysis results. Specific embodiments of this system and its program processing are described below.

[0445] System Overview

[0446] The system is based on the premise that users wear a smart device (e.g., smart glasses or a hands-free phone) and use it to manage their home garden. A camera installed in the smart device captures image data of the soil and plants, which it then sends to a server for analysis. The server analyzes the image data, determines the growth status and any problems, and provides feedback. In addition, sensors are used to capture cultivation environment data (temperature, humidity, pH level, etc.), which is also analyzed by the server, enabling even more detailed feedback to be provided.

[0447] Program processing explanation

[0448] Acquisition and transmission of image data

[0449] The device uses the smart device's camera to capture real-time image data of soil and plants. For example, when a user takes a photo of a tomato leaf, the device captures the image data.

[0450] The device then sends the acquired image data to the server. This data includes not only the image itself but also information about the environment at the time of shooting (time, location, etc.).

[0451] Image analysis and feedback

[0452] The server receives the image data and analyzes it using an AI model installed inside it. The image analysis process detects the plant's growth status, the presence of pests, and signs of disease.

[0453] The server receives the analysis results and generates a report on the current status and the best course of action based on the results. For example, it may provide specific advice such as, "There are pests on the tomato leaves, so they need to be removed immediately."

[0454] The device receives the analysis results sent from the server and notifies the user in a variety of ways, including voice, screen display, and vibration, in the form that is easiest for the user to understand.

[0455] Environmental data acquisition and analysis

[0456] The device periodically acquires environmental data such as temperature, humidity, and pH level using soil sensors and external sensors, and transmits this data to the server in real time.

[0457] The server then analyzes the received environmental data to predict crop growth and determine optimal treatment options. This analysis takes into account the seasons, weather, and soil characteristics, supporting long-term cultivation planning.

[0458] User-friendly game-like app

[0459] The application installed on the device is designed to make managing the vegetable garden fun for users. For example, the app encourages user involvement by presenting challenges such as "This week's mission: Increase your tomato harvest!"

[0460] The app evaluates the user's progress and achievement based on their behavioral history and cultivation data. For example, it may motivate them by saying, "You've exceeded your tomato harvest goal. Now, try growing eggplants!"

[0461] Specific examples

[0462] 1. A user wears smart glasses and takes an image of a tomato leaf.

[0463] 2. The device sends image data and environmental data to the server.

[0464] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[0465] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[0466] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[0467] 6. The user follows the app's instructions to remove the pests and carry out any subsequent maintenance work.

[0468] This system allows users to effectively manage their home gardens without requiring advanced technology or large amounts of capital, which is expected to have social benefits such as promoting agriculture, revitalizing local communities, and increasing food supplies.

[0469] The processing flow will be explained below.

[0470] Step 1:

[0471] The user puts on the smart device and starts the application. The smart device automatically initializes the camera and sensors and is ready for use.

[0472] Step 2:

[0473] The device uses the smart device's camera to take real-time images of plants and soil, for example, a user captures an image of a tomato leaf.

[0474] Step 3:

[0475] The device sends the captured image data to the server. This data includes not only the image data but also environmental data (such as time and location) at the time of capture.

[0476] Step 4:

[0477] The server inputs the received image data into an AI model and begins analysis, which detects the plant's growth status, the presence or absence of pests, signs of disease, and more.

[0478] Step 5:

[0479] The server generates the analysis results and sends them to the device. These results include the current state of the plants and the optimal course of action. For example, the results may include a message such as, "Pests have been found on tomato leaves. Please take measures to remove them."

[0480] Step 6:

[0481] The device will then notify the user of the analysis results received from the server. Notification methods vary, including voice, screen display, and vibration, and are provided in the form that is easiest for the user to understand.

[0482] Step 7:

[0483] The device uses soil sensors and external sensors to collect environmental data such as temperature, humidity, and pH level, which is then sent to a server in real time.

[0484] Step 8:

[0485] The server receives environmental data and then analyzes it to provide detailed plant growth predictions and optimal countermeasures. The analysis results also include information on long-term cultivation plans.

[0486] Step 9:

[0487] The device will notify the user of growth forecasts and advice on how to deal with the situation. For example, it will give advice such as, "There will be a lot of rain this week, so please refrain from watering."

[0488] Step 10:

[0489] The application installed on the device provides users with game-like missions, such as "This week's mission: Increase your tomato harvest!"

[0490] Step 11:

[0491] The application guides the user to perform specific farming tasks, such as removing pests from tomato leaves and then adding the appropriate fertilizer.

[0492] Step 12:

[0493] The terminals periodically collect data and send it to a server, which then monitors agricultural data for the entire region, enabling the analysis of performance and problems across the region and the proposal of improvement measures.

[0494] These are the specific processing steps of the system, which enables users to effectively manage their home gardens without requiring advanced technology or large amounts of capital.

[0495] Example 1

[0496] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0497] Conventional home garden management systems require users to visually check the condition of plants and soil and take manual action, which requires advanced technology and specialized knowledge. Furthermore, it is difficult to quickly respond to changes in the cultivation environment, resulting in problems such as slow plant growth and increased damage from diseases and pests. Furthermore, it is difficult for users to maintain an interest in garden management, which often results in inappropriate management.

[0498] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0499] In this invention, the server includes: means for acquiring image data of plants and soil using a camera mounted on a smart device; means for transmitting the acquired image data together with environmental information at the time of capture to the server; means for analyzing the received image data using an artificial intelligence model; means for providing feedback to the user on the plant's growth status, the presence of pests, and signs of disease based on the analysis results; means for generating and notifying the user of reports on optimal treatment methods and the plant's condition; means for acquiring temperature, humidity, and pH data using an environmental sensor and transmitting the data to the server; and software means for analyzing the received environmental data and providing plant growth predictions and optimal cultivation methods. This allows users to efficiently and effectively manage their home gardens without specialized knowledge. Furthermore, the software means for providing a game-like cultivation management experience helps maintain user involvement.

[0500] A "smart device" is an electronic device that can be worn or carried by a user and is equipped with a camera and sensors.

[0501] A "camera" is a photographing device for capturing images and videos, and in the present invention is installed in a smart device.

[0502] "Image data" refers to digitized visual information captured by a camera, and is data used to evaluate the condition of plants and soil through analysis.

[0503] "Environment information" is context data such as the time and location at the time of shooting, and is transmitted to the server together with the image data.

[0504] A "server" is a computer system that receives, analyzes, stores, and transmits data over a network.

[0505] An "artificial intelligence model" refers to an algorithm that has been trained to perform analysis and predictions based on input data.

[0506] "Feedback" refers to providing information or advice generated based on the analysis results to the user.

[0507] A "sensor" is a device that detects physical environmental information (e.g., temperature, humidity, pH concentration, etc.) and outputs it as digital data.

[0508] "Growth prediction" refers to predicting the future growth state of a plant based on past and current data.

[0509] "Cultivation methods" refer to the procedures and treatments required for plants to grow healthily in optimal conditions.

[0510] A "report" is a document or digital file that summarizes the analysis results and feedback information.

[0511] "User" refers to an individual who uses this system to manage a home garden.

[0512] The present invention is a system that acquires image data of plants and soil using a camera mounted on a smart device, communicates with a server equipped with artificial intelligence that analyzes the image data, and receives the analysis results. Specific embodiments of the present invention will be described below.

[0513] System Overview

[0514] The system aims to enable users to manage their home vegetable gardens using a smart device (e.g., smart glasses or a hands-free phone). The smart device's built-in camera captures image data of the soil and plants, which is then sent to a server for analysis. The server analyzes the image data, determines the plant's growth status and any problems, and provides feedback. Environmental sensors are also used to capture cultivation environment data (temperature, humidity, pH concentration, etc.), which is also analyzed by the server to provide more detailed feedback.

[0515] Hardware and software used

[0516] Smart device: Has the ability to connect a camera to acquire image data and environmental sensors.

[0517] Server: Equipped with artificial intelligence models (e.g., TensorFlow or PyTorch) for analyzing data.

[0518] Environmental sensors: DHT22 (temperature and humidity sensor), pH meter, etc. are used.

[0519] Image data acquisition and analysis

[0520] The device uses the smart device's camera to acquire real-time image data of plants and soil. For example, when a user takes a photo of a tomato leaf, the camera captures the image in high resolution. The acquired image data is then sent to the server along with environmental information at the time of the photo (e.g., photo time, location, etc.).

[0521] The server then analyzes the received image data using an artificial intelligence model. During the analysis process, features within the image are extracted to detect the plant's growth status, the presence of pests, signs of disease, and so on.

[0522] Analysis result feedback

[0523] Based on the analysis results, the server generates a report on the optimal method of dealing with the problem and the condition of the plant. For example, if there are pests on tomato leaves, it generates specific advice such as, "There are pests on the tomato leaves. Please remove them immediately."

[0524] The analysis results are sent to the device and provided to the user in a variety of ways, including voice notification, screen display, and vibration alert, allowing the user to take appropriate action quickly.

[0525] Environmental data acquisition and analysis

[0526] The device periodically collects environmental data such as temperature, humidity, and pH level using soil sensors and external sensors, and transmits this data to the server in real time.

[0527] The server analyzes the received environmental data and proposes optimal cultivation methods and forecasts for plant growth. This analysis takes into account the season, weather, and soil characteristics, and also provides support for long-term cultivation planning.

[0528] Game-like application

[0529] The application installed on the device is designed to make managing a home vegetable garden fun for users. For example, it encourages user involvement by presenting challenges such as, "This week's mission: Increase your tomato harvest!" The app also evaluates the user's progress and achievement based on their behavioral history and cultivation data, and motivates them by saying, "Your tomato harvest exceeded your goal. Let's try growing eggplants next!"

[0530] Examples of specific examples and prompts

[0531] 1. A user wears smart glasses and takes an image of a tomato leaf.

[0532] 2. The device sends image data and environmental data to the server.

[0533] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[0534] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[0535] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[0536] 6. The user follows the app's instructions to remove the pests and carry out any subsequent maintenance work.

[0537] Prompt Sentence Examples

[0538] "Take an image of a tomato leaf and analyze it to detect whether there are any pests on the leaf."

[0539] "Please suggest specific measures to take if there are pests on tomato leaves."

[0540] "Predict plant growth based on temperature, humidity, and pH data."

[0541] In this way, the present invention allows users to efficiently and effectively manage their home gardens without requiring advanced skills or specialized knowledge. Furthermore, the software means that allows users to manage cultivation in a game-like manner can keep users engaged.

[0542] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0543] Step 1: Acquiring image data

[0544] The device uses the smart device's camera to capture image data of plants and soil. When a user wears smart glasses and takes a picture of a tomato leaf, the camera captures the image in high resolution. The input is the video from the camera attached to the smart glasses. The output is the captured image data.

[0545] Step 2: Sending image data

[0546] The device sends the image data it has acquired along with environmental information at the time of shooting (such as the time of shooting and location information) to the server. The input is the image data and environmental information acquired in step 1. Specifically, the data is transferred securely using the HTTPS protocol. The output is the image data and environmental information received by the server.

[0547] Step 3: Analyzing the image data

[0548] The image data received by the server is analyzed using an internal artificial intelligence model (e.g., using TensorFlow or PyTorch). The input is the image data and environmental information sent in step 2. Features within the image are extracted to detect the plant's growth status, the presence or absence of pests, signs of disease, etc. The output is the analysis results.

[0549] Step 4: Generate analysis results

[0550] The server generates feedback (report) to provide to the user based on the analysis results. The input is the analysis results from step 3. Specifically, it generates advice such as "There are pests on the tomato leaves. Please remove them immediately" based on the plant's growth status and any problems. The output is the generated feedback report.

[0551] Step 5: Notification of analysis results

[0552] The device receives the analysis results sent from the server and notifies the user. The input is the feedback report generated in step 4. Notification methods include voice notification, screen display, and vibration alert. The output is the notification information received by the user.

[0553] Step 6: Get environment data

[0554] The terminal uses soil sensors and external sensors to acquire environmental data such as temperature, humidity, and pH concentration. The input is real-time environmental data acquired from the sensors. The output is the acquired environmental data.

[0555] Step 7: Sending environment data

[0556] The terminal transmits the environmental data acquired to the server. The input is the environmental data acquired in step 6. Specifically, the MQTT protocol is used to efficiently transfer the data. The output is the environmental data received by the server.

[0557] Step 8: Analyze environmental data

[0558] The server analyzes the received environmental data and makes plant growth predictions and proposes optimal cultivation methods. The input is the environmental data sent in step 7. Regression analysis and machine learning algorithms are used to analyze data trends and predict future growth. The output is the analysis results and proposals.

[0559] Step 9: User-responsive application behavior

[0560] The application installed on the device presents the user with home garden management tasks (e.g., "This week's mission: Increase your tomato harvest!") and evaluates their progress and achievement. The inputs are the analysis results from Steps 4 and 8, as well as the user's behavioral history and cultivation data. The output is the task presented to the user and its evaluation results.

[0561] Through this series of processing steps, users can efficiently and effectively manage their home gardens without needing advanced skills or specialized knowledge.

[0562] (Application example 1)

[0563] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0564] Conventional home garden management systems have the drawback of requiring users to visit the site in person to check the status of the plants, which is time-consuming and labor-intensive. Furthermore, accurate assessment of plant growth and the presence of pests requires specialized knowledge, making it difficult for beginners. Furthermore, when learning and involvement are required, real-world experiments are required, increasing the risk of failure.

[0565] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0566] In this invention, the server includes means for acquiring image data of plants and soil using a camera mounted on a smart device, means for communicating with the server equipped with artificial intelligence that analyzes the acquired image data and receiving the analysis results, means for providing feedback to the user on the plant growth status and the presence or absence of pests based on the received analysis results, software means for predicting plant growth and providing optimal countermeasures, and software means for allowing the user to manage and learn about their home garden in a virtual environment. This allows the user to effectively manage and learn about their home garden in a virtual environment, saving time and effort and reducing the risk of failure.

[0567] A "smart device" is an electronic device equipped with a camera, sensor, and communication function, and capable of acquiring and transmitting image data and environmental data.

[0568] "Camera" means a photographic device that can capture image data and store or transmit it as electronic data.

[0569] "Plants" are green plants grown in home gardens and agricultural activities.

[0570] "Soil" is the natural material containing organic matter and minerals that serves as a substrate for plant growth.

[0571] "Image data" refers to visual information captured by a camera and stored electronically.

[0572] "Artificial intelligence" is a technology that enables computer systems to analyze data and assist in problem-solving and decision-making.

[0573] A "server" is a computer system that processes data over a network and provides analytical results to other devices.

[0574] "Communication" is the process and means for sending and receiving data.

[0575] "Analysis results" are the detection results and judgment results of data analyzed by artificial intelligence.

[0576] "Feedback" is the process of providing analysis results and recommendations to users.

[0577] The "growth status of the plant" refers to the health and developmental progress of the plant during its cultivation.

[0578] "Pests" are insects or other harmful organisms that cause damage to plants.

[0579] "Best practices" are specific actions or processes recommended to promote plant growth and solve the problem.

[0580] "Software means" means means for executing computer programs and providing specific functions or services.

[0581] A "virtual environment" is a fictitious space or scenario generated by a computer system.

[0582] A "home garden" is a small-scale farm where individuals cultivate food within their homes or in their local neighborhoods.

[0583] "Management" is the process of monitoring and caring for plants to promote their healthy growth.

[0584] "Learning" is the activity of a user acquiring and understanding information in order to improve their knowledge or skills.

[0585] System Overview

[0586] The system of the present invention consists of a smart device, a server, and a user. The user uses a smart device such as smart glasses or a head-mounted display to acquire image data of plants and soil and transmits the data to the server. The server analyzes the acquired data, determines the plant's growth status and the presence of pests, and provides feedback. The smart device also acquires environmental data (temperature, humidity, pH concentration, etc.) and transmits this to the server, enabling more detailed analysis and feedback.

[0587] Image data acquisition and analysis process

[0588] Image data of plants and soil is acquired using a camera mounted on a smart device. The user wears smart glasses and takes images of plants in a virtual environment. Image data and environmental data are then captured in real time and sent to a server.

[0589] The server uses its built-in artificial intelligence model to analyze the received image data. The image analysis process detects the plant's growth status, the presence of pests, signs of disease, etc. It also analyzes environmental data (temperature, humidity, pH level, etc.) and generates the optimal countermeasures based on this.

[0590] Providing feedback

[0591] The analysis results are sent from the server to the smart device and notified to the user. This notification is given in the form of voice, screen display, vibration, etc., and is provided in the most understandable way for the user. For example, specific advice such as "There are pests on the tomato leaves. Please remove them" is provided.

[0592] Virtual plant management

[0593] Furthermore, the system allows users to manage and learn about their home gardens in a virtual environment. For example, when a user wears smart glasses and checks the quality of tomato leaves in a virtual sunroom, the user captures an image, and the server analyzes it to determine whether there are any pests and provides feedback.

[0594] Hardware and software used

[0595] Hardware: smart glasses, head-mounted displays, cameras, sensors

[0596] Software: Python, OpenCV, Requests, Artificial Intelligence Models

[0597] Specific examples

[0598] Example: A user uses smart glasses to take a photo of the leaves of a tomato plant they are growing in a virtual sunroom, and the system analyzes whether or not there are any pests and notifies the user.

[0599] Example prompt: Please provide a description of an application that takes pictures of virtual plants, analyzes their growth status, and provides appropriate measures. Required information includes image data of the virtual plants, and environmental data such as temperature, humidity, and pH level.

[0600] This system allows users to effectively manage and learn about home gardens in a virtual environment without visiting the site, saving time and effort. Even beginners can learn proper cultivation methods without requiring specialized knowledge.

[0601] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0602] Step 1:

[0603] The user photographs the plants they are growing in the virtual sunroom using smart glasses or a head-mounted display. The input is image data of the plants and soil captured by the smart device's camera. For data processing, the smart device captures the image data in real time and adds environmental data (time, location, etc.). The output is the captured image data and accompanying environmental data.

[0604] Step 2:

[0605] The terminal transmits the acquired image data and environmental data to the server via the Internet. The input is the image data and environmental data captured in the previous step. As a data calculation, the terminal converts these data into an appropriate format and transmits it. The output is the data transmitted to the server.

[0606] Step 3:

[0607] The server analyzes the received image data. At this stage, the input is the image data and environmental data sent from the device. The server uses a generative AI model to analyze the plant's growth status, the presence or absence of pests, signs of disease, etc. For data calculation, image analysis algorithms are used to extract various information and identify problems. The output is feedback data containing the analysis results.

[0608] Step 4:

[0609] The server generates feedback for the user based on the analysis results. The input is various information obtained through image analysis (e.g., plant health, presence of pests, environmental conditions). For data processing, the server compiles this information and creates a report in an easy-to-understand format for the user. The output is a feedback message sent to the user.

[0610] Step 5:

[0611] The terminal notifies the user of the feedback received from the server. At this stage, the input is the feedback message sent from the server. As a data computation, the terminal presents the feedback message to the user in the form of text, sound, vibration, etc. The output is the user receiving the feedback information.

[0612] Step 6:

[0613] The user takes specific action based on the feedback. The input is the feedback message notified by the terminal. As data processing, the user adjusts the plant management method based on the feedback. The output is that appropriate management is carried out and the health of the plant is maintained or improved.

[0614] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0615] The present invention is a home garden management system that combines an emotion engine that recognizes the user's emotions. This system uses cameras and sensors installed in smart devices to collect plant and soil data, and links with a server that analyzes the data to provide the analysis results as feedback to the user. It also recognizes the user's emotional state and provides feedback and advice based on that emotion. A specific embodiment of this system and its program processing are described below.

[0616] System Overview

[0617] The system is based on the premise that users wear a smart device (e.g., smart glasses or a hands-free phone) and use it to manage their home garden. A camera installed in the smart device captures image data of the soil and plants, which it then sends to a server for analysis. The server analyzes the image data, determines the growth status and any problems, and provides feedback. In addition, sensors are used to capture cultivation environment data (temperature, humidity, pH level, etc.), which is also analyzed by the server, enabling even more detailed feedback to be provided.

[0618] Furthermore, by incorporating an emotion engine, the system can recognize the user's emotional state and provide feedback and advice based on that emotion, allowing the user to perform farm work more appropriately and with greater satisfaction.

[0619] Program processing explanation

[0620] Acquisition and transmission of image data

[0621] The terminal uses the smart device's camera to capture real-time image data of soil and plants. For example, a user captures an image of a tomato leaf.

[0622] The device then sends the acquired image data to the server. This data includes not only the image itself but also information about the environment at the time of shooting (time, location, etc.).

[0623] Image analysis and feedback

[0624] The server inputs the received image data into an AI model and begins analysis, which detects the plant's growth status, the presence or absence of pests, signs of disease, and more.

[0625] The server receives the analysis results and generates a report on the current status and the best course of action based on the results. For example, it may provide specific advice such as, "There are pests on the tomato leaves, so they need to be removed immediately."

[0626] The device receives the analysis results sent from the server and notifies the user in a variety of ways, including voice, screen display, and vibration, in the form that is easiest for the user to understand.

[0627] Environmental data acquisition and analysis

[0628] The device periodically acquires environmental data such as temperature, humidity, and pH level using soil sensors and external sensors, and transmits this data to the server in real time.

[0629] The server then analyzes the received environmental data to predict crop growth and determine optimal treatment options. This analysis takes into account the seasons, weather, and soil characteristics, supporting long-term cultivation planning.

[0630] User Emotion Recognition and Feedback

[0631] The device uses its built-in emotion engine to analyze the user's facial expressions and voice to recognize their emotional state. For example, if the user is feeling frustrated while working, the device can detect that emotion.

[0632] The server generates feedback and advice based on the emotion analysis results sent from the emotion engine. For example, if the user is feeling stressed, the server may provide advice such as "It would be good to take a short break."

[0633] The device notifies the user based on their emotions, allowing the user to respond optimally according to their emotional state.

[0634] Specific examples

[0635] 1. A user wears smart glasses and takes an image of a tomato leaf.

[0636] 2. The device sends image data and environmental data to the server.

[0637] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[0638] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[0639] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[0640] 6. The device uses the emotion engine to analyze the user's emotional state and detects that the user is dissatisfied.

[0641] 7. Based on the emotion analysis results, the server generates advice such as "It might be a good idea to take a short break" and sends it to the device.

[0642] 8. The device provides emotion-based advice to the user.

[0643] 9. The user follows the app's instructions to remove the pests and carry out any subsequent maintenance work.

[0644] This system allows users to effectively manage their home gardens without requiring advanced technology or large amounts of capital, and it also improves work efficiency and satisfaction through feedback based on users' emotions.

[0645] The processing flow will be explained below.

[0646] Step 1:

[0647] The user puts on the smart device and starts the application. The smart device automatically initializes the camera and sensors and is ready for use.

[0648] Step 2:

[0649] The device uses the smart device's camera to take real-time images of plants and soil, for example, a user captures an image of a tomato leaf.

[0650] Step 3:

[0651] The device sends the captured image data to the server. This data includes not only the image data but also environmental information (such as time and location) at the time of capture.

[0652] Step 4:

[0653] The server inputs the received image data into an AI model and begins analysis, which detects the plant's growth status, the presence or absence of pests, signs of disease, and more.

[0654] Step 5:

[0655] The server generates the analysis results and sends them to the device. These results include the current state of the plants and the optimal course of action. For example, the results may include a message such as, "Pests have been found on tomato leaves. Please take measures to remove them."

[0656] Step 6:

[0657] The device will then notify the user of the analysis results received from the server. Notification methods vary, including voice, screen display, and vibration, and are provided in the form that is easiest for the user to understand.

[0658] Step 7:

[0659] The device uses soil sensors and external sensors to collect environmental data such as temperature, humidity, and pH level, which is then sent to a server in real time.

[0660] Step 8:

[0661] The server then analyzes the received environmental data to provide detailed plant growth predictions and optimal countermeasures, including information on long-term cultivation plans.

[0662] Step 9:

[0663] The device will notify the user of growth forecasts and advice on how to deal with the situation. For example, it will give advice such as, "There will be a lot of rain this week, so please refrain from watering."

[0664] Step 10:

[0665] The application installed on the device provides users with game-like missions, such as "This week's mission: Increase your tomato harvest!"

[0666] Step 11:

[0667] The application guides the user to perform specific farming tasks, such as removing pests from tomato leaves and then adding the appropriate fertilizer.

[0668] Step 12:

[0669] The device uses an emotion engine to analyze the user's facial expressions and voice to recognize their emotional state. For example, if the user is feeling frustrated while working, that emotion can be detected.

[0670] Step 13:

[0671] The server generates feedback and advice based on the user's emotions based on the emotion analysis results sent from the emotion engine. For example, if the user is feeling stressed, the server generates advice such as "It would be good to take a short break."

[0672] Step 14:

[0673] The device will notify the user based on their emotions. For example, the device will tell the user, "It would be good to take a short break," by displaying a message on the screen or by voice.

[0674] Step 15:

[0675] The terminals periodically collect data and send it to a server, which then monitors agricultural data for the entire region, enabling the analysis of performance and problems across the region and the proposal of improvement measures.

[0676] Example 2

[0677] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0678] Conventional home garden management systems only collect and analyze plant growth and environmental data, but are indifferent to the user's emotional state. As a result, they do not take into account the user's emotions or stress levels and are unable to provide appropriate feedback or advice, resulting in problems that reduce user satisfaction and work efficiency. Furthermore, conventional systems often lack real-time data analysis and feedback, making it difficult to quickly address plant problems.

[0679] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0680] In this invention, the server includes means for acquiring image data of plants and soil using a camera mounted on the smart device, means for communicating with the server equipped with artificial intelligence that analyzes the acquired image data and receiving the analysis results, means for incorporating an emotion engine that recognizes the user's emotional state and provides feedback and advice based on the recognition results, means for providing feedback to the user on the plant's growth status and the presence or absence of pests based on the image data and environmental data, and software means for providing plant growth predictions and optimal countermeasures. This allows the user to not only respond appropriately to the plant's growth status and cultivation environment, but also receive feedback and advice based on their own emotional state, enabling high satisfaction and efficient home vegetable garden management.

[0681] A "smart device" is an electronic device equipped with a camera and sensors that can be operated directly or worn by the user.

[0682] A "camera" is an optical device for acquiring image data.

[0683] "Image data" is digital data representing visual information captured by a camera.

[0684] "Artificial intelligence" refers to software technology that automatically analyzes data and makes decisions.

[0685] A "server" is a computer system used to analyze data and provide information.

[0686] "Analysis results" refers to the information obtained after artificial intelligence analyzes image data and environmental data.

[0687] "Emotional state" refers to the user's psychological and emotional state, and is recognized from data such as facial expressions and voice.

[0688] The "emotion engine" is a system that analyzes the user's emotional state and reflects the results in feedback and advice.

[0689] "Feedback" refers to information that conveys analysis results and advice to the user.

[0690] "Environmental data" refers to data on the plant's growing environment, such as temperature, humidity, and pH level.

[0691] A "sensor" is a device for collecting environmental data.

[0692] "Growth prediction" refers to information that predicts the future growth state of a plant.

[0693] The "optimal countermeasure" is a countermeasure proposed based on the analysis results and growth forecasts.

[0694] "Software means" refers to a computer program for realizing a specific function.

[0695] MODE FOR CARRYING OUT THE INVENTION

[0696] This invention is a home vegetable garden management system that combines an emotion engine that recognizes the user's emotions. This system uses cameras and sensors installed in smart devices to collect plant and soil data, and connects to a server that analyzes the data to provide the analysis results as feedback to the user. It also recognizes the user's emotional state and provides feedback and advice according to that emotion.

[0697] The system is based on the premise that users wear a smart device (e.g., smart glasses or a hands-free phone) and use it to manage their home garden. The camera on the smart device captures image data of the soil and plants and sends the data to a server for analysis.

[0698] The server uses a generative AI model (e.g., TensorFlow) to analyze the received image data. This model is trained to detect plant growth status, the presence of pests, signs of disease, etc. Based on the analysis results, it generates a detailed report on the current status and optimal measures. The analysis results are fed back to the smart device user in real time.

[0699] The device then periodically collects cultivation environment data (temperature, humidity, pH level, etc.) using sensors. This data is also sent to the server in real time and used for environmental data analysis. The server then uses the environmental data to make growth predictions and provide feedback on long-term cultivation plans and specific countermeasures.

[0700] Furthermore, the system is equipped with a user emotion engine. The device detects the user's facial expressions and voice and analyzes their emotional state. For example, if the user says "I'm tired," that information is sent to the server. The server generates optimal advice for the user based on the emotion analysis results. For example, if the user is feeling stressed, feedback such as "It would be good to take a short break" is generated and notified to the user via the device.

[0701] Specific examples

[0702] For example, you can use the system by following these steps:

[0703] 1. A user wears smart glasses and takes an image of a tomato leaf.

[0704] 2. The device sends image data and environmental data to the server.

[0705] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[0706] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[0707] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[0708] 6. The device uses the emotion engine to analyze the user's emotional state and detects that the user is dissatisfied.

[0709] 7. Based on the emotion analysis results, the server generates advice such as "It might be a good idea to take a short break" and sends it to the device.

[0710] 8. The device provides emotion-based advice to the user.

[0711] Prompt Sentence Examples

[0712] "Use this home garden management system to check whether there are any pests on your tomato leaves. Also, analyze the user's emotional state to provide appropriate feedback."

[0713] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0714] Step 1:

[0715] The terminal acquires soil and plant image data in real time using the smart device's camera. The input is image data captured through the smart device's camera. The output is high-resolution image data stored on the terminal and environmental information at the time of capture (e.g., GPS location information, timestamp). In concrete terms, a user uses smart glasses to capture an image of a tomato leaf.

[0716] Step 2:

[0717] The device sends the acquired image data to the server. The input is the image data and environmental information captured in step 1. The output is the data sent to the server. Specifically, the device transfers data to the server via a REST API using asynchronous communication.

[0718] Step 3:

[0719] The image data received by the server is input into a generative AI model for analysis. The input is the image data and environmental information sent to the server. The output is the analysis results, such as the plant's growth status, the presence or absence of pests, and signs of disease. Specifically, the server performs image analysis using a generative AI model such as TensorFlow and stores the results in a database.

[0720] Step 4:

[0721] The server generates feedback based on the analysis results. The input is the analysis result from step 3. The output is feedback information to the user. Specifically, the server generates specific advice such as "There are pests on the tomato leaves, so please remove them immediately."

[0722] Step 5:

[0723] The device notifies the user of the analysis results sent from the server. The input is the feedback information sent from the server. The output is the notification received by the user. Specifically, the device notifies the user in multiple ways, such as by voice, on-screen display, or vibration.

[0724] Step 6:

[0725] The device periodically obtains environmental data (temperature, humidity, pH concentration, etc.) using soil sensors and external sensors. The input is real-time data obtained from the sensors. The output is environmental data that is temporarily stored on the device. Specifically, the device obtains temperature and humidity data from the sensors every 30 minutes and stores it in the cache.

[0726] Step 7:

[0727] The terminal sends the acquired environmental data to the server. The input is the environmental data acquired in step 6. The output is the environmental data sent to the server. Specifically, the data is transferred to the server using real-time streaming technology (e.g., Kafka).

[0728] Step 8:

[0729] The server analyzes the received environmental data. The input is the environmental data sent to the server. The output is the environmental analysis results, which include growth predictions and countermeasures. Specifically, the server uses the environmental data to run growth prediction algorithms and simulations, and stores the results in a database.

[0730] Step 9:

[0731] The server generates feedback based on the results of the environmental analysis. The input is the environmental analysis result from step 8. The output is feedback information based on the environment to the user. Specifically, the server generates advice such as "The temperature is too high, so it would be a good idea to use the shade during the day" and sends it to the device.

[0732] Step 10:

[0733] The device uses an emotion engine to analyze the user's facial expressions and voice and recognize the user's emotional state. The input is the user's facial expression and voice data. The output is the recognized emotional state data. Specifically, the device's emotion engine detects a user statement such as "I'm tired" and sends the emotional data to the server.

[0734] Step 11:

[0735] The server generates feedback based on the emotion analysis results. The input is the emotional state data obtained in step 10. The output is feedback information based on the user's emotions. Specifically, the server generates advice such as "It might be a good idea to take a short break" and sends it to the device.

[0736] Step 12:

[0737] The device notifies the user of emotion-based feedback. The input is emotion-based feedback information sent from the server. The output is the notification received by the user. Specifically, the device notifies the user by voice, saying, "It might be a good idea to take a short break."

[0738] (Application example 2)

[0739] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0740] Conventional home garden management systems and factory quality control systems have limited means for appropriately monitoring the condition of plants and products. Furthermore, no systems existed that took into account the emotional state of workers, creating challenges in terms of work efficiency and satisfaction. This created a need for a system that could accurately grasp the growth status of plants and provide feedback based on workers' emotions.

[0741] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0742] In this invention, the server includes means for acquiring image data of plants and soil using a camera mounted on the smart device, means for communicating with the server equipped with artificial intelligence that analyzes the acquired image data and receiving the analysis results, means for providing feedback to the user on the plant growth status and the presence or absence of pests based on the received analysis results, software means for providing plant growth predictions and optimal countermeasures, and means for analyzing the emotional state of the worker using an emotion engine mounted on the smart device and providing feedback and advice based on the emotion. This makes it possible to analyze the state of the plants with high accuracy and provide feedback that takes the worker's emotion into consideration, thereby improving work efficiency and satisfaction.

[0743] A "smart device" is an advanced information terminal equipped with a camera, sensors, etc., that enables data acquisition and communication.

[0744] A "camera" is a device that captures optical images as electronic data.

[0745] "Soil" refers to the ground that serves as the base for plant growth, and is generally a natural substance that contains nutrients and moisture.

[0746] "Image data" is an electronic representation of visual information captured by a camera.

[0747] "Artificial intelligence" refers to computer systems that simulate human intelligence and have the ability to automatically analyze and solve specific problems.

[0748] A "server" is a computer system that stores and processes data over a network.

[0749] "User" refers to a person who uses this system and manages or works on plants.

[0750] "Feedback" is the process of notifying the user of analysis results and advice.

[0751] "Software means" is a computer program designed to perform a specific function.

[0752] An "emotion engine" is a technology that analyzes and judges a user's emotional state from facial expressions, voice, etc.

[0753] "Workers" refers to those involved in quality control and product manufacturing within the factory.

[0754] An "emotional state" refers to a temporary psychological state that an individual has, such as stress or satisfaction.

[0755] "Feedback and advice" refers to specific instructions or suggestions provided to a user or worker.

[0756] "Plant growth prediction" is the process of predicting the future growth state of a plant based on collected data.

[0757] The "best solution" is the most effective action or measure proposed to solve the current problem.

[0758] MODE FOR CARRYING OUT THE INVENTION

[0759] This invention relates to a home vegetable garden management system using smart devices and a server, and a quality control system using factory robots. This system uses cameras and sensors to acquire plant and product data, analyzes the data, and provides feedback to users. It also analyzes the emotional state of workers and provides advice based on that data, improving work efficiency and satisfaction.

[0760] System configuration

[0761] This system mainly consists of the following components:

[0762] Smart Devices

[0763] The smart glass is equipped with a camera, microphone, and temperature and humidity sensors, which allow it to capture plant and product data, as well as environmental data and the emotional state of workers.

[0764] server

[0765] The server has advanced analytical capabilities and analyzes the acquired data using artificial intelligence models (TensorFlow, OpenCV, etc.), and feeds back the results of image analysis, environmental data analysis, and emotion analysis to the smart device.

[0766] Software Means

[0767] Dedicated software for analysis and feedback is installed, which manages the entire process of receiving, analyzing, and providing feedback on data.

[0768] Specific Examples

[0769] A specific example will be described below.

[0770] Image data acquisition and analysis

[0771] A camera installed in the smart glass captures image data of plants (for example, tomato leaves in a home garden) or products (products in a factory). The captured data is sent in real time to a server, which then inputs it into an artificial intelligence model and begins analysis. The analysis results include the plant's growth status, the presence or absence of pests, or abnormalities in the product's quality. Feedback is sent from the server to the smart device, and the user receives specific advice.

[0772] Environmental data acquisition and analysis

[0773] Temperature and humidity sensors installed in the smart glass periodically collect data on the plant cultivation environment and the factory environment. This data is sent to a server, which analyzes it according to the season, weather, and the characteristics of the working environment. As a result, plant growth forecasts and optimal treatment methods are provided to the user.

[0774] Emotional state analysis and feedback

[0775] The smart device's built-in emotion engine captures the worker's facial expressions and voice data. This data is sent to a server in real time, and the server uses an emotion analysis model to analyze the worker's emotional state. For example, if a worker is feeling stressed, the system will detect that emotion and provide advice such as "take a short break."

[0776] Prompt Sentence Examples

[0777] Below are examples of specific prompt statements that are executed by this system.

[0778] Text format

[0779] Data Acquisition

[0780] 1. Taking images of plants and products with smart glass.

[0781] 2. Obtain real-time temperature and humidity data from environmental sensors.

[0782] 3. Photograph the worker's facial expressions and analyze their emotions.

[0783] Data analysis

[0784] 1. Image data is sent to a server and analyzed using an artificial intelligence model.

[0785] 2. Environmental data is sent to a server and analyzed according to seasons and weather conditions.

[0786] 3. The emotion data is sent to the server and analyzed using the emotion analysis model.

[0787] Providing feedback

[0788] 1. Provide feedback to the user based on the analysis results.

[0789] 2. Provide appropriate advice based on the worker's emotional state.

[0790] This system analyzes the condition of plants and products with high accuracy and provides feedback that takes into account the emotions of workers, thereby improving work efficiency and satisfaction.

[0791] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0792] Step 1:

[0793] Capture visual images using optical cameras and other devices from smart glasses

[0794] The camera installed in the device (smart glass) captures image data of plants and products, while environmental data such as temperature and humidity are simultaneously collected by sensors.

[0795] Input: Plant images, environmental sensor data (temperature, humidity)

[0796] Output: Captured image data and sensor data

[0797] Step 2:

[0798] Captured image data and sensor data are sent to a cloud server

[0799] The device transmits the acquired image data and environmental data to the server in real time.

[0800] Input: Image data and sensor data

[0801] Output: Data sent to the server

[0802] Step 3:

[0803] Analyze data using cloud-based AI models

[0804] The server inputs the received image data into an artificial intelligence model (e.g., TensorFlow) to analyze the plant's health and product quality abnormalities. Environmental data is also analyzed in a similar manner.

[0805] Input: Data received by the server

[0806] Output: Detected growth status, presence or absence of pests, or quality abnormalities

[0807] Step 4:

[0808] Create work guidelines and recommendations based on the analysis results

[0809] Based on the analysis results, the server generates feedback including plant growth predictions and optimal countermeasures, as well as suggestions for improving quality at the factory.

[0810] Input: Detection results

[0811] Output: Feedback and suggestions

[0812] Step 5:

[0813] Analyze worker emotions using captured facial expressions and voice

[0814] The terminal captures the worker's facial expression and voice data and sends it to the emotion engine, where the server analyzes the worker's emotional state using an emotion analysis model.

[0815] Input: facial expression and voice data

[0816] Output: Emotional status (stress, satisfaction, etc.)

[0817] Step 6:

[0818] Generate feedback and recommendations based on sentiment data

[0819] Based on the emotional data, the server generates feedback appropriate to the worker's psychological state and suggested actions, such as "take a break."

[0820] Input: Emotional status

[0821] Output: Emotion-based advice and recommendations

[0822] Step 7:

[0823] Send all generated feedback to the device

[0824] The server sends the final feedback, growth forecast, countermeasures, and advice to the worker to the terminal and notifies the user.

[0825] Input: Generated feedback data

[0826] Output: Notification information to the device

[0827] Step 8:

[0828] Users act on feedback

[0829] The user takes necessary measures and actions based on the notifications and advice provided by the device.

[0830] Input: Notification information from the device

[0831] Output: The measures or actions taken

[0832] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0833] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0834] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0835] [Third embodiment]

[0836] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0837] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0838] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0839] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0840] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0841] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0842] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0843] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0844] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0845] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0846] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0847] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0848] The present invention is a system that uses a camera mounted on a smart device to acquire image data of plants and soil, communicates with a server equipped with artificial intelligence that analyzes the image data, and receives the analysis results. Specific embodiments of this system and its program processing are described below.

[0849] System Overview

[0850] The system is based on the premise that users wear a smart device (e.g., smart glasses or a hands-free phone) and use it to manage their home garden. A camera installed in the smart device captures image data of the soil and plants, which it then sends to a server for analysis. The server analyzes the image data, determines the growth status and any problems, and provides feedback. In addition, sensors are used to capture cultivation environment data (temperature, humidity, pH level, etc.), which is also analyzed by the server, enabling even more detailed feedback to be provided.

[0851] Program processing explanation

[0852] Acquisition and transmission of image data

[0853] The device uses the smart device's camera to capture real-time image data of soil and plants. For example, when a user takes a photo of a tomato leaf, the device captures the image data.

[0854] The device then sends the acquired image data to the server. This data includes not only the image itself but also information about the environment at the time of shooting (time, location, etc.).

[0855] Image analysis and feedback

[0856] The server receives the image data and analyzes it using an AI model installed inside it. The image analysis process detects the plant's growth status, the presence of pests, and signs of disease.

[0857] The server receives the analysis results and generates a report on the current status and the best course of action based on the results. For example, it may provide specific advice such as, "There are pests on the tomato leaves, so they need to be removed immediately."

[0858] The device receives the analysis results sent from the server and notifies the user in a variety of ways, including voice, screen display, and vibration, in the form that is easiest for the user to understand.

[0859] Environmental data acquisition and analysis

[0860] The device periodically acquires environmental data such as temperature, humidity, and pH level using soil sensors and external sensors, and transmits this data to the server in real time.

[0861] The server then analyzes the received environmental data to predict crop growth and determine optimal treatment options. This analysis takes into account the seasons, weather, and soil characteristics, supporting long-term cultivation planning.

[0862] User-friendly game-like app

[0863] The application installed on the device is designed to make managing the vegetable garden fun for users. For example, the app encourages user involvement by presenting challenges such as "This week's mission: Increase your tomato harvest!"

[0864] The app evaluates the user's progress and achievement based on their behavioral history and cultivation data. For example, it may motivate them by saying, "You've exceeded your tomato harvest goal. Now, try growing eggplants!"

[0865] Specific examples

[0866] 1. A user wears smart glasses and takes an image of a tomato leaf.

[0867] 2. The device sends image data and environmental data to the server.

[0868] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[0869] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[0870] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[0871] 6. The user follows the app's instructions to remove the pests and carry out any subsequent maintenance work.

[0872] This system allows users to effectively manage their home gardens without requiring advanced technology or large amounts of capital, which is expected to have social benefits such as promoting agriculture, revitalizing local communities, and increasing food supplies.

[0873] The processing flow will be explained below.

[0874] Step 1:

[0875] The user puts on the smart device and starts the application. The smart device automatically initializes the camera and sensors and is ready for use.

[0876] Step 2:

[0877] The device uses the smart device's camera to take real-time images of plants and soil, for example, a user captures an image of a tomato leaf.

[0878] Step 3:

[0879] The device sends the captured image data to the server. This data includes not only the image data but also environmental data (such as time and location) at the time of capture.

[0880] Step 4:

[0881] The server inputs the received image data into an AI model and begins analysis, which detects the plant's growth status, the presence or absence of pests, signs of disease, and more.

[0882] Step 5:

[0883] The server generates the analysis results and sends them to the device. These results include the current state of the plants and the optimal course of action. For example, the results may include a message such as, "Pests have been found on tomato leaves. Please take measures to remove them."

[0884] Step 6:

[0885] The device will then notify the user of the analysis results received from the server. Notification methods vary, including voice, screen display, and vibration, and are provided in the form that is easiest for the user to understand.

[0886] Step 7:

[0887] The device uses soil sensors and external sensors to collect environmental data such as temperature, humidity, and pH level, which is then sent to a server in real time.

[0888] Step 8:

[0889] The server receives environmental data and then analyzes it to provide detailed plant growth predictions and optimal countermeasures. The analysis results also include information on long-term cultivation plans.

[0890] Step 9:

[0891] The device will notify the user of growth forecasts and advice on how to deal with the situation. For example, it will give advice such as, "There will be a lot of rain this week, so please refrain from watering."

[0892] Step 10:

[0893] The application installed on the device provides users with game-like missions, such as "This week's mission: Increase your tomato harvest!"

[0894] Step 11:

[0895] The application guides the user to perform specific farming tasks, such as removing pests from tomato leaves and then adding the appropriate fertilizer.

[0896] Step 12:

[0897] The terminals periodically collect data and send it to a server, which then monitors agricultural data for the entire region, enabling the analysis of performance and problems across the region and the proposal of improvement measures.

[0898] These are the specific processing steps of the system, which enables users to effectively manage their home gardens without requiring advanced technology or large amounts of capital.

[0899] Example 1

[0900] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0901] Conventional home garden management systems require users to visually check the condition of plants and soil and take manual action, which requires advanced technology and specialized knowledge. Furthermore, it is difficult to quickly respond to changes in the cultivation environment, resulting in problems such as slow plant growth and increased damage from diseases and pests. Furthermore, it is difficult for users to maintain an interest in garden management, which often results in inappropriate management.

[0902] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0903] In this invention, the server includes: means for acquiring image data of plants and soil using a camera mounted on a smart device; means for transmitting the acquired image data together with environmental information at the time of capture to the server; means for analyzing the received image data using an artificial intelligence model; means for providing feedback to the user on the plant's growth status, the presence of pests, and signs of disease based on the analysis results; means for generating and notifying the user of reports on optimal treatment methods and the plant's condition; means for acquiring temperature, humidity, and pH data using an environmental sensor and transmitting the data to the server; and software means for analyzing the received environmental data and providing plant growth predictions and optimal cultivation methods. This allows users to efficiently and effectively manage their home gardens without specialized knowledge. Furthermore, the software means for providing a game-like cultivation management experience helps maintain user involvement.

[0904] A "smart device" is an electronic device that can be worn or carried by a user and is equipped with a camera and sensors.

[0905] A "camera" is a photographing device for capturing images and videos, and in the present invention is installed in a smart device.

[0906] "Image data" refers to digitized visual information captured by a camera, and is data used to evaluate the condition of plants and soil through analysis.

[0907] "Environment information" is context data such as the time and location at the time of shooting, and is transmitted to the server together with the image data.

[0908] A "server" is a computer system that receives, analyzes, stores, and transmits data over a network.

[0909] An "artificial intelligence model" refers to an algorithm that has been trained to perform analysis and predictions based on input data.

[0910] "Feedback" refers to providing information or advice generated based on the analysis results to the user.

[0911] A "sensor" is a device that detects physical environmental information (e.g., temperature, humidity, pH concentration, etc.) and outputs it as digital data.

[0912] "Growth prediction" refers to predicting the future growth state of a plant based on past and current data.

[0913] "Cultivation methods" refer to the procedures and treatments required for plants to grow healthily in optimal conditions.

[0914] A "report" is a document or digital file that summarizes the analysis results and feedback information.

[0915] "User" refers to an individual who uses this system to manage a home garden.

[0916] The present invention is a system that acquires image data of plants and soil using a camera mounted on a smart device, communicates with a server equipped with artificial intelligence that analyzes the image data, and receives the analysis results. Specific embodiments of the present invention will be described below.

[0917] System Overview

[0918] The system aims to enable users to manage their home vegetable gardens using a smart device (e.g., smart glasses or a hands-free phone). The smart device's built-in camera captures image data of the soil and plants, which is then sent to a server for analysis. The server analyzes the image data, determines the plant's growth status and any problems, and provides feedback. Environmental sensors are also used to capture cultivation environment data (temperature, humidity, pH concentration, etc.), which is also analyzed by the server to provide more detailed feedback.

[0919] Hardware and software used

[0920] Smart device: Has the ability to connect a camera to acquire image data and environmental sensors.

[0921] Server: Equipped with artificial intelligence models (e.g., TensorFlow or PyTorch) for analyzing data.

[0922] Environmental sensors: DHT22 (temperature and humidity sensor), pH meter, etc. are used.

[0923] Image data acquisition and analysis

[0924] The device uses the smart device's camera to acquire real-time image data of plants and soil. For example, when a user takes a photo of a tomato leaf, the camera captures the image in high resolution. The acquired image data is then sent to the server along with environmental information at the time of the photo (e.g., photo time, location, etc.).

[0925] The server then analyzes the received image data using an artificial intelligence model. During the analysis process, features within the image are extracted to detect the plant's growth status, the presence of pests, signs of disease, and so on.

[0926] Analysis result feedback

[0927] Based on the analysis results, the server generates a report on the optimal method of dealing with the problem and the condition of the plant. For example, if there are pests on tomato leaves, it generates specific advice such as, "There are pests on the tomato leaves. Please remove them immediately."

[0928] The analysis results are sent to the device and provided to the user in a variety of ways, including voice notification, screen display, and vibration alert, allowing the user to take appropriate action quickly.

[0929] Environmental data acquisition and analysis

[0930] The device periodically collects environmental data such as temperature, humidity, and pH level using soil sensors and external sensors, and transmits this data to the server in real time.

[0931] The server analyzes the received environmental data and proposes optimal cultivation methods and forecasts for plant growth. This analysis takes into account the season, weather, and soil characteristics, and also provides support for long-term cultivation planning.

[0932] Game-like application

[0933] The application installed on the device is designed to make managing a home vegetable garden fun for users. For example, it encourages user involvement by presenting challenges such as, "This week's mission: Increase your tomato harvest!" The app also evaluates the user's progress and achievement based on their behavioral history and cultivation data, and motivates them by saying, "Your tomato harvest exceeded your goal. Let's try growing eggplants next!"

[0934] Examples of specific examples and prompts

[0935] 1. A user wears smart glasses and takes an image of a tomato leaf.

[0936] 2. The device sends image data and environmental data to the server.

[0937] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[0938] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[0939] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[0940] 6. The user follows the app's instructions to remove the pests and carry out any subsequent maintenance work.

[0941] Prompt Sentence Examples

[0942] "Take an image of a tomato leaf and analyze it to detect whether there are any pests on the leaf."

[0943] "Please suggest specific measures to take if there are pests on tomato leaves."

[0944] "Predict plant growth based on temperature, humidity, and pH data."

[0945] In this way, the present invention allows users to efficiently and effectively manage their home gardens without requiring advanced skills or specialized knowledge. Furthermore, the software means that allows users to manage cultivation in a game-like manner can keep users engaged.

[0946] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0947] Step 1: Acquiring image data

[0948] The device uses the smart device's camera to capture image data of plants and soil. When a user wears smart glasses and takes a picture of a tomato leaf, the camera captures the image in high resolution. The input is the video from the camera attached to the smart glasses. The output is the captured image data.

[0949] Step 2: Sending image data

[0950] The device sends the image data it has acquired along with environmental information at the time of shooting (such as the time of shooting and location information) to the server. The input is the image data and environmental information acquired in step 1. Specifically, the data is transferred securely using the HTTPS protocol. The output is the image data and environmental information received by the server.

[0951] Step 3: Analyzing the image data

[0952] The image data received by the server is analyzed using an internal artificial intelligence model (e.g., using TensorFlow or PyTorch). The input is the image data and environmental information sent in step 2. Features within the image are extracted to detect the plant's growth status, the presence or absence of pests, signs of disease, etc. The output is the analysis results.

[0953] Step 4: Generate analysis results

[0954] The server generates feedback (report) to provide to the user based on the analysis results. The input is the analysis results from step 3. Specifically, it generates advice such as "There are pests on the tomato leaves. Please remove them immediately" based on the plant's growth status and any problems. The output is the generated feedback report.

[0955] Step 5: Notification of analysis results

[0956] The device receives the analysis results sent from the server and notifies the user. The input is the feedback report generated in step 4. Notification methods include voice notification, screen display, and vibration alert. The output is the notification information received by the user.

[0957] Step 6: Get environment data

[0958] The terminal uses soil sensors and external sensors to acquire environmental data such as temperature, humidity, and pH concentration. The input is real-time environmental data acquired from the sensors. The output is the acquired environmental data.

[0959] Step 7: Sending environment data

[0960] The terminal transmits the environmental data acquired to the server. The input is the environmental data acquired in step 6. Specifically, the MQTT protocol is used to efficiently transfer the data. The output is the environmental data received by the server.

[0961] Step 8: Analyze environmental data

[0962] The server analyzes the received environmental data and makes plant growth predictions and proposes optimal cultivation methods. The input is the environmental data sent in step 7. Regression analysis and machine learning algorithms are used to analyze data trends and predict future growth. The output is the analysis results and proposals.

[0963] Step 9: User-responsive application behavior

[0964] The application installed on the device presents the user with home garden management tasks (e.g., "This week's mission: Increase your tomato harvest!") and evaluates their progress and achievement. The inputs are the analysis results from Steps 4 and 8, as well as the user's behavioral history and cultivation data. The output is the task presented to the user and its evaluation results.

[0965] Through this series of processing steps, users can efficiently and effectively manage their home gardens without needing advanced skills or specialized knowledge.

[0966] (Application example 1)

[0967] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0968] Conventional home garden management systems have the drawback of requiring users to visit the site in person to check the status of the plants, which is time-consuming and labor-intensive. Furthermore, accurate assessment of plant growth and the presence of pests requires specialized knowledge, making it difficult for beginners. Furthermore, when learning and involvement are required, real-world experiments are required, increasing the risk of failure.

[0969] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0970] In this invention, the server includes means for acquiring image data of plants and soil using a camera mounted on a smart device, means for communicating with the server equipped with artificial intelligence that analyzes the acquired image data and receiving the analysis results, means for providing feedback to the user on the plant growth status and the presence or absence of pests based on the received analysis results, software means for predicting plant growth and providing optimal countermeasures, and software means for allowing the user to manage and learn about their home garden in a virtual environment. This allows the user to effectively manage and learn about their home garden in a virtual environment, saving time and effort and reducing the risk of failure.

[0971] A "smart device" is an electronic device equipped with a camera, sensor, and communication function, and capable of acquiring and transmitting image data and environmental data.

[0972] "Camera" means a photographic device that can capture image data and store or transmit it as electronic data.

[0973] "Plants" are green plants grown in home gardens and agricultural activities.

[0974] "Soil" is the natural material containing organic matter and minerals that serves as a substrate for plant growth.

[0975] "Image data" refers to visual information captured by a camera and stored electronically.

[0976] "Artificial intelligence" is a technology that enables computer systems to analyze data and assist in problem-solving and decision-making.

[0977] A "server" is a computer system that processes data over a network and provides analytical results to other devices.

[0978] "Communication" is the process and means for sending and receiving data.

[0979] "Analysis results" are the detection results and judgment results of data analyzed by artificial intelligence.

[0980] "Feedback" is the process of providing analysis results and recommendations to users.

[0981] The "growth status of the plant" refers to the health and developmental progress of the plant during its cultivation.

[0982] "Pests" are insects or other harmful organisms that cause damage to plants.

[0983] "Best practices" are specific actions or processes recommended to promote plant growth and solve the problem.

[0984] "Software means" means means for executing computer programs and providing specific functions or services.

[0985] A "virtual environment" is a fictitious space or scenario generated by a computer system.

[0986] A "home garden" is a small-scale farm where individuals cultivate food within their homes or in their local neighborhoods.

[0987] "Management" is the process of monitoring and caring for plants to promote their healthy growth.

[0988] "Learning" is the activity of a user acquiring and understanding information in order to improve their knowledge or skills.

[0989] System Overview

[0990] The system of the present invention consists of a smart device, a server, and a user. The user uses a smart device such as smart glasses or a head-mounted display to acquire image data of plants and soil and transmits the data to the server. The server analyzes the acquired data, determines the plant's growth status and the presence of pests, and provides feedback. The smart device also acquires environmental data (temperature, humidity, pH concentration, etc.) and transmits this to the server, enabling more detailed analysis and feedback.

[0991] Image data acquisition and analysis process

[0992] Image data of plants and soil is acquired using a camera mounted on a smart device. The user wears smart glasses and takes images of plants in a virtual environment. Image data and environmental data are then captured in real time and sent to a server.

[0993] The server uses its built-in artificial intelligence model to analyze the received image data. The image analysis process detects the plant's growth status, the presence of pests, signs of disease, etc. It also analyzes environmental data (temperature, humidity, pH level, etc.) and generates the optimal countermeasures based on this.

[0994] Providing feedback

[0995] The analysis results are sent from the server to the smart device and notified to the user. This notification is given in the form of voice, screen display, vibration, etc., and is provided in the most understandable way for the user. For example, specific advice such as "There are pests on the tomato leaves. Please remove them" is provided.

[0996] Virtual plant management

[0997] Furthermore, the system allows users to manage and learn about their home gardens in a virtual environment. For example, when a user wears smart glasses and checks the quality of tomato leaves in a virtual sunroom, the user captures an image, and the server analyzes it to determine whether there are any pests and provides feedback.

[0998] Hardware and software used

[0999] Hardware: smart glasses, head-mounted displays, cameras, sensors

[1000] Software: Python, OpenCV, Requests, Artificial Intelligence Models

[1001] Specific examples

[1002] Example: A user uses smart glasses to take a photo of the leaves of a tomato plant they are growing in a virtual sunroom, and the system analyzes whether or not there are any pests and notifies the user.

[1003] Example prompt: Please provide a description of an application that takes pictures of virtual plants, analyzes their growth status, and provides appropriate measures. Required information includes image data of the virtual plants, and environmental data such as temperature, humidity, and pH level.

[1004] This system allows users to effectively manage and learn about home gardens in a virtual environment without visiting the site, saving time and effort. Even beginners can learn proper cultivation methods without requiring specialized knowledge.

[1005] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1006] Step 1:

[1007] The user photographs the plants they are growing in the virtual sunroom using smart glasses or a head-mounted display. The input is image data of the plants and soil captured by the smart device's camera. For data processing, the smart device captures the image data in real time and adds environmental data (time, location, etc.). The output is the captured image data and accompanying environmental data.

[1008] Step 2:

[1009] The terminal transmits the acquired image data and environmental data to the server via the Internet. The input is the image data and environmental data captured in the previous step. As a data calculation, the terminal converts these data into an appropriate format and transmits it. The output is the data transmitted to the server.

[1010] Step 3:

[1011] The server analyzes the received image data. At this stage, the input is the image data and environmental data sent from the device. The server uses a generative AI model to analyze the plant's growth status, the presence or absence of pests, signs of disease, etc. For data calculation, image analysis algorithms are used to extract various information and identify problems. The output is feedback data containing the analysis results.

[1012] Step 4:

[1013] The server generates feedback for the user based on the analysis results. The input is various information obtained through image analysis (e.g., plant health, presence of pests, environmental conditions). For data processing, the server compiles this information and creates a report in an easy-to-understand format for the user. The output is a feedback message sent to the user.

[1014] Step 5:

[1015] The terminal notifies the user of the feedback received from the server. At this stage, the input is the feedback message sent from the server. As a data computation, the terminal presents the feedback message to the user in the form of text, sound, vibration, etc. The output is the user receiving the feedback information.

[1016] Step 6:

[1017] The user takes specific action based on the feedback. The input is the feedback message notified by the terminal. As data processing, the user adjusts the plant management method based on the feedback. The output is that appropriate management is carried out and the health of the plant is maintained or improved.

[1018] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1019] The present invention is a home garden management system that combines an emotion engine that recognizes the user's emotions. This system uses cameras and sensors installed in smart devices to collect plant and soil data, and links with a server that analyzes the data to provide the analysis results as feedback to the user. It also recognizes the user's emotional state and provides feedback and advice based on that emotion. A specific embodiment of this system and its program processing are described below.

[1020] System Overview

[1021] The system is based on the premise that users wear a smart device (e.g., smart glasses or a hands-free phone) and use it to manage their home garden. A camera installed in the smart device captures image data of the soil and plants, which it then sends to a server for analysis. The server analyzes the image data, determines the growth status and any problems, and provides feedback. In addition, sensors are used to capture cultivation environment data (temperature, humidity, pH level, etc.), which is also analyzed by the server, enabling even more detailed feedback to be provided.

[1022] Furthermore, by incorporating an emotion engine, the system can recognize the user's emotional state and provide feedback and advice based on that emotion, allowing the user to perform farm work more appropriately and with greater satisfaction.

[1023] Program processing explanation

[1024] Acquisition and transmission of image data

[1025] The terminal uses the smart device's camera to capture real-time image data of soil and plants. For example, a user captures an image of a tomato leaf.

[1026] The device then sends the acquired image data to the server. This data includes not only the image itself but also information about the environment at the time of shooting (time, location, etc.).

[1027] Image analysis and feedback

[1028] The server inputs the received image data into an AI model and begins analysis, which detects the plant's growth status, the presence or absence of pests, signs of disease, and more.

[1029] The server receives the analysis results and generates a report on the current status and the best course of action based on the results. For example, it may provide specific advice such as, "There are pests on the tomato leaves, so they need to be removed immediately."

[1030] The device receives the analysis results sent from the server and notifies the user in a variety of ways, including voice, screen display, and vibration, in the form that is easiest for the user to understand.

[1031] Environmental data acquisition and analysis

[1032] The device periodically acquires environmental data such as temperature, humidity, and pH level using soil sensors and external sensors, and transmits this data to the server in real time.

[1033] The server then analyzes the received environmental data to predict crop growth and determine optimal treatment options. This analysis takes into account the seasons, weather, and soil characteristics, supporting long-term cultivation planning.

[1034] User Emotion Recognition and Feedback

[1035] The device uses its built-in emotion engine to analyze the user's facial expressions and voice to recognize their emotional state. For example, if the user is feeling frustrated while working, the device can detect that emotion.

[1036] The server generates feedback and advice based on the emotion analysis results sent from the emotion engine. For example, if the user is feeling stressed, the server may provide advice such as "It would be good to take a short break."

[1037] The device notifies the user based on their emotions, allowing the user to respond optimally according to their emotional state.

[1038] Specific examples

[1039] 1. A user wears smart glasses and takes an image of a tomato leaf.

[1040] 2. The device sends image data and environmental data to the server.

[1041] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[1042] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[1043] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[1044] 6. The device uses the emotion engine to analyze the user's emotional state and detects that the user is dissatisfied.

[1045] 7. Based on the emotion analysis results, the server generates advice such as "It might be a good idea to take a short break" and sends it to the device.

[1046] 8. The device provides emotion-based advice to the user.

[1047] 9. The user follows the app's instructions to remove the pests and carry out any subsequent maintenance work.

[1048] This system allows users to effectively manage their home gardens without requiring advanced technology or large amounts of capital, and it also improves work efficiency and satisfaction through feedback based on users' emotions.

[1049] The processing flow will be explained below.

[1050] Step 1:

[1051] The user puts on the smart device and starts the application. The smart device automatically initializes the camera and sensors and is ready for use.

[1052] Step 2:

[1053] The device uses the smart device's camera to take real-time images of plants and soil, for example, a user captures an image of a tomato leaf.

[1054] Step 3:

[1055] The device sends the captured image data to the server. This data includes not only the image data but also environmental information (such as time and location) at the time of capture.

[1056] Step 4:

[1057] The server inputs the received image data into an AI model and begins analysis, which detects the plant's growth status, the presence or absence of pests, signs of disease, and more.

[1058] Step 5:

[1059] The server generates the analysis results and sends them to the device. These results include the current state of the plants and the optimal course of action. For example, the results may include a message such as, "Pests have been found on tomato leaves. Please take measures to remove them."

[1060] Step 6:

[1061] The device will then notify the user of the analysis results received from the server. Notification methods vary, including voice, screen display, and vibration, and are provided in the form that is easiest for the user to understand.

[1062] Step 7:

[1063] The device uses soil sensors and external sensors to collect environmental data such as temperature, humidity, and pH level, which is then sent to a server in real time.

[1064] Step 8:

[1065] The server then analyzes the received environmental data to provide detailed plant growth predictions and optimal countermeasures, including information on long-term cultivation plans.

[1066] Step 9:

[1067] The device will notify the user of growth forecasts and advice on how to deal with the situation. For example, it will give advice such as, "There will be a lot of rain this week, so please refrain from watering."

[1068] Step 10:

[1069] The application installed on the device provides users with game-like missions, such as "This week's mission: Increase your tomato harvest!"

[1070] Step 11:

[1071] The application guides the user to perform specific farming tasks, such as removing pests from tomato leaves and then adding the appropriate fertilizer.

[1072] Step 12:

[1073] The device uses an emotion engine to analyze the user's facial expressions and voice to recognize their emotional state. For example, if the user is feeling frustrated while working, that emotion can be detected.

[1074] Step 13:

[1075] The server generates feedback and advice based on the user's emotions based on the emotion analysis results sent from the emotion engine. For example, if the user is feeling stressed, the server generates advice such as "It would be good to take a short break."

[1076] Step 14:

[1077] The device will notify the user based on their emotions. For example, the device will tell the user, "It would be good to take a short break," by displaying a message on the screen or by voice.

[1078] Step 15:

[1079] The terminals periodically collect data and send it to a server, which then monitors agricultural data for the entire region, enabling the analysis of performance and problems across the region and the proposal of improvement measures.

[1080] Example 2

[1081] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1082] Conventional home garden management systems only collect and analyze plant growth and environmental data, but are indifferent to the user's emotional state. As a result, they do not take into account the user's emotions or stress levels and are unable to provide appropriate feedback or advice, resulting in problems that reduce user satisfaction and work efficiency. Furthermore, conventional systems often lack real-time data analysis and feedback, making it difficult to quickly address plant problems.

[1083] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1084] In this invention, the server includes means for acquiring image data of plants and soil using a camera mounted on the smart device, means for communicating with the server equipped with artificial intelligence that analyzes the acquired image data and receiving the analysis results, means for incorporating an emotion engine that recognizes the user's emotional state and provides feedback and advice based on the recognition results, means for providing feedback to the user on the plant's growth status and the presence or absence of pests based on the image data and environmental data, and software means for providing plant growth predictions and optimal countermeasures. This allows the user to not only respond appropriately to the plant's growth status and cultivation environment, but also receive feedback and advice based on their own emotional state, enabling high satisfaction and efficient home vegetable garden management.

[1085] A "smart device" is an electronic device equipped with a camera and sensors that can be operated directly or worn by the user.

[1086] A "camera" is an optical device for acquiring image data.

[1087] "Image data" is digital data representing visual information captured by a camera.

[1088] "Artificial intelligence" refers to software technology that automatically analyzes data and makes decisions.

[1089] A "server" is a computer system used to analyze data and provide information.

[1090] "Analysis results" refers to the information obtained after artificial intelligence analyzes image data and environmental data.

[1091] "Emotional state" refers to the user's psychological and emotional state, and is recognized from data such as facial expressions and voice.

[1092] The "emotion engine" is a system that analyzes the user's emotional state and reflects the results in feedback and advice.

[1093] "Feedback" refers to information that conveys analysis results and advice to the user.

[1094] "Environmental data" refers to data on the plant's growing environment, such as temperature, humidity, and pH level.

[1095] A "sensor" is a device for collecting environmental data.

[1096] "Growth prediction" refers to information that predicts the future growth state of a plant.

[1097] The "optimal countermeasure" is a countermeasure proposed based on the analysis results and growth forecasts.

[1098] "Software means" refers to a computer program for realizing a specific function.

[1099] MODE FOR CARRYING OUT THE INVENTION

[1100] This invention is a home vegetable garden management system that combines an emotion engine that recognizes the user's emotions. This system uses cameras and sensors installed in smart devices to collect plant and soil data, and connects to a server that analyzes the data to provide the analysis results as feedback to the user. It also recognizes the user's emotional state and provides feedback and advice according to that emotion.

[1101] The system is based on the premise that users wear a smart device (e.g., smart glasses or a hands-free phone) and use it to manage their home garden. The camera on the smart device captures image data of the soil and plants and sends the data to a server for analysis.

[1102] The server uses a generative AI model (e.g., TensorFlow) to analyze the received image data. This model is trained to detect plant growth status, the presence of pests, signs of disease, etc. Based on the analysis results, it generates a detailed report on the current status and optimal measures. The analysis results are fed back to the smart device user in real time.

[1103] The device then periodically collects cultivation environment data (temperature, humidity, pH level, etc.) using sensors. This data is also sent to the server in real time and used for environmental data analysis. The server then uses the environmental data to make growth predictions and provide feedback on long-term cultivation plans and specific countermeasures.

[1104] Furthermore, the system is equipped with a user emotion engine. The device detects the user's facial expressions and voice and analyzes their emotional state. For example, if the user says "I'm tired," that information is sent to the server. The server generates optimal advice for the user based on the emotion analysis results. For example, if the user is feeling stressed, feedback such as "It would be good to take a short break" is generated and notified to the user via the device.

[1105] Specific examples

[1106] For example, you can use the system by following these steps:

[1107] 1. A user wears smart glasses and takes an image of a tomato leaf.

[1108] 2. The device sends image data and environmental data to the server.

[1109] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[1110] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[1111] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[1112] 6. The device uses the emotion engine to analyze the user's emotional state and detects that the user is dissatisfied.

[1113] 7. Based on the emotion analysis results, the server generates advice such as "It might be a good idea to take a short break" and sends it to the device.

[1114] 8. The device provides emotion-based advice to the user.

[1115] Prompt Sentence Examples

[1116] "Use this home garden management system to check whether there are any pests on your tomato leaves. Also, analyze the user's emotional state to provide appropriate feedback."

[1117] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1118] Step 1:

[1119] The terminal acquires soil and plant image data in real time using the smart device's camera. The input is image data captured through the smart device's camera. The output is high-resolution image data stored on the terminal and environmental information at the time of capture (e.g., GPS location information, timestamp). In concrete terms, a user uses smart glasses to capture an image of a tomato leaf.

[1120] Step 2:

[1121] The device sends the acquired image data to the server. The input is the image data and environmental information captured in step 1. The output is the data sent to the server. Specifically, the device transfers data to the server via a REST API using asynchronous communication.

[1122] Step 3:

[1123] The image data received by the server is input into a generative AI model for analysis. The input is the image data and environmental information sent to the server. The output is the analysis results, such as the plant's growth status, the presence or absence of pests, and signs of disease. Specifically, the server performs image analysis using a generative AI model such as TensorFlow and stores the results in a database.

[1124] Step 4:

[1125] The server generates feedback based on the analysis results. The input is the analysis result from step 3. The output is feedback information to the user. Specifically, the server generates specific advice such as "There are pests on the tomato leaves, so please remove them immediately."

[1126] Step 5:

[1127] The device notifies the user of the analysis results sent from the server. The input is the feedback information sent from the server. The output is the notification received by the user. Specifically, the device notifies the user in multiple ways, such as by voice, on-screen display, or vibration.

[1128] Step 6:

[1129] The device periodically obtains environmental data (temperature, humidity, pH concentration, etc.) using soil sensors and external sensors. The input is real-time data obtained from the sensors. The output is environmental data that is temporarily stored on the device. Specifically, the device obtains temperature and humidity data from the sensors every 30 minutes and stores it in the cache.

[1130] Step 7:

[1131] The terminal sends the acquired environmental data to the server. The input is the environmental data acquired in step 6. The output is the environmental data sent to the server. Specifically, the data is transferred to the server using real-time streaming technology (e.g., Kafka).

[1132] Step 8:

[1133] The server analyzes the received environmental data. The input is the environmental data sent to the server. The output is the environmental analysis results, which include growth predictions and countermeasures. Specifically, the server uses the environmental data to run growth prediction algorithms and simulations, and stores the results in a database.

[1134] Step 9:

[1135] The server generates feedback based on the results of the environmental analysis. The input is the environmental analysis result from step 8. The output is feedback information based on the environment to the user. Specifically, the server generates advice such as "The temperature is too high, so it would be a good idea to use the shade during the day" and sends it to the device.

[1136] Step 10:

[1137] The device uses an emotion engine to analyze the user's facial expressions and voice and recognize the user's emotional state. The input is the user's facial expression and voice data. The output is the recognized emotional state data. Specifically, the device's emotion engine detects a user statement such as "I'm tired" and sends the emotional data to the server.

[1138] Step 11:

[1139] The server generates feedback based on the emotion analysis results. The input is the emotional state data obtained in step 10. The output is feedback information based on the user's emotions. Specifically, the server generates advice such as "It might be a good idea to take a short break" and sends it to the device.

[1140] Step 12:

[1141] The device notifies the user of emotion-based feedback. The input is emotion-based feedback information sent from the server. The output is the notification received by the user. Specifically, the device notifies the user by voice, saying, "It might be a good idea to take a short break."

[1142] (Application example 2)

[1143] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1144] Conventional home garden management systems and factory quality control systems have limited means for appropriately monitoring the condition of plants and products. Furthermore, no systems existed that took into account the emotional state of workers, creating challenges in terms of work efficiency and satisfaction. This created a need for a system that could accurately grasp the growth status of plants and provide feedback based on workers' emotions.

[1145] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1146] In this invention, the server includes means for acquiring image data of plants and soil using a camera mounted on the smart device, means for communicating with the server equipped with artificial intelligence that analyzes the acquired image data and receiving the analysis results, means for providing feedback to the user on the plant growth status and the presence or absence of pests based on the received analysis results, software means for providing plant growth predictions and optimal countermeasures, and means for analyzing the emotional state of the worker using an emotion engine mounted on the smart device and providing feedback and advice based on the emotion. This makes it possible to analyze the state of the plants with high accuracy and provide feedback that takes the worker's emotion into consideration, thereby improving work efficiency and satisfaction.

[1147] A "smart device" is an advanced information terminal equipped with a camera, sensors, etc., that enables data acquisition and communication.

[1148] A "camera" is a device that captures optical images as electronic data.

[1149] "Soil" refers to the ground that serves as the base for plant growth, and is generally a natural substance that contains nutrients and moisture.

[1150] "Image data" is an electronic representation of visual information captured by a camera.

[1151] "Artificial intelligence" refers to computer systems that simulate human intelligence and have the ability to automatically analyze and solve specific problems.

[1152] A "server" is a computer system that stores and processes data over a network.

[1153] "User" refers to a person who uses this system and manages or works on plants.

[1154] "Feedback" is the process of notifying the user of analysis results and advice.

[1155] "Software means" is a computer program designed to perform a specific function.

[1156] An "emotion engine" is a technology that analyzes and judges a user's emotional state from facial expressions, voice, etc.

[1157] "Workers" refers to those involved in quality control and product manufacturing within the factory.

[1158] An "emotional state" refers to a temporary psychological state that an individual has, such as stress or satisfaction.

[1159] "Feedback and advice" refers to specific instructions or suggestions provided to a user or worker.

[1160] "Plant growth prediction" is the process of predicting the future growth state of a plant based on collected data.

[1161] The "best solution" is the most effective action or measure proposed to solve the current problem.

[1162] MODE FOR CARRYING OUT THE INVENTION

[1163] This invention relates to a home vegetable garden management system using smart devices and a server, and a quality control system using factory robots. This system uses cameras and sensors to acquire plant and product data, analyzes the data, and provides feedback to users. It also analyzes the emotional state of workers and provides advice based on that data, improving work efficiency and satisfaction.

[1164] System configuration

[1165] This system mainly consists of the following components:

[1166] Smart Devices

[1167] The smart glass is equipped with a camera, microphone, and temperature and humidity sensors, which allow it to capture plant and product data, as well as environmental data and the emotional state of workers.

[1168] server

[1169] The server has advanced analytical capabilities and analyzes the acquired data using artificial intelligence models (TensorFlow, OpenCV, etc.), and feeds back the results of image analysis, environmental data analysis, and emotion analysis to the smart device.

[1170] Software Means

[1171] Dedicated software for analysis and feedback is installed, which manages the entire process of receiving, analyzing, and providing feedback on data.

[1172] Specific Examples

[1173] A specific example will be described below.

[1174] Image data acquisition and analysis

[1175] A camera installed in the smart glass captures image data of plants (for example, tomato leaves in a home garden) or products (products in a factory). The captured data is sent in real time to a server, which then inputs it into an artificial intelligence model and begins analysis. The analysis results include the plant's growth status, the presence or absence of pests, or abnormalities in the product's quality. Feedback is sent from the server to the smart device, and the user receives specific advice.

[1176] Environmental data acquisition and analysis

[1177] Temperature and humidity sensors installed in the smart glass periodically collect data on the plant cultivation environment and the factory environment. This data is sent to a server, which analyzes it according to the season, weather, and the characteristics of the working environment. As a result, plant growth forecasts and optimal treatment methods are provided to the user.

[1178] Emotional state analysis and feedback

[1179] The smart device's built-in emotion engine captures the worker's facial expressions and voice data. This data is sent to a server in real time, and the server uses an emotion analysis model to analyze the worker's emotional state. For example, if a worker is feeling stressed, the system will detect that emotion and provide advice such as "take a short break."

[1180] Prompt Sentence Examples

[1181] Below are examples of specific prompt statements that are executed by this system.

[1182] Text format

[1183] Data Acquisition

[1184] 1. Taking images of plants and products with smart glass.

[1185] 2. Obtain real-time temperature and humidity data from environmental sensors.

[1186] 3. Photograph the worker's facial expressions and analyze their emotions.

[1187] Data analysis

[1188] 1. Image data is sent to a server and analyzed using an artificial intelligence model.

[1189] 2. Environmental data is sent to a server and analyzed according to seasons and weather conditions.

[1190] 3. The emotion data is sent to the server and analyzed using the emotion analysis model.

[1191] Providing feedback

[1192] 1. Provide feedback to the user based on the analysis results.

[1193] 2. Provide appropriate advice based on the worker's emotional state.

[1194] This system analyzes the condition of plants and products with high accuracy and provides feedback that takes into account the emotions of workers, thereby improving work efficiency and satisfaction.

[1195] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1196] Step 1:

[1197] Capture visual images using optical cameras and other devices from smart glasses

[1198] The camera installed in the device (smart glass) captures image data of plants and products, while environmental data such as temperature and humidity are simultaneously collected by sensors.

[1199] Input: Plant images, environmental sensor data (temperature, humidity)

[1200] Output: Captured image data and sensor data

[1201] Step 2:

[1202] Captured image data and sensor data are sent to a cloud server

[1203] The device transmits the acquired image data and environmental data to the server in real time.

[1204] Input: Image data and sensor data

[1205] Output: Data sent to the server

[1206] Step 3:

[1207] Analyze data using cloud-based AI models

[1208] The server inputs the received image data into an artificial intelligence model (e.g., TensorFlow) to analyze the plant's health and product quality abnormalities. Environmental data is also analyzed in a similar manner.

[1209] Input: Data received by the server

[1210] Output: Detected growth status, presence or absence of pests, or quality abnormalities

[1211] Step 4:

[1212] Create work guidelines and recommendations based on the analysis results

[1213] Based on the analysis results, the server generates feedback including plant growth predictions and optimal countermeasures, as well as suggestions for improving quality at the factory.

[1214] Input: Detection results

[1215] Output: Feedback and suggestions

[1216] Step 5:

[1217] Analyze worker emotions using captured facial expressions and voice

[1218] The terminal captures the worker's facial expression and voice data and sends it to the emotion engine, where the server analyzes the worker's emotional state using an emotion analysis model.

[1219] Input: facial expression and voice data

[1220] Output: Emotional status (stress, satisfaction, etc.)

[1221] Step 6:

[1222] Generate feedback and recommendations based on sentiment data

[1223] Based on the emotional data, the server generates feedback appropriate to the worker's psychological state and suggested actions, such as "take a break."

[1224] Input: Emotional status

[1225] Output: Emotion-based advice and recommendations

[1226] Step 7:

[1227] Send all generated feedback to the device

[1228] The server sends the final feedback, growth forecast, countermeasures, and advice to the worker to the terminal and notifies the user.

[1229] Input: Generated feedback data

[1230] Output: Notification information to the device

[1231] Step 8:

[1232] Users act on feedback

[1233] The user takes necessary measures and actions based on the notifications and advice provided by the device.

[1234] Input: Notification information from the device

[1235] Output: The measures or actions taken

[1236] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1237] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1238] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1239] [Fourth embodiment]

[1240] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1241] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1242] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1243] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1244] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1245] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1246] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1247] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1248] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1249] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1250] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1251] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1252] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1253] The present invention is a system that uses a camera mounted on a smart device to acquire image data of plants and soil, communicates with a server equipped with artificial intelligence that analyzes the image data, and receives the analysis results. Specific embodiments of this system and its program processing are described below.

[1254] System Overview

[1255] The system is based on the premise that users wear a smart device (e.g., smart glasses or a hands-free phone) and use it to manage their home garden. A camera installed in the smart device captures image data of the soil and plants, which it then sends to a server for analysis. The server analyzes the image data, determines the growth status and any problems, and provides feedback. In addition, sensors are used to capture cultivation environment data (temperature, humidity, pH level, etc.), which is also analyzed by the server, enabling even more detailed feedback to be provided.

[1256] Program processing explanation

[1257] Acquisition and transmission of image data

[1258] The device uses the smart device's camera to capture real-time image data of soil and plants. For example, when a user takes a photo of a tomato leaf, the device captures the image data.

[1259] The device then sends the acquired image data to the server. This data includes not only the image itself but also information about the environment at the time of shooting (time, location, etc.).

[1260] Image analysis and feedback

[1261] The server receives the image data and analyzes it using an AI model installed inside it. The image analysis process detects the plant's growth status, the presence of pests, and signs of disease.

[1262] The server receives the analysis results and generates a report on the current status and the best course of action based on the results. For example, it may provide specific advice such as, "There are pests on the tomato leaves, so they need to be removed immediately."

[1263] The device receives the analysis results sent from the server and notifies the user in a variety of ways, including voice, screen display, and vibration, in the form that is easiest for the user to understand.

[1264] Environmental data acquisition and analysis

[1265] The device periodically acquires environmental data such as temperature, humidity, and pH level using soil sensors and external sensors, and transmits this data to the server in real time.

[1266] The server then analyzes the received environmental data to predict crop growth and determine optimal treatment options. This analysis takes into account the seasons, weather, and soil characteristics, supporting long-term cultivation planning.

[1267] User-friendly game-like app

[1268] The application installed on the device is designed to make managing the vegetable garden fun for users. For example, the app encourages user involvement by presenting challenges such as "This week's mission: Increase your tomato harvest!"

[1269] The app evaluates the user's progress and achievement based on their behavioral history and cultivation data. For example, it may motivate them by saying, "You've exceeded your tomato harvest goal. Now, try growing eggplants!"

[1270] Specific examples

[1271] 1. A user wears smart glasses and takes an image of a tomato leaf.

[1272] 2. The device sends image data and environmental data to the server.

[1273] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[1274] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[1275] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[1276] 6. The user follows the app's instructions to remove the pests and carry out any subsequent maintenance work.

[1277] This system allows users to effectively manage their home gardens without requiring advanced technology or large amounts of capital, which is expected to have social benefits such as promoting agriculture, revitalizing local communities, and increasing food supplies.

[1278] The processing flow will be explained below.

[1279] Step 1:

[1280] The user puts on the smart device and starts the application. The smart device automatically initializes the camera and sensors and is ready for use.

[1281] Step 2:

[1282] The device uses the smart device's camera to take real-time images of plants and soil, for example, a user captures an image of a tomato leaf.

[1283] Step 3:

[1284] The device sends the captured image data to the server. This data includes not only the image data but also environmental data (such as time and location) at the time of capture.

[1285] Step 4:

[1286] The server inputs the received image data into an AI model and begins analysis, which detects the plant's growth status, the presence or absence of pests, signs of disease, and more.

[1287] Step 5:

[1288] The server generates the analysis results and sends them to the device. These results include the current state of the plants and the optimal course of action. For example, the results may include a message such as, "Pests have been found on tomato leaves. Please take measures to remove them."

[1289] Step 6:

[1290] The device will then notify the user of the analysis results received from the server. Notification methods vary, including voice, screen display, and vibration, and are provided in the form that is easiest for the user to understand.

[1291] Step 7:

[1292] The device uses soil sensors and external sensors to collect environmental data such as temperature, humidity, and pH level, which is then sent to a server in real time.

[1293] Step 8:

[1294] The server receives environmental data and then analyzes it to provide detailed plant growth predictions and optimal countermeasures. The analysis results also include information on long-term cultivation plans.

[1295] Step 9:

[1296] The device will notify the user of growth forecasts and advice on how to deal with the situation. For example, it will give advice such as, "There will be a lot of rain this week, so please refrain from watering."

[1297] Step 10:

[1298] The application installed on the device provides users with game-like missions, such as "This week's mission: Increase your tomato harvest!"

[1299] Step 11:

[1300] The application guides the user to perform specific farming tasks, such as removing pests from tomato leaves and then adding the appropriate fertilizer.

[1301] Step 12:

[1302] The terminals periodically collect data and send it to a server, which then monitors agricultural data for the entire region, enabling the analysis of performance and problems across the region and the proposal of improvement measures.

[1303] These are the specific processing steps of the system, which enables users to effectively manage their home gardens without requiring advanced technology or large amounts of capital.

[1304] Example 1

[1305] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1306] Conventional home garden management systems require users to visually check the condition of plants and soil and take manual action, which requires advanced technology and specialized knowledge. Furthermore, it is difficult to quickly respond to changes in the cultivation environment, resulting in problems such as slow plant growth and increased damage from diseases and pests. Furthermore, it is difficult for users to maintain an interest in garden management, which often results in inappropriate management.

[1307] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1308] In this invention, the server includes: means for acquiring image data of plants and soil using a camera mounted on a smart device; means for transmitting the acquired image data together with environmental information at the time of capture to the server; means for analyzing the received image data using an artificial intelligence model; means for providing feedback to the user on the plant's growth status, the presence of pests, and signs of disease based on the analysis results; means for generating and notifying the user of reports on optimal treatment methods and the plant's condition; means for acquiring temperature, humidity, and pH data using an environmental sensor and transmitting the data to the server; and software means for analyzing the received environmental data and providing plant growth predictions and optimal cultivation methods. This allows users to efficiently and effectively manage their home gardens without specialized knowledge. Furthermore, the software means for providing a game-like cultivation management experience helps maintain user involvement.

[1309] A "smart device" is an electronic device that can be worn or carried by a user and is equipped with a camera and sensors.

[1310] A "camera" is a photographing device for capturing images and videos, and in the present invention is installed in a smart device.

[1311] "Image data" refers to digitized visual information captured by a camera, and is data used to evaluate the condition of plants and soil through analysis.

[1312] "Environment information" is context data such as the time and location at the time of shooting, and is transmitted to the server together with the image data.

[1313] A "server" is a computer system that receives, analyzes, stores, and transmits data over a network.

[1314] An "artificial intelligence model" refers to an algorithm that has been trained to perform analysis and predictions based on input data.

[1315] "Feedback" refers to providing information or advice generated based on the analysis results to the user.

[1316] A "sensor" is a device that detects physical environmental information (e.g., temperature, humidity, pH concentration, etc.) and outputs it as digital data.

[1317] "Growth prediction" refers to predicting the future growth state of a plant based on past and current data.

[1318] "Cultivation methods" refer to the procedures and treatments required for plants to grow healthily in optimal conditions.

[1319] A "report" is a document or digital file that summarizes the analysis results and feedback information.

[1320] "User" refers to an individual who uses this system to manage a home garden.

[1321] The present invention is a system that acquires image data of plants and soil using a camera mounted on a smart device, communicates with a server equipped with artificial intelligence that analyzes the image data, and receives the analysis results. Specific embodiments of the present invention will be described below.

[1322] System Overview

[1323] The system aims to enable users to manage their home vegetable gardens using a smart device (e.g., smart glasses or a hands-free phone). The smart device's built-in camera captures image data of the soil and plants, which is then sent to a server for analysis. The server analyzes the image data, determines the plant's growth status and any problems, and provides feedback. Environmental sensors are also used to capture cultivation environment data (temperature, humidity, pH concentration, etc.), which is also analyzed by the server to provide more detailed feedback.

[1324] Hardware and software used

[1325] Smart device: Has the ability to connect a camera to acquire image data and environmental sensors.

[1326] Server: Equipped with artificial intelligence models (e.g., TensorFlow or PyTorch) for analyzing data.

[1327] Environmental sensors: DHT22 (temperature and humidity sensor), pH meter, etc. are used.

[1328] Image data acquisition and analysis

[1329] The device uses the smart device's camera to acquire real-time image data of plants and soil. For example, when a user takes a photo of a tomato leaf, the camera captures the image in high resolution. The acquired image data is then sent to the server along with environmental information at the time of the photo (e.g., photo time, location, etc.).

[1330] The server then analyzes the received image data using an artificial intelligence model. During the analysis process, features within the image are extracted to detect the plant's growth status, the presence of pests, signs of disease, and so on.

[1331] Analysis result feedback

[1332] Based on the analysis results, the server generates a report on the optimal method of dealing with the problem and the condition of the plant. For example, if there are pests on tomato leaves, it generates specific advice such as, "There are pests on the tomato leaves. Please remove them immediately."

[1333] The analysis results are sent to the device and provided to the user in a variety of ways, including voice notification, screen display, and vibration alert, allowing the user to take appropriate action quickly.

[1334] Environmental data acquisition and analysis

[1335] The device periodically collects environmental data such as temperature, humidity, and pH level using soil sensors and external sensors, and transmits this data to the server in real time.

[1336] The server analyzes the received environmental data and proposes optimal cultivation methods and forecasts for plant growth. This analysis takes into account the season, weather, and soil characteristics, and also provides support for long-term cultivation planning.

[1337] Game-like application

[1338] The application installed on the device is designed to make managing a home vegetable garden fun for users. For example, it encourages user involvement by presenting challenges such as, "This week's mission: Increase your tomato harvest!" The app also evaluates the user's progress and achievement based on their behavioral history and cultivation data, and motivates them by saying, "Your tomato harvest exceeded your goal. Let's try growing eggplants next!"

[1339] Examples of specific examples and prompts

[1340] 1. A user wears smart glasses and takes an image of a tomato leaf.

[1341] 2. The device sends image data and environmental data to the server.

[1342] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[1343] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[1344] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[1345] 6. The user follows the app's instructions to remove the pests and carry out any subsequent maintenance work.

[1346] Prompt Sentence Examples

[1347] "Take an image of a tomato leaf and analyze it to detect whether there are any pests on the leaf."

[1348] "Please suggest specific measures to take if there are pests on tomato leaves."

[1349] "Predict plant growth based on temperature, humidity, and pH data."

[1350] In this way, the present invention allows users to efficiently and effectively manage their home gardens without requiring advanced skills or specialized knowledge. Furthermore, the software means that allows users to manage cultivation in a game-like manner can keep users engaged.

[1351] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1352] Step 1: Acquiring image data

[1353] The device uses the smart device's camera to capture image data of plants and soil. When a user wears smart glasses and takes a picture of a tomato leaf, the camera captures the image in high resolution. The input is the video from the camera attached to the smart glasses. The output is the captured image data.

[1354] Step 2: Sending image data

[1355] The device sends the image data it has acquired along with environmental information at the time of shooting (such as the time of shooting and location information) to the server. The input is the image data and environmental information acquired in step 1. Specifically, the data is transferred securely using the HTTPS protocol. The output is the image data and environmental information received by the server.

[1356] Step 3: Analyzing the image data

[1357] The image data received by the server is analyzed using an internal artificial intelligence model (e.g., using TensorFlow or PyTorch). The input is the image data and environmental information sent in step 2. Features within the image are extracted to detect the plant's growth status, the presence or absence of pests, signs of disease, etc. The output is the analysis results.

[1358] Step 4: Generate analysis results

[1359] The server generates feedback (report) to provide to the user based on the analysis results. The input is the analysis results from step 3. Specifically, it generates advice such as "There are pests on the tomato leaves. Please remove them immediately" based on the plant's growth status and any problems. The output is the generated feedback report.

[1360] Step 5: Notification of analysis results

[1361] The device receives the analysis results sent from the server and notifies the user. The input is the feedback report generated in step 4. Notification methods include voice notification, screen display, and vibration alert. The output is the notification information received by the user.

[1362] Step 6: Get environment data

[1363] The terminal uses soil sensors and external sensors to acquire environmental data such as temperature, humidity, and pH concentration. The input is real-time environmental data acquired from the sensors. The output is the acquired environmental data.

[1364] Step 7: Sending environment data

[1365] The terminal transmits the environmental data acquired to the server. The input is the environmental data acquired in step 6. Specifically, the MQTT protocol is used to efficiently transfer the data. The output is the environmental data received by the server.

[1366] Step 8: Analyze environmental data

[1367] The server analyzes the received environmental data and makes plant growth predictions and proposes optimal cultivation methods. The input is the environmental data sent in step 7. Regression analysis and machine learning algorithms are used to analyze data trends and predict future growth. The output is the analysis results and proposals.

[1368] Step 9: User-responsive application behavior

[1369] The application installed on the device presents the user with home garden management tasks (e.g., "This week's mission: Increase your tomato harvest!") and evaluates their progress and achievement. The inputs are the analysis results from Steps 4 and 8, as well as the user's behavioral history and cultivation data. The output is the task presented to the user and its evaluation results.

[1370] Through this series of processing steps, users can efficiently and effectively manage their home gardens without needing advanced skills or specialized knowledge.

[1371] (Application example 1)

[1372] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1373] Conventional home garden management systems have the drawback of requiring users to visit the site in person to check the status of the plants, which is time-consuming and labor-intensive. Furthermore, accurate assessment of plant growth and the presence of pests requires specialized knowledge, making it difficult for beginners. Furthermore, when learning and involvement are required, real-world experiments are required, increasing the risk of failure.

[1374] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1375] In this invention, the server includes means for acquiring image data of plants and soil using a camera mounted on a smart device, means for communicating with the server equipped with artificial intelligence that analyzes the acquired image data and receiving the analysis results, means for providing feedback to the user on the plant growth status and the presence or absence of pests based on the received analysis results, software means for predicting plant growth and providing optimal countermeasures, and software means for allowing the user to manage and learn about their home garden in a virtual environment. This allows the user to effectively manage and learn about their home garden in a virtual environment, saving time and effort and reducing the risk of failure.

[1376] A "smart device" is an electronic device equipped with a camera, sensor, and communication function, and capable of acquiring and transmitting image data and environmental data.

[1377] "Camera" means a photographic device that can capture image data and store or transmit it as electronic data.

[1378] "Plants" are green plants grown in home gardens and agricultural activities.

[1379] "Soil" is the natural material containing organic matter and minerals that serves as a substrate for plant growth.

[1380] "Image data" refers to visual information captured by a camera and stored electronically.

[1381] "Artificial intelligence" is a technology that enables computer systems to analyze data and assist in problem-solving and decision-making.

[1382] A "server" is a computer system that processes data over a network and provides analytical results to other devices.

[1383] "Communication" is the process and means for sending and receiving data.

[1384] "Analysis results" are the detection results and judgment results of data analyzed by artificial intelligence.

[1385] "Feedback" is the process of providing analysis results and recommendations to users.

[1386] The "growth status of the plant" refers to the health and developmental progress of the plant during its cultivation.

[1387] "Pests" are insects or other harmful organisms that cause damage to plants.

[1388] "Best practices" are specific actions or processes recommended to promote plant growth and solve the problem.

[1389] "Software means" means means for executing computer programs and providing specific functions or services.

[1390] A "virtual environment" is a fictitious space or scenario generated by a computer system.

[1391] A "home garden" is a small-scale farm where individuals cultivate food within their homes or in their local neighborhoods.

[1392] "Management" is the process of monitoring and caring for plants to promote their healthy growth.

[1393] "Learning" is the activity of a user acquiring and understanding information in order to improve their knowledge or skills.

[1394] System Overview

[1395] The system of the present invention consists of a smart device, a server, and a user. The user uses a smart device such as smart glasses or a head-mounted display to acquire image data of plants and soil and transmits the data to the server. The server analyzes the acquired data, determines the plant's growth status and the presence of pests, and provides feedback. The smart device also acquires environmental data (temperature, humidity, pH concentration, etc.) and transmits this to the server, enabling more detailed analysis and feedback.

[1396] Image data acquisition and analysis process

[1397] Image data of plants and soil is acquired using a camera mounted on a smart device. The user wears smart glasses and takes images of plants in a virtual environment. Image data and environmental data are then captured in real time and sent to a server.

[1398] The server uses its built-in artificial intelligence model to analyze the received image data. The image analysis process detects the plant's growth status, the presence of pests, signs of disease, etc. It also analyzes environmental data (temperature, humidity, pH level, etc.) and generates the optimal countermeasures based on this.

[1399] Providing feedback

[1400] The analysis results are sent from the server to the smart device and notified to the user. This notification is given in the form of voice, screen display, vibration, etc., and is provided in the most understandable way for the user. For example, specific advice such as "There are pests on the tomato leaves. Please remove them" is provided.

[1401] Virtual plant management

[1402] Furthermore, the system allows users to manage and learn about their home gardens in a virtual environment. For example, when a user wears smart glasses and checks the quality of tomato leaves in a virtual sunroom, the user captures an image, and the server analyzes it to determine whether there are any pests and provides feedback.

[1403] Hardware and software used

[1404] Hardware: smart glasses, head-mounted displays, cameras, sensors

[1405] Software: Python, OpenCV, Requests, Artificial Intelligence Models

[1406] Specific examples

[1407] Example: A user uses smart glasses to take a photo of the leaves of a tomato plant they are growing in a virtual sunroom, and the system analyzes whether or not there are any pests and notifies the user.

[1408] Example prompt: Please provide a description of an application that takes pictures of virtual plants, analyzes their growth status, and provides appropriate measures. Required information includes image data of the virtual plants, and environmental data such as temperature, humidity, and pH level.

[1409] This system allows users to effectively manage and learn about home gardens in a virtual environment without visiting the site, saving time and effort. Even beginners can learn proper cultivation methods without requiring specialized knowledge.

[1410] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1411] Step 1:

[1412] The user photographs the plants they are growing in the virtual sunroom using smart glasses or a head-mounted display. The input is image data of the plants and soil captured by the smart device's camera. For data processing, the smart device captures the image data in real time and adds environmental data (time, location, etc.). The output is the captured image data and accompanying environmental data.

[1413] Step 2:

[1414] The terminal transmits the acquired image data and environmental data to the server via the Internet. The input is the image data and environmental data captured in the previous step. As a data calculation, the terminal converts these data into an appropriate format and transmits it. The output is the data transmitted to the server.

[1415] Step 3:

[1416] The server analyzes the received image data. At this stage, the input is the image data and environmental data sent from the device. The server uses a generative AI model to analyze the plant's growth status, the presence or absence of pests, signs of disease, etc. For data calculation, image analysis algorithms are used to extract various information and identify problems. The output is feedback data containing the analysis results.

[1417] Step 4:

[1418] The server generates feedback for the user based on the analysis results. The input is various information obtained through image analysis (e.g., plant health, presence of pests, environmental conditions). For data processing, the server compiles this information and creates a report in an easy-to-understand format for the user. The output is a feedback message sent to the user.

[1419] Step 5:

[1420] The terminal notifies the user of the feedback received from the server. At this stage, the input is the feedback message sent from the server. As a data computation, the terminal presents the feedback message to the user in the form of text, sound, vibration, etc. The output is the user receiving the feedback information.

[1421] Step 6:

[1422] The user takes specific action based on the feedback. The input is the feedback message notified by the terminal. As data processing, the user adjusts the plant management method based on the feedback. The output is that appropriate management is carried out and the health of the plant is maintained or improved.

[1423] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1424] The present invention is a home garden management system that combines an emotion engine that recognizes the user's emotions. This system uses cameras and sensors installed in smart devices to collect plant and soil data, and links with a server that analyzes the data to provide the analysis results as feedback to the user. It also recognizes the user's emotional state and provides feedback and advice based on that emotion. A specific embodiment of this system and its program processing are described below.

[1425] System Overview

[1426] The system is based on the premise that users wear a smart device (e.g., smart glasses or a hands-free phone) and use it to manage their home garden. A camera installed in the smart device captures image data of the soil and plants, which it then sends to a server for analysis. The server analyzes the image data, determines the growth status and any problems, and provides feedback. In addition, sensors are used to capture cultivation environment data (temperature, humidity, pH level, etc.), which is also analyzed by the server, enabling even more detailed feedback to be provided.

[1427] Furthermore, by incorporating an emotion engine, the system can recognize the user's emotional state and provide feedback and advice based on that emotion, allowing the user to perform farm work more appropriately and with greater satisfaction.

[1428] Program processing explanation

[1429] Acquisition and transmission of image data

[1430] The terminal uses the smart device's camera to capture real-time image data of soil and plants. For example, a user captures an image of a tomato leaf.

[1431] The device then sends the acquired image data to the server. This data includes not only the image itself but also information about the environment at the time of shooting (time, location, etc.).

[1432] Image analysis and feedback

[1433] The server inputs the received image data into an AI model and begins analysis, which detects the plant's growth status, the presence or absence of pests, signs of disease, and more.

[1434] The server receives the analysis results and generates a report on the current status and the best course of action based on the results. For example, it may provide specific advice such as, "There are pests on the tomato leaves, so they need to be removed immediately."

[1435] The device receives the analysis results sent from the server and notifies the user in a variety of ways, including voice, screen display, and vibration, in the form that is easiest for the user to understand.

[1436] Environmental data acquisition and analysis

[1437] The device periodically acquires environmental data such as temperature, humidity, and pH level using soil sensors and external sensors, and transmits this data to the server in real time.

[1438] The server then analyzes the received environmental data to predict crop growth and determine optimal treatment options. This analysis takes into account the seasons, weather, and soil characteristics, supporting long-term cultivation planning.

[1439] User Emotion Recognition and Feedback

[1440] The device uses its built-in emotion engine to analyze the user's facial expressions and voice to recognize their emotional state. For example, if the user is feeling frustrated while working, the device can detect that emotion.

[1441] The server generates feedback and advice based on the emotion analysis results sent from the emotion engine. For example, if the user is feeling stressed, the server may provide advice such as "It would be good to take a short break."

[1442] The device notifies the user based on their emotions, allowing the user to respond optimally according to their emotional state.

[1443] Specific examples

[1444] 1. A user wears smart glasses and takes an image of a tomato leaf.

[1445] 2. The device sends image data and environmental data to the server.

[1446] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[1447] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[1448] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[1449] 6. The device uses the emotion engine to analyze the user's emotional state and detects that the user is dissatisfied.

[1450] 7. Based on the emotion analysis results, the server generates advice such as "It might be a good idea to take a short break" and sends it to the device.

[1451] 8. The device provides emotion-based advice to the user.

[1452] 9. The user follows the app's instructions to remove the pests and carry out any subsequent maintenance work.

[1453] This system allows users to effectively manage their home gardens without requiring advanced technology or large amounts of capital, and it also improves work efficiency and satisfaction through feedback based on users' emotions.

[1454] The processing flow will be explained below.

[1455] Step 1:

[1456] The user puts on the smart device and starts the application. The smart device automatically initializes the camera and sensors and is ready for use.

[1457] Step 2:

[1458] The device uses the smart device's camera to take real-time images of plants and soil, for example, a user captures an image of a tomato leaf.

[1459] Step 3:

[1460] The device sends the captured image data to the server. This data includes not only the image data but also environmental information (such as time and location) at the time of capture.

[1461] Step 4:

[1462] The server inputs the received image data into an AI model and begins analysis, which detects the plant's growth status, the presence or absence of pests, signs of disease, and more.

[1463] Step 5:

[1464] The server generates the analysis results and sends them to the device. These results include the current state of the plants and the optimal course of action. For example, the results may include a message such as, "Pests have been found on tomato leaves. Please take measures to remove them."

[1465] Step 6:

[1466] The device will then notify the user of the analysis results received from the server. Notification methods vary, including voice, screen display, and vibration, and are provided in the form that is easiest for the user to understand.

[1467] Step 7:

[1468] The device uses soil sensors and external sensors to collect environmental data such as temperature, humidity, and pH level, which is then sent to a server in real time.

[1469] Step 8:

[1470] The server then analyzes the received environmental data to provide detailed plant growth predictions and optimal countermeasures, including information on long-term cultivation plans.

[1471] Step 9:

[1472] The device will notify the user of growth forecasts and advice on how to deal with the situation. For example, it will give advice such as, "There will be a lot of rain this week, so please refrain from watering."

[1473] Step 10:

[1474] The application installed on the device provides users with game-like missions, such as "This week's mission: Increase your tomato harvest!"

[1475] Step 11:

[1476] The application guides the user to perform specific farming tasks, such as removing pests from tomato leaves and then adding the appropriate fertilizer.

[1477] Step 12:

[1478] The device uses an emotion engine to analyze the user's facial expressions and voice to recognize their emotional state. For example, if the user is feeling frustrated while working, that emotion can be detected.

[1479] Step 13:

[1480] The server generates feedback and advice based on the user's emotions based on the emotion analysis results sent from the emotion engine. For example, if the user is feeling stressed, the server generates advice such as "It would be good to take a short break."

[1481] Step 14:

[1482] The device will notify the user based on their emotions. For example, the device will tell the user, "It would be good to take a short break," by displaying a message on the screen or by voice.

[1483] Step 15:

[1484] The terminals periodically collect data and send it to a server, which then monitors agricultural data for the entire region, enabling the analysis of performance and problems across the region and the proposal of improvement measures.

[1485] Example 2

[1486] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1487] Conventional home garden management systems only collect and analyze plant growth and environmental data, but are indifferent to the user's emotional state. As a result, they do not take into account the user's emotions or stress levels and are unable to provide appropriate feedback or advice, resulting in problems that reduce user satisfaction and work efficiency. Furthermore, conventional systems often lack real-time data analysis and feedback, making it difficult to quickly address plant problems.

[1488] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1489] In this invention, the server includes means for acquiring image data of plants and soil using a camera mounted on the smart device, means for communicating with the server equipped with artificial intelligence that analyzes the acquired image data and receiving the analysis results, means for incorporating an emotion engine that recognizes the user's emotional state and provides feedback and advice based on the recognition results, means for providing feedback to the user on the plant's growth status and the presence or absence of pests based on the image data and environmental data, and software means for providing plant growth predictions and optimal countermeasures. This allows the user to not only respond appropriately to the plant's growth status and cultivation environment, but also receive feedback and advice based on their own emotional state, enabling high satisfaction and efficient home vegetable garden management.

[1490] A "smart device" is an electronic device equipped with a camera and sensors that can be operated directly or worn by the user.

[1491] A "camera" is an optical device for acquiring image data.

[1492] "Image data" is digital data representing visual information captured by a camera.

[1493] "Artificial intelligence" refers to software technology that automatically analyzes data and makes decisions.

[1494] A "server" is a computer system used to analyze data and provide information.

[1495] "Analysis results" refers to the information obtained after artificial intelligence analyzes image data and environmental data.

[1496] "Emotional state" refers to the user's psychological and emotional state, and is recognized from data such as facial expressions and voice.

[1497] The "emotion engine" is a system that analyzes the user's emotional state and reflects the results in feedback and advice.

[1498] "Feedback" refers to information that conveys analysis results and advice to the user.

[1499] "Environmental data" refers to data on the plant's growing environment, such as temperature, humidity, and pH level.

[1500] A "sensor" is a device for collecting environmental data.

[1501] "Growth prediction" refers to information that predicts the future growth state of a plant.

[1502] The "optimal countermeasure" is a countermeasure proposed based on the analysis results and growth forecasts.

[1503] "Software means" refers to a computer program for realizing a specific function.

[1504] MODE FOR CARRYING OUT THE INVENTION

[1505] This invention is a home vegetable garden management system that combines an emotion engine that recognizes the user's emotions. This system uses cameras and sensors installed in smart devices to collect plant and soil data, and connects to a server that analyzes the data to provide the analysis results as feedback to the user. It also recognizes the user's emotional state and provides feedback and advice according to that emotion.

[1506] The system is based on the premise that users wear a smart device (e.g., smart glasses or a hands-free phone) and use it to manage their home garden. The camera on the smart device captures image data of the soil and plants and sends the data to a server for analysis.

[1507] The server uses a generative AI model (e.g., TensorFlow) to analyze the received image data. This model is trained to detect plant growth status, the presence of pests, signs of disease, etc. Based on the analysis results, it generates a detailed report on the current status and optimal measures. The analysis results are fed back to the smart device user in real time.

[1508] The device then periodically collects cultivation environment data (temperature, humidity, pH level, etc.) using sensors. This data is also sent to the server in real time and used for environmental data analysis. The server then uses the environmental data to make growth predictions and provide feedback on long-term cultivation plans and specific countermeasures.

[1509] Furthermore, the system is equipped with a user emotion engine. The device detects the user's facial expressions and voice and analyzes their emotional state. For example, if the user says "I'm tired," that information is sent to the server. The server generates optimal advice for the user based on the emotion analysis results. For example, if the user is feeling stressed, feedback such as "It would be good to take a short break" is generated and notified to the user via the device.

[1510] Specific examples

[1511] For example, you can use the system by following these steps:

[1512] 1. A user wears smart glasses and takes an image of a tomato leaf.

[1513] 2. The device sends image data and environmental data to the server.

[1514] 3. The server analyzes the data and detects the presence of pests on the tomato leaves.

[1515] 4. The server sends the analysis results to the device and suggests specific countermeasures.

[1516] 5. The device notifies the user, "There are pests on the tomato leaves. Please remove them."

[1517] 6. The device uses the emotion engine to analyze the user's emotional state and detects that the user is dissatisfied.

[1518] 7. Based on the emotion analysis results, the server generates advice such as "It might be a good idea to take a short break" and sends it to the device.

[1519] 8. The device provides emotion-based advice to the user.

[1520] Prompt Sentence Examples

[1521] "Use this home garden management system to check whether there are any pests on your tomato leaves. Also, analyze the user's emotional state to provide appropriate feedback."

[1522] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1523] Step 1:

[1524] The terminal acquires soil and plant image data in real time using the smart device's camera. The input is image data captured through the smart device's camera. The output is high-resolution image data stored on the terminal and environmental information at the time of capture (e.g., GPS location information, timestamp). In concrete terms, a user uses smart glasses to capture an image of a tomato leaf.

[1525] Step 2:

[1526] The device sends the acquired image data to the server. The input is the image data and environmental information captured in step 1. The output is the data sent to the server. Specifically, the device transfers data to the server via a REST API using asynchronous communication.

[1527] Step 3:

[1528] The image data received by the server is input into a generative AI model for analysis. The input is the image data and environmental information sent to the server. The output is the analysis results, such as the plant's growth status, the presence or absence of pests, and signs of disease. Specifically, the server performs image analysis using a generative AI model such as TensorFlow and stores the results in a database.

[1529] Step 4:

[1530] The server generates feedback based on the analysis results. The input is the analysis result from step 3. The output is feedback information to the user. Specifically, the server generates specific advice such as "There are pests on the tomato leaves, so please remove them immediately."

[1531] Step 5:

[1532] The device notifies the user of the analysis results sent from the server. The input is the feedback information sent from the server. The output is the notification received by the user. Specifically, the device notifies the user in multiple ways, such as by voice, on-screen display, or vibration.

[1533] Step 6:

[1534] The device periodically obtains environmental data (temperature, humidity, pH concentration, etc.) using soil sensors and external sensors. The input is real-time data obtained from the sensors. The output is environmental data that is temporarily stored on the device. Specifically, the device obtains temperature and humidity data from the sensors every 30 minutes and stores it in the cache.

[1535] Step 7:

[1536] The terminal sends the acquired environmental data to the server. The input is the environmental data acquired in step 6. The output is the environmental data sent to the server. Specifically, the data is transferred to the server using real-time streaming technology (e.g., Kafka).

[1537] Step 8:

[1538] The server analyzes the received environmental data. The input is the environmental data sent to the server. The output is the environmental analysis results, which include growth predictions and countermeasures. Specifically, the server uses the environmental data to run growth prediction algorithms and simulations, and stores the results in a database.

[1539] Step 9:

[1540] The server generates feedback based on the results of the environmental analysis. The input is the environmental analysis result from step 8. The output is feedback information based on the environment to the user. Specifically, the server generates advice such as "The temperature is too high, so it would be a good idea to use the shade during the day" and sends it to the device.

[1541] Step 10:

[1542] The device uses an emotion engine to analyze the user's facial expressions and voice and recognize the user's emotional state. The input is the user's facial expression and voice data. The output is the recognized emotional state data. Specifically, the device's emotion engine detects a user statement such as "I'm tired" and sends the emotional data to the server.

[1543] Step 11:

[1544] The server generates feedback based on the emotion analysis results. The input is the emotional state data obtained in step 10. The output is feedback information based on the user's emotions. Specifically, the server generates advice such as "It might be a good idea to take a short break" and sends it to the device.

[1545] Step 12:

[1546] The device notifies the user of emotion-based feedback. The input is emotion-based feedback information sent from the server. The output is the notification received by the user. Specifically, the device notifies the user by voice, saying, "It might be a good idea to take a short break."

[1547] (Application example 2)

[1548] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1549] Conventional home garden management systems and factory quality control systems have limited means for appropriately monitoring the condition of plants and products. Furthermore, no systems existed that took into account the emotional state of workers, creating challenges in terms of work efficiency and satisfaction. This created a need for a system that could accurately grasp the growth status of plants and provide feedback based on workers' emotions.

[1550] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1551] In this invention, the server includes means for acquiring image data of plants and soil using a camera mounted on the smart device, means for communicating with the server equipped with artificial intelligence that analyzes the acquired image data and receiving the analysis results, means for providing feedback to the user on the plant growth status and the presence or absence of pests based on the received analysis results, software means for providing plant growth predictions and optimal countermeasures, and means for analyzing the emotional state of the worker using an emotion engine mounted on the smart device and providing feedback and advice based on the emotion. This makes it possible to analyze the state of the plants with high accuracy and provide feedback that takes the worker's emotion into consideration, thereby improving work efficiency and satisfaction.

[1552] A "smart device" is an advanced information terminal equipped with a camera, sensors, etc., that enables data acquisition and communication.

[1553] A "camera" is a device that captures optical images as electronic data.

[1554] "Soil" refers to the ground that serves as the base for plant growth, and is generally a natural substance that contains nutrients and moisture.

[1555] "Image data" is an electronic representation of visual information captured by a camera.

[1556] "Artificial intelligence" refers to computer systems that simulate human intelligence and have the ability to automatically analyze and solve specific problems.

[1557] A "server" is a computer system that stores and processes data over a network.

[1558] "User" refers to a person who uses this system and manages or works on plants.

[1559] "Feedback" is the process of notifying the user of analysis results and advice.

[1560] "Software means" is a computer program designed to perform a specific function.

[1561] An "emotion engine" is a technology that analyzes and judges a user's emotional state from facial expressions, voice, etc.

[1562] "Workers" refers to those involved in quality control and product manufacturing within the factory.

[1563] An "emotional state" refers to a temporary psychological state that an individual has, such as stress or satisfaction.

[1564] "Feedback and advice" refers to specific instructions or suggestions provided to a user or worker.

[1565] "Plant growth prediction" is the process of predicting the future growth state of a plant based on collected data.

[1566] The "best solution" is the most effective action or measure proposed to solve the current problem.

[1567] MODE FOR CARRYING OUT THE INVENTION

[1568] This invention relates to a home vegetable garden management system using smart devices and a server, and a quality control system using factory robots. This system uses cameras and sensors to acquire plant and product data, analyzes the data, and provides feedback to users. It also analyzes the emotional state of workers and provides advice based on that data, improving work efficiency and satisfaction.

[1569] System configuration

[1570] This system mainly consists of the following components:

[1571] Smart Devices

[1572] The smart glass is equipped with a camera, microphone, and temperature and humidity sensors, which allow it to capture plant and product data, as well as environmental data and the emotional state of workers.

[1573] server

[1574] The server has advanced analytical capabilities and analyzes the acquired data using artificial intelligence models (TensorFlow, OpenCV, etc.), and feeds back the results of image analysis, environmental data analysis, and emotion analysis to the smart device.

[1575] Software Means

[1576] Dedicated software for analysis and feedback is installed, which manages the entire process of receiving, analyzing, and providing feedback on data.

[1577] Specific Examples

[1578] A specific example will be described below.

[1579] Image data acquisition and analysis

[1580] A camera installed in the smart glass captures image data of plants (for example, tomato leaves in a home garden) or products (products in a factory). The captured data is sent in real time to a server, which then inputs it into an artificial intelligence model and begins analysis. The analysis results include the plant's growth status, the presence or absence of pests, or abnormalities in the product's quality. Feedback is sent from the server to the smart device, and the user receives specific advice.

[1581] Environmental data acquisition and analysis

[1582] Temperature and humidity sensors installed in the smart glass periodically collect data on the plant cultivation environment and the factory environment. This data is sent to a server, which analyzes it according to the season, weather, and the characteristics of the working environment. As a result, plant growth forecasts and optimal treatment methods are provided to the user.

[1583] Emotional state analysis and feedback

[1584] The smart device's built-in emotion engine captures the worker's facial expressions and voice data. This data is sent to a server in real time, and the server uses an emotion analysis model to analyze the worker's emotional state. For example, if a worker is feeling stressed, the system will detect that emotion and provide advice such as "take a short break."

[1585] Prompt Sentence Examples

[1586] Below are examples of specific prompt statements that are executed by this system.

[1587] Text format

[1588] Data Acquisition

[1589] 1. Taking images of plants and products with smart glass.

[1590] 2. Obtain real-time temperature and humidity data from environmental sensors.

[1591] 3. Photograph the worker's facial expressions and analyze their emotions.

[1592] Data analysis

[1593] 1. Image data is sent to a server and analyzed using an artificial intelligence model.

[1594] 2. Environmental data is sent to a server and analyzed according to seasons and weather conditions.

[1595] 3. The emotion data is sent to the server and analyzed using the emotion analysis model.

[1596] Providing feedback

[1597] 1. Provide feedback to the user based on the analysis results.

[1598] 2. Provide appropriate advice based on the worker's emotional state.

[1599] This system analyzes the condition of plants and products with high accuracy and provides feedback that takes into account the emotions of workers, thereby improving work efficiency and satisfaction.

[1600] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1601] Step 1:

[1602] Capture visual images using optical cameras and other devices from smart glasses

[1603] The camera installed in the device (smart glass) captures image data of plants and products, while environmental data such as temperature and humidity are simultaneously collected by sensors.

[1604] Input: Plant images, environmental sensor data (temperature, humidity)

[1605] Output: Captured image data and sensor data

[1606] Step 2:

[1607] Captured image data and sensor data are sent to a cloud server

[1608] The device transmits the acquired image data and environmental data to the server in real time.

[1609] Input: Image data and sensor data

[1610] Output: Data sent to the server

[1611] Step 3:

[1612] Analyze data using cloud-based AI models

[1613] The server inputs the received image data into an artificial intelligence model (e.g., TensorFlow) to analyze the plant's health and product quality abnormalities. Environmental data is also analyzed in a similar manner.

[1614] Input: Data received by the server

[1615] Output: Detected growth status, presence or absence of pests, or quality abnormalities

[1616] Step 4:

[1617] Create work guidelines and recommendations based on the analysis results

[1618] Based on the analysis results, the server generates feedback including plant growth predictions and optimal countermeasures, as well as suggestions for improving quality at the factory.

[1619] Input: Detection results

[1620] Output: Feedback and suggestions

[1621] Step 5:

[1622] Analyze worker emotions using captured facial expressions and voice

[1623] The terminal captures the worker's facial expression and voice data and sends it to the emotion engine, where the server analyzes the worker's emotional state using an emotion analysis model.

[1624] Input: facial expression and voice data

[1625] Output: Emotional status (stress, satisfaction, etc.)

[1626] Step 6:

[1627] Generate feedback and recommendations based on sentiment data

[1628] Based on the emotional data, the server generates feedback appropriate to the worker's psychological state and suggested actions, such as "take a break."

[1629] Input: Emotional status

[1630] Output: Emotion-based advice and recommendations

[1631] Step 7:

[1632] Send all generated feedback to the device

[1633] The server sends the final feedback, growth forecast, countermeasures, and advice to the worker to the terminal and notifies the user.

[1634] Input: Generated feedback data

[1635] Output: Notification information to the device

[1636] Step 8:

[1637] Users act on feedback

[1638] The user takes necessary measures and actions based on the notifications and advice provided by the device.

[1639] Input: Notification information from the device

[1640] Output: The measures or actions taken

[1641] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1642] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1643] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1644] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1645] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1646] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1647] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1648] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1649] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1650] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1651] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1652] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1653] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1654] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1655] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1656] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1657] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1658] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1659] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1660] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1661] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1662] The following is further disclosed regarding the above embodiment.

[1663] (Claim 1)

[1664] A means for acquiring image data of plants and soil using a camera mounted on a smart device;

[1665] means for communicating with a server equipped with artificial intelligence for analyzing the acquired image data and receiving the analysis results;

[1666] A means for providing feedback to the user on the plant's growth status and the presence or absence of pests based on the received analysis results;

[1667] software means for predicting plant growth and providing optimal treatment;

[1668] A system including:

[1669] (Claim 2)

[1670] 2. The system according to claim 1, further comprising a sensor for collecting data on the plant cultivation environment, transmitting the collected data to a server, and notifying the user of the analysis results.

[1671] (Claim 3)

[1672] 2. The system according to claim 1, further comprising software means for providing plant growth predictions and cultivation missions in a game-like manner to encourage user participation.

[1673] "Example 1"

[1674] (Claim 1)

[1675] A means for acquiring image data of plants and soil using a camera mounted on a smart device;

[1676] means for transmitting the acquired image data together with environmental information at the time of photographing to a server;

[1677] means for analyzing the received image data using an artificial intelligence model;

[1678] A method for providing feedback to users on the plant's growth status, the presence or absence of pests, and signs of disease based on the analysis results.

[1679] A means for generating and informing the user of the best course of action and a report on the state of the plant;

[1680] A means for acquiring temperature, humidity, and pH data using environmental sensors and sending it to a server;

[1681] software means for analyzing the received environmental data and providing plant growth predictions and optimal cultivation methods;

[1682] A system including:

[1683] (Claim 2)

[1684] 2. The system according to claim 1, further comprising an environmental sensor for collecting cultivation environment data, transmitting the collected data to a server, and notifying the user of the analysis results.

[1685] (Claim 3)

[1686] 2. The system according to claim 1, further comprising software means for providing plant growth predictions and cultivation missions in a game-like manner to encourage user participation.

[1687] "Application Example 1"

[1688] (Claim 1)

[1689] A means for acquiring image data of plants and soil using a camera mounted on a smart device;

[1690] means for communicating with a server equipped with artificial intelligence for analyzing the acquired image data and receiving the analysis results;

[1691] A means for providing feedback to the user on the plant's growth status and the presence or absence of pests based on the received analysis results;

[1692] software means for predicting plant growth and providing optimal treatment;

[1693] A software tool that allows users to manage and learn about home gardens in a virtual environment;

[1694] A system including:

[1695] (Claim 2)

[1696] 2. The system according to claim 1, further comprising a sensor for collecting data on the plant cultivation environment, transmitting the collected data to a server, and notifying the user of the analysis results.

[1697] (Claim 3)

[1698] 2. The system according to claim 1, further comprising software means for providing plant growth predictions and cultivation missions in a game-like manner to encourage user participation.

[1699] "Example 2: Combining Emotion Engines"

[1700] (Claim 1)

[1701] A means for acquiring image data of plants and soil using a camera mounted on a smart device;

[1702] means for communicating with a server equipped with artificial intelligence for analyzing the acquired image data and receiving the analysis results;

[1703] means for incorporating an emotion engine that recognizes the user's emotional state and provides feedback and advice based on the recognition result;

[1704] a means for providing feedback to a user on the state of plant growth and the presence or absence of pests based on image data and environmental data;

[1705] software means for predicting plant growth and providing optimal treatment;

[1706] A system including:

[1707] (Claim 2)

[1708] 2. The system according to claim 1, further comprising a sensor for collecting data on the plant cultivation environment, transmitting the collected data to a server, and notifying the user of the analysis results.

[1709] (Claim 3)

[1710] 2. The system according to claim 1, further comprising software means for providing plant growth predictions and cultivation missions in a game-like manner to encourage user participation.

[1711] "Application example 2 when combining emotion engines"

[1712] (Claim 1)

[1713] A means for acquiring image data of plants and soil using a camera mounted on a smart device;

[1714] means for communicating with a server equipped with artificial intelligence for analyzing the acquired image data and receiving the analysis results;

[1715] A means for providing feedback to the user on the plant's growth status and the presence or absence of pests based on the received analysis results;

[1716] software means for predicting plant growth and providing optimal treatment;

[1717] A means for analyzing the emotional state of a worker using an emotion engine installed in a smart device and providing feedback and advice based on that emotion;

[1718] A system including:

[1719] (Claim 2)

[1720] 2. The system according to claim 1, further comprising a sensor for collecting data on the plant cultivation environment, transmitting the collected data to a server, and notifying the user of the analysis results.

[1721] (Claim 3)

[1722] The system of claim 1, comprising software means for providing plant growth predictions and cultivation missions in a game-like manner, encouraging user engagement, and further providing feedback and advice based on the worker's emotional state. [Explanation of symbols]

[1723] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring image data of plants and soil using a camera mounted on a smart device; means for communicating with a server equipped with artificial intelligence for analyzing the acquired image data and receiving the analysis results; A means for providing feedback to the user on the plant's growth status and the presence or absence of pests based on the received analysis results; software means for predicting plant growth and providing optimal treatment; A system including:

2. 2. The system according to claim 1, further comprising a sensor for collecting data on the cultivation environment of the plant, transmitting the collected data to a server and notifying the user of the analysis results.

3. 2. The system according to claim 1, further comprising software means for providing plant growth predictions and cultivation missions in a game-like manner to promote user participation.

Images