system

Abstract

Provide a system. 【Solution means】 Means for photographing a plant using an image acquisition device, Means for recognizing the type and health condition of a plant using an image analysis device, Means for generating a care plan for the plant based on the identified plant information, Means for presenting the generated care plan to the user, Means for receiving and analyzing plant environmental sensor data using a wireless communication device, Control means for performing optimal automatic care for the plant based on the environmental data, Means for checking the health condition of the plant in real time and receiving notifications using smart devices, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern busy lives, it is difficult to properly care for plants. Especially for beginners and people with time constraints, there is a problem that plants may wither due to over - watering or insufficient fertilization. Also, when managing a large number of plants in large - scale commercial facilities, manual care requires labor and the quality of management may not be consistent. As a result, the growth of plants becomes unstable, and there is a problem that sufficient decorative and environmental effects cannot be enjoyed.

Means for Solving the Problems

[0005] This invention provides a system that uses an image acquisition device and an image analysis device to photograph plants and recognize their species and health status. Based on the identified plant information, the system generates an appropriate care plan and presents it to the user. Furthermore, it includes a control device that analyzes environmental sensor data of plants acquired using a wireless communication device and performs optimal automated care based on this analysis. This eliminates inappropriate manual care and enables efficient management while maintaining the plant's health.

[0006] An "image acquisition device" is a device used to capture images of plants, and includes a camera and sensors.

[0007] An "image analysis device" is a hardware and software system that analyzes images of plants to identify their species and health condition.

[0008] "Plant type" refers to classification information that characterizes a particular plant, and is based on genus, species, or other scientific classifications.

[0009] "Health" refers to an indicator of a plant's growth and vitality, and is evaluated based on factors such as leaf color and shape, and growth rate.

[0010] A "care plan" is a plan that includes the schedule and methods of optimal care to be performed for a particular plant.

[0011] A "wireless communication device" is a device used to exchange data over long distances, and uses Wi-Fi, Bluetooth, or other communication protocols.

[0012] "Environmental sensor data" refers to data obtained from sensors that measure the environmental conditions in which plants are placed, and includes temperature, humidity, light intensity, soil moisture content, and other factors.

[0013] "Automated care" refers to the process by which a system automatically implements appropriate measures according to the needs of the plants, including watering and fertilizing.

[0014] A "control device" is a device or software that controls the various equipment necessary to perform automated care and adjusts each step. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention provides a system for automating plant management, enabling even busy users to maintain the health of their plants. The system primarily consists of "terminals" and a "server."

[0037] First, the terminal periodically takes images of the plant using an image acquisition device. The captured images are transmitted to the server using a wireless communication device. The server analyzes the received images with an image analysis device to identify the plant species and its health status. This analysis uses algorithms to check the color and shape of the plant's leaves and the consistency of its growth.

[0038] Once the server identifies the plant type and health status, it generates an optimal care plan based on that information. This care plan specifies the frequency and amount of watering and fertilizing, as well as the necessary sunlight conditions, which are specific to the plant type. The generated care plan is sent from the server to the terminal, where the user can review it through the interface. Users can adjust the care plan settings, including through paid options.

[0039] Next, the server acquires and analyzes plant environmental sensor data in real time. This allows it to monitor daytime sunlight, humidity, and soil moisture levels to determine if the environment is suitable for plant growth. Based on the results, the automated care control system operates, automatically supplying the plants with the necessary water and fertilizer. For example, the automatic watering system may activate at a specific time, and the fertilizer supply system may be activated when environmental sensor readings fall below a certain threshold.

[0040] As a concrete example, when growing houseplants in an office environment, the system takes a picture of the plant at the start of the workday each morning, and the server analyzes its health. A care plan based on the analysis results is sent to the terminal, which the user reviews and approves. Based on the approved care plan, the server automatically waters the plant and maintains the appropriate soil moisture level. In addition, if the terminal instructs the user to keep the plant at a certain distance from the window, the user will do so.

[0041] With this system, users can maintain the health of their plants without needing specialized knowledge, and can also flexibly adapt to changes in the environment.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The device uses its built-in camera to take pictures of plants. The photos are taken automatically according to a specified schedule.

[0045] Step 2:

[0046] The device uses wireless communication to send images of plants it has photographed to a server. This transmission is carried out using a secure protocol.

[0047] Step 3:

[0048] The server receives the images and passes them to an image analysis device to determine the type and health of the plant. The analysis is performed using an algorithm that analyzes plant characteristics such as leaf color, shape, and size.

[0049] Step 4:

[0050] The server generates an optimal care plan based on the image analysis results, determining the amount of watering, fertilizer, and sunlight required according to the plant type and its current health condition.

[0051] Step 5:

[0052] The server sends the care plan information it generates to the terminal. The terminal notifies the user of this information and provides an interface for approving or modifying the plan.

[0053] Step 6:

[0054] The user reviews and approves the care plan they received. If necessary, they can modify the care plan settings using the terminal interface.

[0055] Step 7:

[0056] Based on care plans approved and modified by the user, the server operates IoT devices to control automatic watering and fertilization. Each device operates according to the instructed time and amount.

[0057] Step 8:

[0058] The device acquires data such as temperature, humidity, and light intensity from surrounding environmental sensors and sends it to the server.

[0059] Step 9:

[0060] The server analyzes data from environmental sensors and adjusts the care plan if necessary. The adjustment results are then sent back to the terminal to notify the user.

[0061] Step 10:

[0062] The server stores and analyzes data, updating the knowledge database about plant care. This allows the system to continuously learn and optimize future care plans based on its experience.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] Traditional plant care primarily relies on manual monitoring and maintenance, which is burdensome, especially for busy users. Furthermore, it often requires specialized knowledge, leading to situations where proper management is not possible. Therefore, there is a need for automated management systems that can efficiently and effectively maintain plant health.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server includes means for identifying the characteristics and health status of plants using image analysis means, means for automatically generating a plant management plan based on the identified plant information, and means for acquiring and analyzing environmental data of the plants using a wireless communication device. This makes it possible to automatically perform the management operations necessary to maintain the health of the plants.

[0068] An "image acquisition device" is a device used to capture images of plants, and includes cameras and sensor devices.

[0069] "Image analysis means" refers to software or algorithms that process acquired image data to identify the characteristics and health status of plants.

[0070] "Characteristics" refer to the unique characteristics of a plant, such as its type, shape, and color, and these are the basis for classifying plants.

[0071] "Health status" refers to indicators that show the vitality and growth of a plant, and includes factors such as the color and shape of the leaves and the condition of the roots.

[0072] "Identification" refers to the process of recognizing, appropriately classifying, and evaluating plants based on their characteristics and health status.

[0073] A "care plan" is a plan that outlines how to care for plants in a way that is optimized according to their characteristics and health condition, and includes things like watering frequency, fertilizer amount, and sunlight requirements.

[0074] A "wireless communication device" is a device that transmits and receives data using wireless signals and provides a means of communicating with a server via a network connection.

[0075] "Surrounding environmental data" refers to data that indicates external environmental conditions that affect plant growth, and includes sunlight, humidity, soil moisture, etc.

[0076] This invention is a system for automating and efficiently managing plants, and is primarily composed of a "terminal" and a "server".

[0077] The device is equipped with a camera to acquire images of plants. The device periodically takes pictures of plants and transmits these images to a server using wireless communication. This transmission is carried out using wireless communication technologies such as Wi-Fi and Bluetooth.

[0078] The server uses image analysis software to analyze the received images. Specifically, it uses image processing libraries such as OpenCV and TENSORFLOW® to identify the characteristics and health of the plants. This includes algorithms to determine the color, shape, and growth consistency of the plant's leaves. Based on the identified information, the server automatically generates an optimal care plan. This care plan includes details on watering, fertilization, and required sunlight conditions, and is customized specifically for the characteristics of each plant.

[0079] The generated management plan is sent from the server to the terminal, which then notifies the user using a GUI (Graphical User Interface). The user can then review and modify it as needed.

[0080] Furthermore, environmental sensors installed on the terminal acquire surrounding environmental data in real time. This data includes sunlight, humidity, and soil moisture. The server analyzes this data to check whether the conditions for plant growth are being properly maintained. Based on the environmental data, the server sends control signals to the terminal to automatically activate watering and fertilizer supply devices if necessary.

[0081] As a concrete example, when managing houseplants in an office environment, the terminal takes a picture of the plant at a set time every morning, the data is analyzed on the server, and a management plan is generated. Users can maintain the health of the plants without specialized knowledge simply by reviewing and approving this management plan during their workday.

[0082] An example of a prompt to be input into a generative AI model is, "Analyze recently taken images of plants and suggest the optimal care plan based on their health condition." By using this prompt, the system can provide an appropriate diagnosis and care plan through natural language processing.

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1:

[0085] The device takes pictures of plants at specified time intervals. The camera captures the plants, generates RAW image data, and then processes the data by converting it to JPEG format, preparing the resulting image file. Next, the device uses a wireless communication device to send the JPEG image file to the server.

[0086] Step 2:

[0087] The server receives image files from the terminal and inputs them into image analysis software. OpenCV and TensorFlow are used to analyze the image data. This analysis analyzes leaf color, shape, and surrounding contrast, extracting data related to the plant's characteristics and health. As a result of the analysis, estimates of the plant's species and health are generated.

[0088] Step 3:

[0089] The server automatically generates an appropriate management plan based on the results of image analysis. The generating AI model creates a care plan tailored to the plant type and health condition, outputting a management plan that includes specific instructions regarding watering, fertilization, and sunlight conditions. This plan is organized as data in JSON format.

[0090] Step 4:

[0091] The server sends the generated management plan data to the terminal. The terminal displays the management plan in a GUI based on the received data. The user can review the plan displayed on the terminal and, if there are any questions, can correct the information or check the details through the UI.

[0092] Step 5:

[0093] The device uses environmental sensors to acquire real-time data on the surrounding environment, such as sunlight, humidity, and soil moisture. The analog signals output from the sensors are converted into digital data, and the environmental data is updated at regular intervals.

[0094] Step 6:

[0095] The server analyzes environmental data sent from the terminal and determines the necessary automated management operations based on the data. For example, if humidity is low, it generates a control command to instruct the watering system to operate and sends it to the terminal. The terminal receives this control command and executes the operation of the automatic watering system or fertilizer dispenser.

[0096] Through these steps, the system automates a series of operations necessary to maintain plant health, enabling proper management without requiring users to possess specialized knowledge.

[0097] (Application Example 1)

[0098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] In modern urban environments, maintaining green spaces in public areas and office buildings is crucial, but individually managing numerous plants is time-consuming and requires advanced expertise. Therefore, there is a need for a system that efficiently maintains and improves the health of plants without requiring specialized knowledge.

[0100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0101] In this invention, the server includes means for taking photographs of plants using an image acquisition device, means for recognizing the type and health status of plants using an image analysis device, means for generating a plant care plan based on the identified plant information, and means for presenting the generated care plan to the user, checking the plant's health status in real time using a smart device, and receiving notifications. This makes it possible for users to easily maintain the health of plants without having specialized knowledge.

[0102] An "image acquisition device" is a device used to photograph plants and has the function of acquiring images to visually understand the health status of the plants.

[0103] An "image analysis device" is a device that analyzes acquired images of plants to recognize the type of plant and its health condition.

[0104] A "care plan generation method" is a means for creating an optimal care plan tailored to the type of plant, based on identified plant information.

[0105] "Means of presenting to the user" refers to means of providing information so that the user can review the generated care plan.

[0106] A "wireless communication device" is a communication device that receives environmental sensor data from plants and transmits it to a server.

[0107] A "control means" is a means that has the function of performing optimal automated care for plants based on environmental data.

[0108] A "smart device" is a device that allows users to check the health status of plants in real time and receive notifications, and functions as an interface with the user.

[0109] The system for carrying out this invention is configured to automatically monitor the health of plants and provide optimal care. The main components of the system include an image acquisition device, an image analysis device, a wireless communication device, an environmental sensor, a control means, and a smart device.

[0110] The server periodically takes images of plants using an image acquisition device. The captured images are transmitted to the server via wireless communication, where an image analysis device analyzes them. This process utilizes an AI model and an algorithm that determines the type and health status of the plants. Based on the analysis results, different care plans are generated for each type of plant and presented to the user.

[0111] The smart device notifies the user of the generated care plan. It also receives environmental sensor data to monitor the plant's health in real time and automatically provides appropriate lighting and water. The smart device functions as a user interface, allowing the user to review the generated care plan and make modifications as needed.

[0112] Furthermore, as a concrete example, in the case of houseplants placed in an office lobby, their health can be checked via smartphone or smart glasses, allowing for appropriate care to be provided. This ensures proper maintenance of the plants even when the person in charge is absent.

[0113] An example of a prompt message is as follows: "Analyze the plant in the image to identify its species and health condition, and then generate the optimal care plan."

[0114] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0115] Step 1:

[0116] The device uses an image acquisition system to capture images of plants at regular intervals. This image acquisition is performed at fixed times during the daytime, and the obtained image data is stored in high resolution. The input is an optical signal captured by the camera, and the output is digital image data.

[0117] Step 2:

[0118] The terminal transmits captured images to the server via wireless communication. The input here is the image data stored on the terminal, and the output is the image file sent to the server. Once the server receives the image, it becomes ready for analysis.

[0119] Step 3:

[0120] The server processes the received image data using an image analysis device and analyzes the plant species and health status using an AI model. The input is the transmitted image file, and the output is the analysis result. This result includes the plant species, abnormalities in leaf color and shape, and whether or not there is poor growth.

[0121] Step 4:

[0122] The server uses a generated AI model based on the analysis results to create an optimal care plan for the plants. The input is the image analysis results, and the output is a specific care plan. This plan includes details such as watering and fertilizing frequency, and sunlight conditions.

[0123] Step 5:

[0124] The care plan is sent from the server to the terminal and notified to the user via a smart device. The input is the created care plan, and the output is the information displayed to the user. This allows the user to review the plan and make adjustments as needed.

[0125] Step 6:

[0126] Users can view care plans and monitor environmental sensor data in real time through a user interface on their mobile device. Inputs are real-time sensor data, and outputs are automated care control instructions. For example, watering is automatically performed when soil moisture falls below a certain threshold.

[0127] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0128] This invention combines an emotion engine with a system that supports plant management to realize personalized plant care that responds to the user's emotions. This system mainly consists of a "terminal," a "server," and an "emotion engine."

[0129] First, the user operates a device to take a picture of a plant. This image is transmitted to a server via wireless communication. The server uses an image analysis device to analyze the type and health of the plant. Next, the server uses an emotion engine to collect the user's emotional data. This emotional data is analyzed from, for example, the user's voice tone, facial expressions, and entered text.

[0130] The emotion engine determines the user's current emotional state, such as stress levels and relaxation needs, and based on this, the server generates a plant care plan. This care plan includes basic care based on the plant type and health condition, as well as adjustments that take the user's emotions into account. For example, if the user is feeling stressed, the system may suggest plants or lighting that have stress-reducing effects.

[0131] The generated care plan is sent to the device and presented to the user through an interface. The user can approve or adjust this plan to achieve more personalized care. Once the care plan is approved, the server controls IoT devices to automatically water plants and administer fertilizer. A mechanism is also incorporated to adjust light levels and ambient sounds based on emotions.

[0132] As a concrete example, consider a user who grows houseplants in their office. When this user is particularly busy, the emotion engine detects the user's stress and suggests rearranging the plants or adjusting the lighting to help alleviate stress. Furthermore, automatic watering is performed according to the plant's needs, ensuring optimal care is provided at the user's requested timing.

[0133] This system allows users to easily provide care tailored to their own emotional state, while also enabling plants to grow healthily.

[0134] The following describes the processing flow.

[0135] Step 1:

[0136] The device takes pictures of plants. The pictures are taken according to a set schedule or a manual trigger by the user.

[0137] Step 2:

[0138] The device sends images of plants it has photographed to the server. This transmission process is conducted via a secure connection.

[0139] Step 3:

[0140] The server processes the received images using an analysis device to identify the plant species and its health condition. This analysis uses algorithms that evaluate leaf color, shape, wilting, and other factors.

[0141] Step 4:

[0142] The server collects user emotion data using an emotion engine. This data is obtained from the user's voice tone, facial expressions, and biometric information from connected wearable devices.

[0143] Step 5:

[0144] The emotion engine analyzes the data it acquires to evaluate the user's stress level and emotional state. This allows it to determine the type of care the user desires (e.g., prioritizing relaxation).

[0145] Step 6:

[0146] The server generates a personalized care plan based on plant analysis information and the user's emotional state. This plan incorporates elements based on the user's emotions in addition to basic plant care.

[0147] Step 7:

[0148] The generated care plan is sent to the terminal and presented to the user through the user interface. The user can review the care plan and make manual adjustments as needed.

[0149] Step 8:

[0150] After user approval or adjustment, the server controls IoT devices and automatically performs planned watering and fertilization. It also adjusts the environment (e.g., changes lighting and music) according to the user's emotional state.

[0151] Step 9:

[0152] The device continuously monitors data from environmental sensors and sends it to the server as needed.

[0153] Step 10:

[0154] The server analyzes this environmental data to determine if the care plan needs to be readjusted. If necessary, it sends the adjustment details to the terminal and notifies the user.

[0155] This entire process automatically provides care tailored to the emotional state of the plant and the user, maintaining both happiness and plant health.

[0156] (Example 2)

[0157] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0158] Conventional plant management systems have struggled to adjust plant care based on the user's emotional state, making personalized management difficult. Furthermore, limited features for presenting and modifying care plans made it challenging to provide prompt and optimal plant care. Therefore, there is a need for flexible and effective plant care that responds to the user's emotions.

[0159] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0160] In this invention, the server includes means for recognizing the type and health status of plants, means for analyzing the user's emotional state and generating a care plan that includes adjustments taking the emotional state into account, and means for presenting the generated care plan to the user through a user interface. This makes it possible to quickly provide personalized plant care based on the user's emotions.

[0161] An "image acquisition device" is a device used to photograph plants and acquire them as digital data.

[0162] An "image analysis device" is a system that analyzes acquired images of plants to determine their species and health condition.

[0163] A "generative AI model" is an artificial intelligence system that generates instructions or suggestions tailored to a specific purpose based on collected data.

[0164] A "care plan" is a plan that outlines the management and care methods required for a plant.

[0165] A "user interface" is a means of interaction that allows a user to input and output information from a system.

[0166] A "wireless communication device" is a device used to send and receive data via wireless technology.

[0167] "Emotional state" refers to the user's psychological and emotional state, and is information inferred from voice tone, facial expressions, and text.

[0168] "Environmental sensor data" refers to data that measures the conditions of the environment in which plants are placed, and includes information such as temperature, humidity, and light intensity.

[0169] A "control mechanism" is a mechanism for automatically adjusting and executing plant care.

[0170] This invention is a system that streamlines plant management and provides personalized care based on the user's emotions. The system mainly consists of an image acquisition device, an image analysis device, a server using a generative AI model, a terminal equipped with a user interface, and a wireless communication device.

[0171] The user first uses a terminal to take images of plants using an image acquisition device. The terminal transmits this image data to a server via wireless communication. The server uses an image analysis device to analyze the type and health status of the plants. At this stage, machine learning algorithms are used to evaluate the health status based on the color and shape of the leaves.

[0172] Next, the server uses a generative AI model to analyze the user's emotional state. The device collects the user's voice and facial expression data and sends it to the server. The server recognizes the emotional state from this data and reflects it in the plant care plan. For example, if the user is feeling stressed, the server generates a care plan that suggests stress-relieving plants and environmental settings.

[0173] The generated care plan is presented to the user via the terminal, allowing them to review and modify it as needed. To facilitate this interaction, the user interface is designed to be intuitive and easy to use. This enables users to provide optimized plant care.

[0174] A concrete example is a user who grows houseplants in their office. If this user is busy and stressed, the server will sense this and suggest plant placement and lighting adjustments that promote relaxation. An example of a prompt would be, "Please suggest plant placement that suits my current mood." This allows the user to quickly implement care that is appropriate for their emotions.

[0175] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0176] Step 1:

[0177] The user uses a terminal to capture images of plants with an image acquisition device. These images are stored in the terminal's memory and transmitted to a server using a wireless communication device. The input is a raw image of the plant, and the output is image data sent to the server.

[0178] Step 2:

[0179] The server takes the received images into an image analysis device and analyzes the plant type and health status. In this step, machine learning algorithms are used to process and calculate data based on leaf color and shape. The input is image data, and the output is the analyzed plant information.

[0180] Step 3:

[0181] The server receives user voice and facial expression data transmitted from the terminal and processes it to analyze the emotional state. The input is the user's emotion-related data, and the output is the inferred emotional state. This utilizes speech recognition and facial recognition technologies.

[0182] Step 4:

[0183] The server uses a generative AI model to generate a care plan based on the plant type, health status, and the user's emotional state. The input is pre-collected plant information and emotional state, and the output is a personalized care plan.

[0184] Step 5:

[0185] The generated care plan is presented to the user through the user interface. The user can review this plan and make modifications as needed. The input is care plan information, and the output is the user-approved care plan.

[0186] Step 6:

[0187] The server controls IoT devices and performs plant care according to the approved care plan. The input in this step is the finalized care plan, and the output is the actual plant care actions taken, such as watering or adjusting the light.

[0188] (Application Example 2)

[0189] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0190] In recent years, technologies that support plant growth have attracted attention, but these technologies typically only generate care plans based on plant physiological data and do not take into account the user's emotional state. As a result, general care is provided regardless of the user's mental stress or comfort, leading to the problem that the optimal plant growing environment for the user is not provided.

[0191] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0192] In this invention, the server includes means for taking photographs of plants using an image acquisition device, means for analyzing the user's emotions, and means for generating a care plan based on identified plant information and emotion data. This makes it possible to provide care that is tailored not only to the plant's health but also to the user's emotional state.

[0193] An "image acquisition device" is a device that has the function of taking images of plants and collects basic data for analyzing the type and health status of the plants.

[0194] An "image analysis device" is a device that processes acquired image data to recognize the type and health status of plants.

[0195] "Means of analyzing emotions" refers to methods for collecting user emotional data and analyzing that data to estimate the user's emotional state.

[0196] "Means for generating care plans" refers to means for creating personalized plant care plans based on identified plant information and user emotional data.

[0197] A "wireless communication device" is a device that receives various sensor data from a remote location and communicates to transmit that data to a server.

[0198] "Control means" refers to means of performing control to execute optimal automated care for plants based on environmental data and emotional data.

[0199] The system for implementing this invention primarily uses a terminal, a server, and an emotion engine.

[0200] The terminal is equipped with an image acquisition device and transmits images of plants taken by the user to a server. The server receives this image data using an image analysis device, which identifies the plant species and its health condition. The plant information obtained through this analysis is used as basic data for care plans.

[0201] In parallel, the emotion engine collects emotional data from the user's voice and text input, and analyzes this data to estimate the user's emotional state. This emotional data is also reflected in the care plan.

[0202] Based on this plant information and emotional data, the server generates an individualized care plan for the plants. This plan includes, for example, the timing of automatic watering and fertilizer distribution, and the adjustment of appropriate lighting. The generated care plan is sent back to the terminal and presented to the user. The user can approve or adjust this care plan. Once the final care plan is confirmed based on the user's actions, the server controls IoT devices via wireless communication to perform automated care. This control includes, for example, lighting operation and ambient sound adjustment.

[0203] As a concrete example, imagine a user who grows houseplants in their office. When the user says, "I want to relieve today's stress," the emotion engine analyzes their voice and determines their stress level. Based on this, it suggests plant placement and lighting adjustments. An example of this prompt is, "Please tell me how to arrange plants and set up lighting in a way that has a relaxing effect, suitable for when the user is feeling stressed."

[0204] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0205] Step 1:

[0206] The user takes a picture of a plant using their device. This image data is transmitted to the server using wireless communication. The input is the image of the plant, and the output is the transfer of the image data to the server.

[0207] Step 2:

[0208] The server processes the received image data using an image analysis device. Specifically, it applies an image analysis algorithm to recognize the plant species and health status. The input is image data of the plant, and the output is data representing the plant species and health status.

[0209] Step 3:

[0210] The user inputs emotion-reflecting data into the device. This data can be voice or text, and the device sends this information to the emotion engine. The input is voice or text data, and the output is emotion data.

[0211] Step 4:

[0212] The server analyzes the user's emotional data using an emotion engine. Specifically, it estimates the user's emotional state using a generative AI model. In this process, emotional parameters such as stress levels and the need for relaxation are generated. The output is the analyzed emotional parameters.

[0213] Step 5:

[0214] The server generates a care plan based on plant information and user emotion data. Utilizing a generation AI model, specific plant care methods tailored to the user's emotions are proposed. The output is a care plan, which includes suggestions for automatic watering timing and lighting adjustments.

[0215] Step 6:

[0216] The generated care plan is transmitted wirelessly to the terminal and presented to the user. The user can review this plan on the screen and make modifications as needed. The input is the care plan presented to the user, and the output is the user-approved or modified care plan.

[0217] Step 7:

[0218] Once the user approves the care plan, the server controls IoT devices and implements specific care based on it. For example, it might rearrange plants or adjust lighting. The output is the implementation of automated care tailored to the user's environment.

[0219] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0220] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0221] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0222] [Second Embodiment]

[0223] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0224] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0225] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0226] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0227] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0228] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0229] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0230] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0231] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0232] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0233] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0234] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0235] This invention provides a system for automating plant management, enabling even busy users to maintain the health of their plants. The system primarily consists of "terminals" and a "server."

[0236] First, the terminal periodically takes images of the plant using an image acquisition device. The captured images are transmitted to the server using a wireless communication device. The server analyzes the received images with an image analysis device to identify the plant species and its health status. This analysis uses algorithms to check the color and shape of the plant's leaves and the consistency of its growth.

[0237] Once the server identifies the plant type and health status, it generates an optimal care plan based on that information. This care plan specifies the frequency and amount of watering and fertilizing, as well as the necessary sunlight conditions, which are specific to the plant type. The generated care plan is sent from the server to the terminal, where the user can review it through the interface. Users can adjust the care plan settings, including through paid options.

[0238] Next, the server acquires and analyzes plant environmental sensor data in real time. This allows it to monitor daytime sunlight, humidity, and soil moisture levels to determine if the environment is suitable for plant growth. Based on the results, the automated care control system operates, automatically supplying the plants with the necessary water and fertilizer. For example, the automatic watering system may activate at a specific time, and the fertilizer supply system may be activated when environmental sensor readings fall below a certain threshold.

[0239] As a concrete example, when growing houseplants in an office environment, the system takes a picture of the plant at the start of the workday each morning, and the server analyzes its health. A care plan based on the analysis results is sent to the terminal, which the user reviews and approves. Based on the approved care plan, the server automatically waters the plant and maintains the appropriate soil moisture level. In addition, if the terminal instructs the user to keep the plant at a certain distance from the window, the user will do so.

[0240] With this system, users can maintain the health of their plants without needing specialized knowledge, and can also flexibly adapt to changes in the environment.

[0241] The following describes the processing flow.

[0242] Step 1:

[0243] The device uses its built-in camera to take pictures of plants. The photos are taken automatically according to a specified schedule.

[0244] Step 2:

[0245] The device uses wireless communication to send images of plants it has photographed to a server. This transmission is carried out using a secure protocol.

[0246] Step 3:

[0247] The server receives the images and passes them to an image analysis device to determine the type and health of the plant. The analysis is performed using an algorithm that analyzes plant characteristics such as leaf color, shape, and size.

[0248] Step 4:

[0249] The server generates an optimal care plan based on the image analysis results, determining the amount of watering, fertilizer, and sunlight required according to the plant type and its current health condition.

[0250] Step 5:

[0251] The server sends the care plan information it generates to the terminal. The terminal notifies the user of this information and provides an interface for approving or modifying the plan.

[0252] Step 6:

[0253] The user reviews and approves the care plan they received. If necessary, they can modify the care plan settings using the terminal interface.

[0254] Step 7:

[0255] Based on care plans approved and modified by the user, the server operates IoT devices to control automatic watering and fertilization. Each device operates according to the instructed time and amount.

[0256] Step 8:

[0257] The device acquires data such as temperature, humidity, and light intensity from surrounding environmental sensors and sends it to the server.

[0258] Step 9:

[0259] The server analyzes data from environmental sensors and adjusts the care plan if necessary. The adjustment results are then sent back to the terminal to notify the user.

[0260] Step 10:

[0261] The server stores and analyzes data, updating the knowledge database about plant care. This allows the system to continuously learn and optimize future care plans based on its experience.

[0262] (Example 1)

[0263] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0264] Traditional plant care primarily relies on manual monitoring and maintenance, which is burdensome, especially for busy users. Furthermore, it often requires specialized knowledge, leading to situations where proper management is not possible. Therefore, there is a need for automated management systems that can efficiently and effectively maintain plant health.

[0265] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0266] In this invention, the server includes means for identifying the characteristics and health status of plants using image analysis means, means for automatically generating a plant management plan based on the identified plant information, and means for acquiring and analyzing environmental data of the plants using a wireless communication device. This makes it possible to automatically perform the management operations necessary to maintain the health of the plants.

[0267] An "image acquisition device" is a device used to capture images of plants, and includes cameras and sensor devices.

[0268] "Image analysis means" refers to software or algorithms that process acquired image data to identify the characteristics and health status of plants.

[0269] "Characteristics" refer to the unique characteristics of a plant, such as its type, shape, and color, and these are the basis for classifying plants.

[0270] "Health status" refers to indicators that show the vitality and growth of a plant, and includes factors such as the color and shape of the leaves and the condition of the roots.

[0271] "Identification" refers to the process of recognizing, appropriately classifying, and evaluating plants based on their characteristics and health status.

[0272] A "care plan" is a plan that outlines how to care for plants in a way that is optimized according to their characteristics and health condition, and includes things like watering frequency, fertilizer amount, and sunlight requirements.

[0273] A "wireless communication device" is a device that transmits and receives data using wireless signals and provides a means of communicating with a server via a network connection.

[0274] "Surrounding environmental data" refers to data that indicates external environmental conditions that affect plant growth, and includes sunlight, humidity, soil moisture, etc.

[0275] This invention is a system for automating and efficiently managing plants, and is primarily composed of a "terminal" and a "server".

[0276] The device is equipped with a camera to acquire images of plants. The device periodically takes pictures of plants and transmits these images to a server using wireless communication. This transmission is carried out using wireless communication technologies such as Wi-Fi and Bluetooth.

[0277] The server uses image analysis software to analyze the received images. Specifically, it uses image processing libraries such as OpenCV and TensorFlow to identify the characteristics and health of the plants. This includes algorithms to determine the color, shape, and growth consistency of the plant's leaves. Based on the identified information, the server automatically generates an optimal care plan. This care plan includes details on watering, fertilization, and required sunlight conditions, and is customized specifically for the characteristics of each plant.

[0278] The generated management plan is sent from the server to the terminal, which then notifies the user using a GUI (Graphical User Interface). The user can then review and modify it as needed.

[0279] Furthermore, environmental sensors installed on the terminal acquire surrounding environmental data in real time. This data includes sunlight, humidity, and soil moisture. The server analyzes this data to check whether the conditions for plant growth are being properly maintained. Based on the environmental data, the server sends control signals to the terminal to automatically activate watering and fertilizer supply devices if necessary.

[0280] As a concrete example, when managing houseplants in an office environment, the terminal takes a picture of the plant at a set time every morning, the data is analyzed on the server, and a management plan is generated. Users can maintain the health of the plants without specialized knowledge simply by reviewing and approving this management plan during their workday.

[0281] An example of a prompt to be input into a generative AI model is, "Analyze recently taken images of plants and suggest the optimal care plan based on their health condition." By using this prompt, the system can provide an appropriate diagnosis and care plan through natural language processing.

[0282] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0283] Step 1:

[0284] The terminal captures images of the plant at specified time intervals. The camera captures the plant, performs data processing such as generating RAW image data and converting it to JPEG format, and prepares the resulting image file. Next, the terminal uses a wireless communication device to transmit the JPEG image file to the server.

[0285] Step 2:

[0286] The server obtains the image file received from the terminal and inputs it into image analysis software. As image analysis means, it uses OpenCV or TensorFlow to perform analysis of the image data. There, it analyzes the color, shape, and contrast of the leaves to extract data regarding the characteristics and health status of the plant. As a result of the analysis, estimated values for the type and health status of the plant are generated.

[0287] Step 3:

[0288] The server automatically generates an appropriate management plan based on the results of the image analysis. The generated AI model creates a care plan according to the type and health status of the plant and outputs a management plan including specific instructions regarding watering, fertilization, and sunlight conditions. This plan is organized as data in JSON format.

[0289] Step 4:

[0290] The server transmits the generated management plan data to the terminal. The terminal displays the management plan on the GUI based on the received data. The user can check the plan displayed on the terminal and, if there are any unclear points, can modify the information and check the details through the UI.

[0291] Step 5:

[0292] The device uses environmental sensors to acquire real-time data on the surrounding environment, such as sunlight, humidity, and soil moisture. The analog signals output from the sensors are converted into digital data, and the environmental data is updated at regular intervals.

[0293] Step 6:

[0294] The server analyzes environmental data sent from the terminal and determines the necessary automated management operations based on the data. For example, if humidity is low, it generates a control command to instruct the watering system to operate and sends it to the terminal. The terminal receives this control command and executes the operation of the automatic watering system or fertilizer dispenser.

[0295] Through these steps, the system automates a series of operations necessary to maintain plant health, enabling proper management without requiring users to possess specialized knowledge.

[0296] (Application Example 1)

[0297] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0298] In modern urban environments, maintaining green spaces in public areas and office buildings is crucial, but individually managing numerous plants is time-consuming and requires advanced expertise. Therefore, there is a need for a system that efficiently maintains and improves the health of plants without requiring specialized knowledge.

[0299] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0300] In this invention, the server includes means for photographing a plant using an image acquisition device, means for recognizing the type and health status of the plant using an image analysis device, means for generating a care plan for the plant based on the identified plant information, and means for presenting the generated care plan to the user and using a smart device to check the health status of the plant in real time and receive notifications. As a result, even without specialized knowledge, the user can easily maintain the health of the plant.

[0301] The "image acquisition device" is a device for photographing a plant and has a function of acquiring an image for visually grasping the health status of the plant.

[0302] The "image analysis device" is a device for analyzing the acquired image of the plant and recognizing the type and health status of the plant.

[0303] The "care plan generation means" is means for creating an optimal care plan according to the type of the plant based on the identified plant information.

[0304] The "means for presenting to the user" is means for providing information so that the user can view the generated care plan.

[0305] The "wireless communication device" is a communication device for receiving environmental sensor data of the plant and transmitting it to the server.

[0306] The "control means" is means having a function of performing optimal automatic care for the plant based on environmental data.

[0307] The "smart device" is a device that can check the health status of the plant in real time and receive notifications and functions as an interface with the user.

[0308] The system for carrying out this invention is configured to automatically monitor the health of plants and provide optimal care. The main components of the system include an image acquisition device, an image analysis device, a wireless communication device, an environmental sensor, a control means, and a smart device.

[0309] The server periodically takes images of plants using an image acquisition device. The captured images are transmitted to the server via wireless communication, where an image analysis device analyzes them. This process utilizes an AI model and an algorithm that determines the type and health status of the plants. Based on the analysis results, different care plans are generated for each type of plant and presented to the user.

[0310] The smart device notifies the user of the generated care plan. It also receives environmental sensor data to monitor the plant's health in real time and automatically provides appropriate lighting and water. The smart device functions as a user interface, allowing the user to review the generated care plan and make modifications as needed.

[0311] Furthermore, as a concrete example, in the case of houseplants placed in an office lobby, their health can be checked via smartphone or smart glasses, allowing for appropriate care to be provided. This ensures proper maintenance of the plants even when the person in charge is absent.

[0312] An example of a prompt message is as follows: "Analyze the plant in the image to identify its species and health condition, and then generate the optimal care plan."

[0313] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0314] Step 1:

[0315] The device uses an image acquisition system to capture images of plants at regular intervals. This image acquisition is performed at fixed times during the daytime, and the obtained image data is stored in high resolution. The input is an optical signal captured by the camera, and the output is digital image data.

[0316] Step 2:

[0317] The terminal transmits captured images to the server via wireless communication. The input here is the image data stored on the terminal, and the output is the image file sent to the server. Once the server receives the image, it becomes ready for analysis.

[0318] Step 3:

[0319] The server processes the received image data using an image analysis device and analyzes the plant species and health status using an AI model. The input is the transmitted image file, and the output is the analysis result. This result includes the plant species, abnormalities in leaf color and shape, and whether or not there is poor growth.

[0320] Step 4:

[0321] The server uses a generated AI model based on the analysis results to create an optimal care plan for the plants. The input is the image analysis results, and the output is a specific care plan. This plan includes details such as watering and fertilizing frequency, and sunlight conditions.

[0322] Step 5:

[0323] The care plan is sent from the server to the terminal and notified to the user via a smart device. The input is the created care plan, and the output is the information displayed to the user. This allows the user to review the plan and make adjustments as needed.

[0324] Step 6:

[0325] Users can view care plans and monitor environmental sensor data in real time through a user interface on their mobile device. Inputs are real-time sensor data, and outputs are automated care control instructions. For example, watering is automatically performed when soil moisture falls below a certain threshold.

[0326] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0327] This invention combines an emotion engine with a system that supports plant management to realize personalized plant care that responds to the user's emotions. This system mainly consists of a "terminal," a "server," and an "emotion engine."

[0328] First, the user operates a device to take a picture of a plant. This image is transmitted to a server via wireless communication. The server uses an image analysis device to analyze the type and health of the plant. Next, the server uses an emotion engine to collect the user's emotional data. This emotional data is analyzed from, for example, the user's voice tone, facial expressions, and entered text.

[0329] The emotion engine determines the user's current emotional state, such as stress levels and relaxation needs, and based on this, the server generates a plant care plan. This care plan includes basic care based on the plant type and health condition, as well as adjustments that take the user's emotions into account. For example, if the user is feeling stressed, the system may suggest plants or lighting that have stress-reducing effects.

[0330] The generated care plan is sent to the device and presented to the user through an interface. The user can approve or adjust this plan to achieve more personalized care. Once the care plan is approved, the server controls IoT devices to automatically water plants and administer fertilizer. A mechanism is also incorporated to adjust light levels and ambient sounds based on emotions.

[0331] As a concrete example, consider a user who grows houseplants in their office. When this user is particularly busy, the emotion engine detects the user's stress and suggests rearranging the plants or adjusting the lighting to help alleviate stress. Furthermore, automatic watering is performed according to the plant's needs, ensuring optimal care is provided at the user's requested timing.

[0332] This system allows users to easily provide care tailored to their own emotional state, while also enabling plants to grow healthily.

[0333] The following describes the processing flow.

[0334] Step 1:

[0335] The device takes pictures of plants. The pictures are taken according to a set schedule or a manual trigger by the user.

[0336] Step 2:

[0337] The device sends images of plants it has photographed to the server. This transmission process is conducted via a secure connection.

[0338] Step 3:

[0339] The server processes the received images using an analysis device to identify the plant species and its health condition. This analysis uses algorithms that evaluate leaf color, shape, wilting, and other factors.

[0340] Step 4:

[0341] The server collects user emotion data using an emotion engine. This data is obtained from the user's voice tone, facial expressions, and biometric information from connected wearable devices.

[0342] Step 5:

[0343] The emotion engine analyzes the data it acquires to evaluate the user's stress level and emotional state. This allows it to determine the type of care the user desires (e.g., prioritizing relaxation).

[0344] Step 6:

[0345] The server generates a personalized care plan based on plant analysis information and the user's emotional state. This plan incorporates elements based on the user's emotions in addition to basic plant care.

[0346] Step 7:

[0347] The generated care plan is sent to the terminal and presented to the user through the user interface. The user can review the care plan and make manual adjustments as needed.

[0348] Step 8:

[0349] After user approval or adjustment, the server controls IoT devices and automatically performs planned watering and fertilization. It also adjusts the environment (e.g., changes lighting and music) according to the user's emotional state.

[0350] Step 9:

[0351] The device continuously monitors data from environmental sensors and sends it to the server as needed.

[0352] Step 10:

[0353] The server analyzes this environmental data to determine if the care plan needs to be readjusted. If necessary, it sends the adjustment details to the terminal and notifies the user.

[0354] This entire process automatically provides care tailored to the emotional state of the plant and the user, maintaining both happiness and plant health.

[0355] (Example 2)

[0356] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0357] Conventional plant management systems have struggled to adjust plant care based on the user's emotional state, making personalized management difficult. Furthermore, limited features for presenting and modifying care plans made it challenging to provide prompt and optimal plant care. Therefore, there is a need for flexible and effective plant care that responds to the user's emotions.

[0358] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0359] In this invention, the server includes means for recognizing the type and health status of plants, means for analyzing the user's emotional state and generating a care plan that includes adjustments taking the emotional state into account, and means for presenting the generated care plan to the user through a user interface. This makes it possible to quickly provide personalized plant care based on the user's emotions.

[0360] An "image acquisition device" is a device used to photograph plants and acquire them as digital data.

[0361] An "image analysis device" is a system that analyzes acquired images of plants to determine their species and health condition.

[0362] A "generative AI model" is an artificial intelligence system that generates instructions or suggestions tailored to a specific purpose based on collected data.

[0363] A "care plan" is a plan that outlines the management and care methods required for a plant.

[0364] A "user interface" is a means of interaction that allows a user to input and output information from a system.

[0365] A "wireless communication device" is a device used to send and receive data via wireless technology.

[0366] "Emotional state" refers to the user's psychological and emotional state, and is information inferred from voice tone, facial expressions, and text.

[0367] "Environmental sensor data" refers to data that measures the conditions of the environment in which plants are placed, and includes information such as temperature, humidity, and light intensity.

[0368] A "control mechanism" is a mechanism for automatically adjusting and executing plant care.

[0369] This invention is a system that streamlines plant management and provides personalized care based on the user's emotions. The system mainly consists of an image acquisition device, an image analysis device, a server using a generative AI model, a terminal equipped with a user interface, and a wireless communication device.

[0370] The user first uses a terminal to take images of plants using an image acquisition device. The terminal transmits this image data to a server via wireless communication. The server uses an image analysis device to analyze the type and health status of the plants. At this stage, machine learning algorithms are used to evaluate the health status based on the color and shape of the leaves.

[0371] Next, the server uses a generative AI model to analyze the user's emotional state. The device collects the user's voice and facial expression data and sends it to the server. The server recognizes the emotional state from this data and reflects it in the plant care plan. For example, if the user is feeling stressed, the server generates a care plan that suggests stress-relieving plants and environmental settings.

[0372] The generated care plan is presented to the user via the terminal, allowing them to review and modify it as needed. To facilitate this interaction, the user interface is designed to be intuitive and easy to use. This enables users to provide optimized plant care.

[0373] A concrete example is a user who grows houseplants in their office. If this user is busy and stressed, the server will sense this and suggest plant placement and lighting adjustments that promote relaxation. An example of a prompt would be, "Please suggest plant placement that suits my current mood." This allows the user to quickly implement care that is appropriate for their emotions.

[0374] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0375] Step 1:

[0376] The user uses a terminal to capture images of plants with an image acquisition device. These images are stored in the terminal's memory and transmitted to a server using a wireless communication device. The input is a raw image of the plant, and the output is image data sent to the server.

[0377] Step 2:

[0378] The server takes the received images into an image analysis device and analyzes the plant type and health status. In this step, machine learning algorithms are used to process and calculate data based on leaf color and shape. The input is image data, and the output is the analyzed plant information.

[0379] Step 3:

[0380] The server receives user voice and facial expression data transmitted from the terminal and processes it to analyze the emotional state. The input is the user's emotion-related data, and the output is the inferred emotional state. This utilizes speech recognition and facial recognition technologies.

[0381] Step 4:

[0382] The server uses a generative AI model to generate a care plan based on the plant type, health status, and the user's emotional state. The input is pre-collected plant information and emotional state, and the output is a personalized care plan.

[0383] Step 5:

[0384] The generated care plan is presented to the user through the user interface. The user can review this plan and make modifications as needed. The input is care plan information, and the output is the user-approved care plan.

[0385] Step 6:

[0386] The server controls IoT devices and performs plant care according to the approved care plan. The input in this step is the finalized care plan, and the output is the actual plant care actions taken, such as watering or adjusting the light.

[0387] (Application Example 2)

[0388] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0389] In recent years, technologies that support plant growth have attracted attention, but these technologies typically only generate care plans based on plant physiological data and do not take into account the user's emotional state. As a result, general care is provided regardless of the user's mental stress or comfort, leading to the problem that the optimal plant growing environment for the user is not provided.

[0390] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0391] In this invention, the server includes means for taking photographs of plants using an image acquisition device, means for analyzing the user's emotions, and means for generating a care plan based on identified plant information and emotion data. This makes it possible to provide care that is tailored not only to the plant's health but also to the user's emotional state.

[0392] An "image acquisition device" is a device that has the function of taking images of plants and collects basic data for analyzing the type and health status of the plants.

[0393] An "image analysis device" is a device that processes acquired image data to recognize the type and health status of plants.

[0394] "Means of analyzing emotions" refers to methods for collecting user emotional data and analyzing that data to estimate the user's emotional state.

[0395] "Means for generating care plans" refers to means for creating personalized plant care plans based on identified plant information and user emotional data.

[0396] A "wireless communication device" is a device that receives various sensor data from a remote location and communicates to transmit that data to a server.

[0397] "Control means" refers to means of performing control to execute optimal automated care for plants based on environmental data and emotional data.

[0398] The system for implementing this invention primarily uses a terminal, a server, and an emotion engine.

[0399] The terminal is equipped with an image acquisition device and transmits images of plants taken by the user to a server. The server receives this image data using an image analysis device, which identifies the plant species and its health condition. The plant information obtained through this analysis is used as basic data for care plans.

[0400] In parallel, the emotion engine collects emotional data from the user's voice and text input, and analyzes this data to estimate the user's emotional state. This emotional data is also reflected in the care plan.

[0401] Based on this plant information and emotional data, the server generates an individualized care plan for the plants. This plan includes, for example, the timing of automatic watering and fertilizer distribution, and the adjustment of appropriate lighting. The generated care plan is sent back to the terminal and presented to the user. The user can approve or adjust this care plan. Once the final care plan is confirmed based on the user's actions, the server controls IoT devices via wireless communication to perform automated care. This control includes, for example, lighting operation and ambient sound adjustment.

[0402] As a concrete example, imagine a user who grows houseplants in their office. When the user says, "I want to relieve today's stress," the emotion engine analyzes their voice and determines their stress level. Based on this, it suggests plant placement and lighting adjustments. An example of this prompt is, "Please tell me how to arrange plants and set up lighting in a way that has a relaxing effect, suitable for when the user is feeling stressed."

[0403] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0404] Step 1:

[0405] The user takes a picture of a plant using their device. This image data is transmitted to the server using wireless communication. The input is the image of the plant, and the output is the transfer of the image data to the server.

[0406] Step 2:

[0407] The server processes the received image data using an image analysis device. Specifically, it applies an image analysis algorithm to recognize the plant species and health status. The input is image data of the plant, and the output is data representing the plant species and health status.

[0408] Step 3:

[0409] The user inputs emotion-reflecting data into the device. This data can be voice or text, and the device sends this information to the emotion engine. The input is voice or text data, and the output is emotion data.

[0410] Step 4:

[0411] The server analyzes the user's emotional data using an emotion engine. Specifically, it estimates the user's emotional state using a generative AI model. In this process, emotional parameters such as stress levels and the need for relaxation are generated. The output is the analyzed emotional parameters.

[0412] Step 5:

[0413] The server generates a care plan based on plant information and user emotion data. Utilizing a generation AI model, specific plant care methods tailored to the user's emotions are proposed. The output is a care plan, which includes suggestions for automatic watering timing and lighting adjustments.

[0414] Step 6:

[0415] The generated care plan is transmitted wirelessly to the terminal and presented to the user. The user can review this plan on the screen and make modifications as needed. The input is the care plan presented to the user, and the output is the user-approved or modified care plan.

[0416] Step 7:

[0417] Once the user approves the care plan, the server controls IoT devices and implements specific care based on it. For example, it might rearrange plants or adjust lighting. The output is the implementation of automated care tailored to the user's environment.

[0418] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0419] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0420] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0421] [Third Embodiment]

[0422] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0423] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0424] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0425] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0426] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0427] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0428] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0429] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0430] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0431] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0432] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0433] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0434] This invention provides a system for automating plant management, enabling even busy users to maintain the health of their plants. The system primarily consists of "terminals" and a "server."

[0435] First, the terminal periodically takes images of the plant using an image acquisition device. The captured images are transmitted to the server using a wireless communication device. The server analyzes the received images with an image analysis device to identify the plant species and its health status. This analysis uses algorithms to check the color and shape of the plant's leaves and the consistency of its growth.

[0436] Once the server identifies the plant type and health status, it generates an optimal care plan based on that information. This care plan specifies the frequency and amount of watering and fertilizing, as well as the necessary sunlight conditions, which are specific to the plant type. The generated care plan is sent from the server to the terminal, where the user can review it through the interface. Users can adjust the care plan settings, including through paid options.

[0437] Next, the server acquires and analyzes plant environmental sensor data in real time. This allows it to monitor daytime sunlight, humidity, and soil moisture levels to determine if the environment is suitable for plant growth. Based on the results, the automated care control system operates, automatically supplying the plants with the necessary water and fertilizer. For example, the automatic watering system may activate at a specific time, and the fertilizer supply system may be activated when environmental sensor readings fall below a certain threshold.

[0438] As a concrete example, when growing houseplants in an office environment, the system takes a picture of the plant at the start of the workday each morning, and the server analyzes its health. A care plan based on the analysis results is sent to the terminal, which the user reviews and approves. Based on the approved care plan, the server automatically waters the plant and maintains the appropriate soil moisture level. In addition, if the terminal instructs the user to keep the plant at a certain distance from the window, the user will do so.

[0439] With this system, users can maintain the health of their plants without needing specialized knowledge, and can also flexibly adapt to changes in the environment.

[0440] The following describes the processing flow.

[0441] Step 1:

[0442] The device uses its built-in camera to take pictures of plants. The photos are taken automatically according to a specified schedule.

[0443] Step 2:

[0444] The device uses wireless communication to send images of plants it has photographed to a server. This transmission is carried out using a secure protocol.

[0445] Step 3:

[0446] The server receives the images and passes them to an image analysis device to determine the type and health of the plant. The analysis is performed using an algorithm that analyzes plant characteristics such as leaf color, shape, and size.

[0447] Step 4:

[0448] The server generates an optimal care plan based on the image analysis results, determining the amount of watering, fertilizer, and sunlight required according to the plant type and its current health condition.

[0449] Step 5:

[0450] The server sends the care plan information it generates to the terminal. The terminal notifies the user of this information and provides an interface for approving or modifying the plan.

[0451] Step 6:

[0452] The user reviews and approves the care plan they received. If necessary, they can modify the care plan settings using the terminal interface.

[0453] Step 7:

[0454] Based on care plans approved and modified by the user, the server operates IoT devices to control automatic watering and fertilization. Each device operates according to the instructed time and amount.

[0455] Step 8:

[0456] The device acquires data such as temperature, humidity, and light intensity from surrounding environmental sensors and sends it to the server.

[0457] Step 9:

[0458] The server analyzes data from environmental sensors and adjusts the care plan if necessary. The adjustment results are then sent back to the terminal to notify the user.

[0459] Step 10:

[0460] The server stores and analyzes data, updating the knowledge database about plant care. This allows the system to continuously learn and optimize future care plans based on its experience.

[0461] (Example 1)

[0462] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0463] Traditional plant care primarily relies on manual monitoring and maintenance, which is burdensome, especially for busy users. Furthermore, it often requires specialized knowledge, leading to situations where proper management is not possible. Therefore, there is a need for automated management systems that can efficiently and effectively maintain plant health.

[0464] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0465] In this invention, the server includes means for identifying the characteristics and health status of plants using image analysis means, means for automatically generating a plant management plan based on the identified plant information, and means for acquiring and analyzing environmental data of the plants using a wireless communication device. This makes it possible to automatically perform the management operations necessary to maintain the health of the plants.

[0466] An "image acquisition device" is a device used to capture images of plants, and includes cameras and sensor devices.

[0467] "Image analysis means" refers to software or algorithms that process acquired image data to identify the characteristics and health status of plants.

[0468] "Characteristics" refer to the unique characteristics of a plant, such as its type, shape, and color, and these are the basis for classifying plants.

[0469] "Health status" refers to indicators that show the vitality and growth of a plant, and includes factors such as the color and shape of the leaves and the condition of the roots.

[0470] "Identification" refers to the process of recognizing, appropriately classifying, and evaluating plants based on their characteristics and health status.

[0471] A "care plan" is a plan that outlines how to care for plants in a way that is optimized according to their characteristics and health condition, and includes things like watering frequency, fertilizer amount, and sunlight requirements.

[0472] A "wireless communication device" is a device that transmits and receives data using wireless signals and provides a means of communicating with a server via a network connection.

[0473] "Surrounding environmental data" refers to data that indicates external environmental conditions that affect plant growth, and includes sunlight, humidity, soil moisture, etc.

[0474] This invention is a system for automating and efficiently managing plants, and is primarily composed of a "terminal" and a "server".

[0475] The device is equipped with a camera to acquire images of plants. The device periodically takes pictures of plants and transmits these images to a server using wireless communication. This transmission is carried out using wireless communication technologies such as Wi-Fi and Bluetooth.

[0476] The server uses image analysis software to analyze the received images. Specifically, it uses image processing libraries such as OpenCV and TensorFlow to identify the characteristics and health of the plants. This includes algorithms to determine the color, shape, and growth consistency of the plant's leaves. Based on the identified information, the server automatically generates an optimal care plan. This care plan includes details on watering, fertilization, and required sunlight conditions, and is customized specifically for the characteristics of each plant.

[0477] The generated management plan is sent from the server to the terminal, which then notifies the user using a GUI (Graphical User Interface). The user can then review and modify it as needed.

[0478] Furthermore, environmental sensors installed on the terminal acquire surrounding environmental data in real time. This data includes sunlight, humidity, and soil moisture. The server analyzes this data to check whether the conditions for plant growth are being properly maintained. Based on the environmental data, the server sends control signals to the terminal to automatically activate watering and fertilizer supply devices if necessary.

[0479] As a concrete example, when managing houseplants in an office environment, the terminal takes a picture of the plant at a set time every morning, the data is analyzed on the server, and a management plan is generated. Users can maintain the health of the plants without specialized knowledge simply by reviewing and approving this management plan during their workday.

[0480] An example of a prompt to be input into a generative AI model is, "Analyze recently taken images of plants and suggest the optimal care plan based on their health condition." By using this prompt, the system can provide an appropriate diagnosis and care plan through natural language processing.

[0481] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0482] Step 1:

[0483] The device takes pictures of plants at specified time intervals. The camera captures the plants, generates RAW image data, and then processes the data by converting it to JPEG format, preparing the resulting image file. Next, the device uses a wireless communication device to send the JPEG image file to the server.

[0484] Step 2:

[0485] The server receives image files from the terminal and inputs them into image analysis software. OpenCV and TensorFlow are used to analyze the image data. This analysis analyzes leaf color, shape, and surrounding contrast, extracting data related to the plant's characteristics and health. As a result of the analysis, estimates of the plant's species and health are generated.

[0486] Step 3:

[0487] The server automatically generates an appropriate management plan based on the results of image analysis. The generating AI model creates a care plan tailored to the plant type and health condition, outputting a management plan that includes specific instructions regarding watering, fertilization, and sunlight conditions. This plan is organized as data in JSON format.

[0488] Step 4:

[0489] The server sends the generated management plan data to the terminal. The terminal displays the management plan in a GUI based on the received data. The user can review the plan displayed on the terminal and, if there are any questions, can correct the information or check the details through the UI.

[0490] Step 5:

[0491] The device uses environmental sensors to acquire real-time data on the surrounding environment, such as sunlight, humidity, and soil moisture. The analog signals output from the sensors are converted into digital data, and the environmental data is updated at regular intervals.

[0492] Step 6:

[0493] The server analyzes environmental data sent from the terminal and determines the necessary automated management operations based on the data. For example, if humidity is low, it generates a control command to instruct the watering system to operate and sends it to the terminal. The terminal receives this control command and executes the operation of the automatic watering system or fertilizer dispenser.

[0494] Through these steps, the system automates a series of operations necessary to maintain plant health, enabling proper management without requiring users to possess specialized knowledge.

[0495] (Application Example 1)

[0496] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0497] In modern urban environments, maintaining green spaces in public areas and office buildings is crucial, but individually managing numerous plants is time-consuming and requires advanced expertise. Therefore, there is a need for a system that efficiently maintains and improves the health of plants without requiring specialized knowledge.

[0498] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0499] In this invention, the server includes means for taking photographs of plants using an image acquisition device, means for recognizing the type and health status of plants using an image analysis device, means for generating a plant care plan based on the identified plant information, and means for presenting the generated care plan to the user, checking the plant's health status in real time using a smart device, and receiving notifications. This makes it possible for users to easily maintain the health of plants without having specialized knowledge.

[0500] An "image acquisition device" is a device used to photograph plants and has the function of acquiring images to visually understand the health status of the plants.

[0501] An "image analysis device" is a device that analyzes acquired images of plants to recognize the type of plant and its health condition.

[0502] A "care plan generation method" is a means for creating an optimal care plan tailored to the type of plant, based on identified plant information.

[0503] "Means of presenting to the user" refers to means of providing information so that the user can review the generated care plan.

[0504] A "wireless communication device" is a communication device that receives environmental sensor data from plants and transmits it to a server.

[0505] A "control means" is a means that has the function of performing optimal automated care for plants based on environmental data.

[0506] A "smart device" is a device that allows users to check the health status of plants in real time and receive notifications, and functions as an interface with the user.

[0507] The system for carrying out this invention is configured to automatically monitor the health of plants and provide optimal care. The main components of the system include an image acquisition device, an image analysis device, a wireless communication device, an environmental sensor, a control means, and a smart device.

[0508] The server periodically takes images of plants using an image acquisition device. The captured images are transmitted to the server via wireless communication, where an image analysis device analyzes them. This process utilizes an AI model and an algorithm that determines the type and health status of the plants. Based on the analysis results, different care plans are generated for each type of plant and presented to the user.

[0509] The smart device notifies the user of the generated care plan. It also receives environmental sensor data to monitor the plant's health in real time and automatically provides appropriate lighting and water. The smart device functions as a user interface, allowing the user to review the generated care plan and make modifications as needed.

[0510] Furthermore, as a concrete example, in the case of houseplants placed in an office lobby, their health can be checked via smartphone or smart glasses, allowing for appropriate care to be provided. This ensures proper maintenance of the plants even when the person in charge is absent.

[0511] An example of a prompt message is as follows: "Analyze the plant in the image to identify its species and health condition, and then generate the optimal care plan."

[0512] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0513] Step 1:

[0514] The device uses an image acquisition system to capture images of plants at regular intervals. This image acquisition is performed at fixed times during the daytime, and the obtained image data is stored in high resolution. The input is an optical signal captured by the camera, and the output is digital image data.

[0515] Step 2:

[0516] The terminal transmits captured images to the server via wireless communication. The input here is the image data stored on the terminal, and the output is the image file sent to the server. Once the server receives the image, it becomes ready for analysis.

[0517] Step 3:

[0518] The server processes the received image data using an image analysis device and analyzes the plant species and health status using an AI model. The input is the transmitted image file, and the output is the analysis result. This result includes the plant species, abnormalities in leaf color and shape, and whether or not there is poor growth.

[0519] Step 4:

[0520] The server uses a generated AI model based on the analysis results to create an optimal care plan for the plants. The input is the image analysis results, and the output is a specific care plan. This plan includes details such as watering and fertilizing frequency, and sunlight conditions.

[0521] Step 5:

[0522] The care plan is sent from the server to the terminal and notified to the user via a smart device. The input is the created care plan, and the output is the information displayed to the user. This allows the user to review the plan and make adjustments as needed.

[0523] Step 6:

[0524] Users can view care plans and monitor environmental sensor data in real time through a user interface on their mobile device. Inputs are real-time sensor data, and outputs are automated care control instructions. For example, watering is automatically performed when soil moisture falls below a certain threshold.

[0525] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0526] This invention combines an emotion engine with a system that supports plant management to realize personalized plant care that responds to the user's emotions. This system mainly consists of a "terminal," a "server," and an "emotion engine."

[0527] First, the user operates a device to take a picture of a plant. This image is transmitted to a server via wireless communication. The server uses an image analysis device to analyze the type and health of the plant. Next, the server uses an emotion engine to collect the user's emotional data. This emotional data is analyzed from, for example, the user's voice tone, facial expressions, and entered text.

[0528] The emotion engine determines the user's current emotional state, such as stress levels and relaxation needs, and based on this, the server generates a plant care plan. This care plan includes basic care based on the plant type and health condition, as well as adjustments that take the user's emotions into account. For example, if the user is feeling stressed, the system may suggest plants or lighting that have stress-reducing effects.

[0529] The generated care plan is sent to the device and presented to the user through an interface. The user can approve or adjust this plan to achieve more personalized care. Once the care plan is approved, the server controls IoT devices to automatically water plants and administer fertilizer. A mechanism is also incorporated to adjust light levels and ambient sounds based on emotions.

[0530] As a concrete example, consider a user who grows houseplants in their office. When this user is particularly busy, the emotion engine detects the user's stress and suggests rearranging the plants or adjusting the lighting to help alleviate stress. Furthermore, automatic watering is performed according to the plant's needs, ensuring optimal care is provided at the user's requested timing.

[0531] This system allows users to easily provide care tailored to their own emotional state, while also enabling plants to grow healthily.

[0532] The following describes the processing flow.

[0533] Step 1:

[0534] The device takes pictures of plants. The pictures are taken according to a set schedule or a manual trigger by the user.

[0535] Step 2:

[0536] The device sends images of plants it has photographed to the server. This transmission process is conducted via a secure connection.

[0537] Step 3:

[0538] The server processes the received images using an analysis device to identify the plant species and its health condition. This analysis uses algorithms that evaluate leaf color, shape, wilting, and other factors.

[0539] Step 4:

[0540] The server collects user emotion data using an emotion engine. This data is obtained from the user's voice tone, facial expressions, and biometric information from connected wearable devices.

[0541] Step 5:

[0542] The emotion engine analyzes the data it acquires to evaluate the user's stress level and emotional state. This allows it to determine the type of care the user desires (e.g., prioritizing relaxation).

[0543] Step 6:

[0544] The server generates a personalized care plan based on plant analysis information and the user's emotional state. This plan incorporates elements based on the user's emotions in addition to basic plant care.

[0545] Step 7:

[0546] The generated care plan is sent to the terminal and presented to the user through the user interface. The user can review the care plan and make manual adjustments as needed.

[0547] Step 8:

[0548] After user approval or adjustment, the server controls IoT devices and automatically performs planned watering and fertilization. It also adjusts the environment (e.g., changes lighting and music) according to the user's emotional state.

[0549] Step 9:

[0550] The device continuously monitors data from environmental sensors and sends it to the server as needed.

[0551] Step 10:

[0552] The server analyzes this environmental data to determine if the care plan needs to be readjusted. If necessary, it sends the adjustment details to the terminal and notifies the user.

[0553] This entire process automatically provides care tailored to the emotional state of the plant and the user, maintaining both happiness and plant health.

[0554] (Example 2)

[0555] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0556] Conventional plant management systems have struggled to adjust plant care based on the user's emotional state, making personalized management difficult. Furthermore, limited features for presenting and modifying care plans made it challenging to provide prompt and optimal plant care. Therefore, there is a need for flexible and effective plant care that responds to the user's emotions.

[0557] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0558] In this invention, the server includes means for recognizing the type and health status of plants, means for analyzing the user's emotional state and generating a care plan that includes adjustments taking the emotional state into account, and means for presenting the generated care plan to the user through a user interface. This makes it possible to quickly provide personalized plant care based on the user's emotions.

[0559] An "image acquisition device" is a device used to photograph plants and acquire them as digital data.

[0560] An "image analysis device" is a system that analyzes acquired images of plants to determine their species and health condition.

[0561] A "generative AI model" is an artificial intelligence system that generates instructions or suggestions tailored to a specific purpose based on collected data.

[0562] A "care plan" is a plan that outlines the management and care methods required for a plant.

[0563] A "user interface" is a means of interaction that allows a user to input and output information from a system.

[0564] A "wireless communication device" is a device used to send and receive data via wireless technology.

[0565] "Emotional state" refers to the user's psychological and emotional state, and is information inferred from voice tone, facial expressions, and text.

[0566] "Environmental sensor data" refers to data that measures the conditions of the environment in which plants are placed, and includes information such as temperature, humidity, and light intensity.

[0567] A "control mechanism" is a mechanism for automatically adjusting and executing plant care.

[0568] This invention is a system that streamlines plant management and provides personalized care based on the user's emotions. The system mainly consists of an image acquisition device, an image analysis device, a server using a generative AI model, a terminal equipped with a user interface, and a wireless communication device.

[0569] The user first uses a terminal to take images of plants using an image acquisition device. The terminal transmits this image data to a server via wireless communication. The server uses an image analysis device to analyze the type and health status of the plants. At this stage, machine learning algorithms are used to evaluate the health status based on the color and shape of the leaves.

[0570] Next, the server uses a generative AI model to analyze the user's emotional state. The device collects the user's voice and facial expression data and sends it to the server. The server recognizes the emotional state from this data and reflects it in the plant care plan. For example, if the user is feeling stressed, the server generates a care plan that suggests stress-relieving plants and environmental settings.

[0571] The generated care plan is presented to the user via the terminal, allowing them to review and modify it as needed. To facilitate this interaction, the user interface is designed to be intuitive and easy to use. This enables users to provide optimized plant care.

[0572] A concrete example is a user who grows houseplants in their office. If this user is busy and stressed, the server will sense this and suggest plant placement and lighting adjustments that promote relaxation. An example of a prompt would be, "Please suggest plant placement that suits my current mood." This allows the user to quickly implement care that is appropriate for their emotions.

[0573] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0574] Step 1:

[0575] The user uses a terminal to capture images of plants with an image acquisition device. These images are stored in the terminal's memory and transmitted to a server using a wireless communication device. The input is a raw image of the plant, and the output is image data sent to the server.

[0576] Step 2:

[0577] The server takes the received images into an image analysis device and analyzes the plant type and health status. In this step, machine learning algorithms are used to process and calculate data based on leaf color and shape. The input is image data, and the output is the analyzed plant information.

[0578] Step 3:

[0579] The server receives user voice and facial expression data transmitted from the terminal and processes it to analyze the emotional state. The input is the user's emotion-related data, and the output is the inferred emotional state. This utilizes speech recognition and facial recognition technologies.

[0580] Step 4:

[0581] The server uses a generative AI model to generate a care plan based on the plant type, health status, and the user's emotional state. The input is pre-collected plant information and emotional state, and the output is a personalized care plan.

[0582] Step 5:

[0583] The generated care plan is presented to the user through the user interface. The user can review this plan and make modifications as needed. The input is care plan information, and the output is the user-approved care plan.

[0584] Step 6:

[0585] The server controls IoT devices and performs plant care according to the approved care plan. The input in this step is the finalized care plan, and the output is the actual plant care actions taken, such as watering or adjusting the light.

[0586] (Application Example 2)

[0587] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0588] In recent years, technologies that support plant growth have attracted attention, but these technologies typically only generate care plans based on plant physiological data and do not take into account the user's emotional state. As a result, general care is provided regardless of the user's mental stress or comfort, leading to the problem that the optimal plant growing environment for the user is not provided.

[0589] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0590] In this invention, the server includes means for taking photographs of plants using an image acquisition device, means for analyzing the user's emotions, and means for generating a care plan based on identified plant information and emotion data. This makes it possible to provide care that is tailored not only to the plant's health but also to the user's emotional state.

[0591] An "image acquisition device" is a device that has the function of taking images of plants and collects basic data for analyzing the type and health status of the plants.

[0592] An "image analysis device" is a device that processes acquired image data to recognize the type and health status of plants.

[0593] "Means of analyzing emotions" refers to methods for collecting user emotional data and analyzing that data to estimate the user's emotional state.

[0594] "Means for generating care plans" refers to means for creating personalized plant care plans based on identified plant information and user emotional data.

[0595] A "wireless communication device" is a device that receives various sensor data from a remote location and communicates to transmit that data to a server.

[0596] "Control means" refers to means of performing control to execute optimal automated care for plants based on environmental data and emotional data.

[0597] The system for implementing this invention primarily uses a terminal, a server, and an emotion engine.

[0598] The terminal is equipped with an image acquisition device and transmits images of plants taken by the user to a server. The server receives this image data using an image analysis device, which identifies the plant species and its health condition. The plant information obtained through this analysis is used as basic data for care plans.

[0599] In parallel, the emotion engine collects emotional data from the user's voice and text input, and analyzes this data to estimate the user's emotional state. This emotional data is also reflected in the care plan.

[0600] Based on this plant information and emotional data, the server generates an individualized care plan for the plants. This plan includes, for example, the timing of automatic watering and fertilizer distribution, and the adjustment of appropriate lighting. The generated care plan is sent back to the terminal and presented to the user. The user can approve or adjust this care plan. Once the final care plan is confirmed based on the user's actions, the server controls IoT devices via wireless communication to perform automated care. This control includes, for example, lighting operation and ambient sound adjustment.

[0601] As a concrete example, imagine a user who grows houseplants in their office. When the user says, "I want to relieve today's stress," the emotion engine analyzes their voice and determines their stress level. Based on this, it suggests plant placement and lighting adjustments. An example of this prompt is, "Please tell me how to arrange plants and set up lighting in a way that has a relaxing effect, suitable for when the user is feeling stressed."

[0602] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0603] Step 1:

[0604] The user takes a picture of a plant using their device. This image data is transmitted to the server using wireless communication. The input is the image of the plant, and the output is the transfer of the image data to the server.

[0605] Step 2:

[0606] The server processes the received image data using an image analysis device. Specifically, it applies an image analysis algorithm to recognize the plant species and health status. The input is image data of the plant, and the output is data representing the plant species and health status.

[0607] Step 3:

[0608] The user inputs emotion-reflecting data into the device. This data can be voice or text, and the device sends this information to the emotion engine. The input is voice or text data, and the output is emotion data.

[0609] Step 4:

[0610] The server analyzes the user's emotional data using an emotion engine. Specifically, it estimates the user's emotional state using a generative AI model. In this process, emotional parameters such as stress levels and the need for relaxation are generated. The output is the analyzed emotional parameters.

[0611] Step 5:

[0612] The server generates a care plan based on plant information and user emotion data. Utilizing a generation AI model, specific plant care methods tailored to the user's emotions are proposed. The output is a care plan, which includes suggestions for automatic watering timing and lighting adjustments.

[0613] Step 6:

[0614] The generated care plan is transmitted wirelessly to the terminal and presented to the user. The user can review this plan on the screen and make modifications as needed. The input is the care plan presented to the user, and the output is the user-approved or modified care plan.

[0615] Step 7:

[0616] Once the user approves the care plan, the server controls IoT devices and implements specific care based on it. For example, it might rearrange plants or adjust lighting. The output is the implementation of automated care tailored to the user's environment.

[0617] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0618] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0619] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0620] [Fourth Embodiment]

[0621] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0622] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0623] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0624] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0625] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0626] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0627] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0628] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0629] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0630] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0631] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0632] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0633] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0634] This invention provides a system for automating plant management, enabling even busy users to maintain the health of their plants. The system primarily consists of "terminals" and a "server."

[0635] First, the terminal periodically takes images of the plant using an image acquisition device. The captured images are transmitted to the server using a wireless communication device. The server analyzes the received images with an image analysis device to identify the plant species and its health status. This analysis uses algorithms to check the color and shape of the plant's leaves and the consistency of its growth.

[0636] Once the server identifies the plant type and health status, it generates an optimal care plan based on that information. This care plan specifies the frequency and amount of watering and fertilizing, as well as the necessary sunlight conditions, which are specific to the plant type. The generated care plan is sent from the server to the terminal, where the user can review it through the interface. Users can adjust the care plan settings, including through paid options.

[0637] Next, the server acquires and analyzes plant environmental sensor data in real time. This allows it to monitor daytime sunlight, humidity, and soil moisture levels to determine if the environment is suitable for plant growth. Based on the results, the automated care control system operates, automatically supplying the plants with the necessary water and fertilizer. For example, the automatic watering system may activate at a specific time, and the fertilizer supply system may be activated when environmental sensor readings fall below a certain threshold.

[0638] As a concrete example, when growing houseplants in an office environment, the system takes a picture of the plant at the start of the workday each morning, and the server analyzes its health. A care plan based on the analysis results is sent to the terminal, which the user reviews and approves. Based on the approved care plan, the server automatically waters the plant and maintains the appropriate soil moisture level. In addition, if the terminal instructs the user to keep the plant at a certain distance from the window, the user will do so.

[0639] With this system, users can maintain the health of their plants without needing specialized knowledge, and can also flexibly adapt to changes in the environment.

[0640] The following describes the processing flow.

[0641] Step 1:

[0642] The device uses its built-in camera to take pictures of plants. The photos are taken automatically according to a specified schedule.

[0643] Step 2:

[0644] The device uses wireless communication to send images of plants it has photographed to a server. This transmission is carried out using a secure protocol.

[0645] Step 3:

[0646] The server receives the images and passes them to an image analysis device to determine the type and health of the plant. The analysis is performed using an algorithm that analyzes plant characteristics such as leaf color, shape, and size.

[0647] Step 4:

[0648] The server generates an optimal care plan based on the image analysis results, determining the amount of watering, fertilizer, and sunlight required according to the plant type and its current health condition.

[0649] Step 5:

[0650] The server sends the care plan information it generates to the terminal. The terminal notifies the user of this information and provides an interface for approving or modifying the plan.

[0651] Step 6:

[0652] The user reviews and approves the care plan they received. If necessary, they can modify the care plan settings using the terminal interface.

[0653] Step 7:

[0654] Based on care plans approved and modified by the user, the server operates IoT devices to control automatic watering and fertilization. Each device operates according to the instructed time and amount.

[0655] Step 8:

[0656] The device acquires data such as temperature, humidity, and light intensity from surrounding environmental sensors and sends it to the server.

[0657] Step 9:

[0658] The server analyzes data from environmental sensors and adjusts the care plan if necessary. The adjustment results are then sent back to the terminal to notify the user.

[0659] Step 10:

[0660] The server stores and analyzes data, updating the knowledge database about plant care. This allows the system to continuously learn and optimize future care plans based on its experience.

[0661] (Example 1)

[0662] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0663] Traditional plant care primarily relies on manual monitoring and maintenance, which is burdensome, especially for busy users. Furthermore, it often requires specialized knowledge, leading to situations where proper management is not possible. Therefore, there is a need for automated management systems that can efficiently and effectively maintain plant health.

[0664] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0665] In this invention, the server includes means for identifying the characteristics and health status of plants using image analysis means, means for automatically generating a plant management plan based on the identified plant information, and means for acquiring and analyzing environmental data of the plants using a wireless communication device. This makes it possible to automatically perform the management operations necessary to maintain the health of the plants.

[0666] An "image acquisition device" is a device used to capture images of plants, and includes cameras and sensor devices.

[0667] "Image analysis means" refers to software or algorithms that process acquired image data to identify the characteristics and health status of plants.

[0668] "Characteristics" refer to the unique characteristics of a plant, such as its type, shape, and color, and these are the basis for classifying plants.

[0669] "Health status" refers to indicators that show the vitality and growth of a plant, and includes factors such as the color and shape of the leaves and the condition of the roots.

[0670] "Identification" refers to the process of recognizing, appropriately classifying, and evaluating plants based on their characteristics and health status.

[0671] A "care plan" is a plan that outlines how to care for plants in a way that is optimized according to their characteristics and health condition, and includes things like watering frequency, fertilizer amount, and sunlight requirements.

[0672] A "wireless communication device" is a device that transmits and receives data using wireless signals and provides a means of communicating with a server via a network connection.

[0673] "Surrounding environmental data" refers to data that indicates external environmental conditions that affect plant growth, and includes sunlight, humidity, soil moisture, etc.

[0674] This invention is a system for automating and efficiently managing plants, and is primarily composed of a "terminal" and a "server".

[0675] The device is equipped with a camera to acquire images of plants. The device periodically takes pictures of plants and transmits these images to a server using wireless communication. This transmission is carried out using wireless communication technologies such as Wi-Fi and Bluetooth.

[0676] The server uses image analysis software to analyze the received images. Specifically, it uses image processing libraries such as OpenCV and TensorFlow to identify the characteristics and health of the plants. This includes algorithms to determine the color, shape, and growth consistency of the plant's leaves. Based on the identified information, the server automatically generates an optimal care plan. This care plan includes details on watering, fertilization, and required sunlight conditions, and is customized specifically for the characteristics of each plant.

[0677] The generated management plan is sent from the server to the terminal, which then notifies the user using a GUI (Graphical User Interface). The user can then review and modify it as needed.

[0678] Furthermore, environmental sensors installed on the terminal acquire surrounding environmental data in real time. This data includes sunlight, humidity, and soil moisture. The server analyzes this data to check whether the conditions for plant growth are being properly maintained. Based on the environmental data, the server sends control signals to the terminal to automatically activate watering and fertilizer supply devices if necessary.

[0679] As a concrete example, when managing houseplants in an office environment, the terminal takes a picture of the plant at a set time every morning, the data is analyzed on the server, and a management plan is generated. Users can maintain the health of the plants without specialized knowledge simply by reviewing and approving this management plan during their workday.

[0680] An example of a prompt to be input into a generative AI model is, "Analyze recently taken images of plants and suggest the optimal care plan based on their health condition." By using this prompt, the system can provide an appropriate diagnosis and care plan through natural language processing.

[0681] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0682] Step 1:

[0683] The device takes pictures of plants at specified time intervals. The camera captures the plants, generates RAW image data, and then processes the data by converting it to JPEG format, preparing the resulting image file. Next, the device uses a wireless communication device to send the JPEG image file to the server.

[0684] Step 2:

[0685] The server receives image files from the terminal and inputs them into image analysis software. OpenCV and TensorFlow are used to analyze the image data. This analysis analyzes leaf color, shape, and surrounding contrast, extracting data related to the plant's characteristics and health. As a result of the analysis, estimates of the plant's species and health are generated.

[0686] Step 3:

[0687] The server automatically generates an appropriate management plan based on the results of image analysis. The generating AI model creates a care plan tailored to the plant type and health condition, outputting a management plan that includes specific instructions regarding watering, fertilization, and sunlight conditions. This plan is organized as data in JSON format.

[0688] Step 4:

[0689] The server sends the generated management plan data to the terminal. The terminal displays the management plan in a GUI based on the received data. The user can review the plan displayed on the terminal and, if there are any questions, can correct the information or check the details through the UI.

[0690] Step 5:

[0691] The device uses environmental sensors to acquire real-time data on the surrounding environment, such as sunlight, humidity, and soil moisture. The analog signals output from the sensors are converted into digital data, and the environmental data is updated at regular intervals.

[0692] Step 6:

[0693] The server analyzes environmental data sent from the terminal and determines the necessary automated management operations based on the data. For example, if humidity is low, it generates a control command to instruct the watering system to operate and sends it to the terminal. The terminal receives this control command and executes the operation of the automatic watering system or fertilizer dispenser.

[0694] Through these steps, the system automates a series of operations necessary to maintain plant health, enabling proper management without requiring users to possess specialized knowledge.

[0695] (Application Example 1)

[0696] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0697] In modern urban environments, maintaining green spaces in public areas and office buildings is crucial, but individually managing numerous plants is time-consuming and requires advanced expertise. Therefore, there is a need for a system that efficiently maintains and improves the health of plants without requiring specialized knowledge.

[0698] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0699] In this invention, the server includes means for taking photographs of plants using an image acquisition device, means for recognizing the type and health status of plants using an image analysis device, means for generating a plant care plan based on the identified plant information, and means for presenting the generated care plan to the user, checking the plant's health status in real time using a smart device, and receiving notifications. This makes it possible for users to easily maintain the health of plants without having specialized knowledge.

[0700] An "image acquisition device" is a device used to photograph plants and has the function of acquiring images to visually understand the health status of the plants.

[0701] An "image analysis device" is a device that analyzes acquired images of plants to recognize the type of plant and its health condition.

[0702] A "care plan generation method" is a means for creating an optimal care plan tailored to the type of plant, based on identified plant information.

[0703] "Means of presenting to the user" refers to means of providing information so that the user can review the generated care plan.

[0704] A "wireless communication device" is a communication device that receives environmental sensor data from plants and transmits it to a server.

[0705] A "control means" is a means that has the function of performing optimal automated care for plants based on environmental data.

[0706] A "smart device" is a device that allows users to check the health status of plants in real time and receive notifications, and functions as an interface with the user.

[0707] The system for carrying out this invention is configured to automatically monitor the health of plants and provide optimal care. The main components of the system include an image acquisition device, an image analysis device, a wireless communication device, an environmental sensor, a control means, and a smart device.

[0708] The server periodically takes images of plants using an image acquisition device. The captured images are transmitted to the server via wireless communication, where an image analysis device analyzes them. This process utilizes an AI model and an algorithm that determines the type and health status of the plants. Based on the analysis results, different care plans are generated for each type of plant and presented to the user.

[0709] The smart device notifies the user of the generated care plan. It also receives environmental sensor data to monitor the plant's health in real time and automatically provides appropriate lighting and water. The smart device functions as a user interface, allowing the user to review the generated care plan and make modifications as needed.

[0710] Furthermore, as a concrete example, in the case of houseplants placed in an office lobby, their health can be checked via smartphone or smart glasses, allowing for appropriate care to be provided. This ensures proper maintenance of the plants even when the person in charge is absent.

[0711] An example of a prompt message is as follows: "Analyze the plant in the image to identify its species and health condition, and then generate the optimal care plan."

[0712] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0713] Step 1:

[0714] The device uses an image acquisition system to capture images of plants at regular intervals. This image acquisition is performed at fixed times during the daytime, and the obtained image data is stored in high resolution. The input is an optical signal captured by the camera, and the output is digital image data.

[0715] Step 2:

[0716] The terminal transmits captured images to the server via wireless communication. The input here is the image data stored on the terminal, and the output is the image file sent to the server. Once the server receives the image, it becomes ready for analysis.

[0717] Step 3:

[0718] The server processes the received image data using an image analysis device and analyzes the plant species and health status using an AI model. The input is the transmitted image file, and the output is the analysis result. This result includes the plant species, abnormalities in leaf color and shape, and whether or not there is poor growth.

[0719] Step 4:

[0720] The server uses a generated AI model based on the analysis results to create an optimal care plan for the plants. The input is the image analysis results, and the output is a specific care plan. This plan includes details such as watering and fertilizing frequency, and sunlight conditions.

[0721] Step 5:

[0722] The care plan is sent from the server to the terminal and notified to the user via a smart device. The input is the created care plan, and the output is the information displayed to the user. This allows the user to review the plan and make adjustments as needed.

[0723] Step 6:

[0724] Users can view care plans and monitor environmental sensor data in real time through a user interface on their mobile device. Inputs are real-time sensor data, and outputs are automated care control instructions. For example, watering is automatically performed when soil moisture falls below a certain threshold.

[0725] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0726] This invention combines an emotion engine with a system that supports plant management to realize personalized plant care that responds to the user's emotions. This system mainly consists of a "terminal," a "server," and an "emotion engine."

[0727] First, the user operates a device to take a picture of a plant. This image is transmitted to a server via wireless communication. The server uses an image analysis device to analyze the type and health of the plant. Next, the server uses an emotion engine to collect the user's emotional data. This emotional data is analyzed from, for example, the user's voice tone, facial expressions, and entered text.

[0728] The emotion engine determines the user's current emotional state, such as stress levels and relaxation needs, and based on this, the server generates a plant care plan. This care plan includes basic care based on the plant type and health condition, as well as adjustments that take the user's emotions into account. For example, if the user is feeling stressed, the system may suggest plants or lighting that have stress-reducing effects.

[0729] The generated care plan is sent to the device and presented to the user through an interface. The user can approve or adjust this plan to achieve more personalized care. Once the care plan is approved, the server controls IoT devices to automatically water plants and administer fertilizer. A mechanism is also incorporated to adjust light levels and ambient sounds based on emotions.

[0730] As a concrete example, consider a user who grows houseplants in their office. When this user is particularly busy, the emotion engine detects the user's stress and suggests rearranging the plants or adjusting the lighting to help alleviate stress. Furthermore, automatic watering is performed according to the plant's needs, ensuring optimal care is provided at the user's requested timing.

[0731] This system allows users to easily provide care tailored to their own emotional state, while also enabling plants to grow healthily.

[0732] The following describes the processing flow.

[0733] Step 1:

[0734] The device takes pictures of plants. The pictures are taken according to a set schedule or a manual trigger by the user.

[0735] Step 2:

[0736] The device sends images of plants it has photographed to the server. This transmission process is conducted via a secure connection.

[0737] Step 3:

[0738] The server processes the received images using an analysis device to identify the plant species and its health condition. This analysis uses algorithms that evaluate leaf color, shape, wilting, and other factors.

[0739] Step 4:

[0740] The server collects user emotion data using an emotion engine. This data is obtained from the user's voice tone, facial expressions, and biometric information from connected wearable devices.

[0741] Step 5:

[0742] The emotion engine analyzes the data it acquires to evaluate the user's stress level and emotional state. This allows it to determine the type of care the user desires (e.g., prioritizing relaxation).

[0743] Step 6:

[0744] The server generates a personalized care plan based on plant analysis information and the user's emotional state. This plan incorporates elements based on the user's emotions in addition to basic plant care.

[0745] Step 7:

[0746] The generated care plan is sent to the terminal and presented to the user through the user interface. The user can review the care plan and make manual adjustments as needed.

[0747] Step 8:

[0748] After user approval or adjustment, the server controls IoT devices and automatically performs planned watering and fertilization. It also adjusts the environment (e.g., changes lighting and music) according to the user's emotional state.

[0749] Step 9:

[0750] The device continuously monitors data from environmental sensors and sends it to the server as needed.

[0751] Step 10:

[0752] The server analyzes this environmental data to determine if the care plan needs to be readjusted. If necessary, it sends the adjustment details to the terminal and notifies the user.

[0753] This entire process automatically provides care tailored to the emotional state of the plant and the user, maintaining both happiness and plant health.

[0754] (Example 2)

[0755] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0756] Conventional plant management systems have struggled to adjust plant care based on the user's emotional state, making personalized management difficult. Furthermore, limited features for presenting and modifying care plans made it challenging to provide prompt and optimal plant care. Therefore, there is a need for flexible and effective plant care that responds to the user's emotions.

[0757] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0758] In this invention, the server includes means for recognizing the type and health status of plants, means for analyzing the user's emotional state and generating a care plan that includes adjustments taking the emotional state into account, and means for presenting the generated care plan to the user through a user interface. This makes it possible to quickly provide personalized plant care based on the user's emotions.

[0759] An "image acquisition device" is a device used to photograph plants and acquire them as digital data.

[0760] An "image analysis device" is a system that analyzes acquired images of plants to determine their species and health condition.

[0761] A "generative AI model" is an artificial intelligence system that generates instructions or suggestions tailored to a specific purpose based on collected data.

[0762] A "care plan" is a plan that outlines the management and care methods required for a plant.

[0763] A "user interface" is a means of interaction that allows a user to input and output information from a system.

[0764] A "wireless communication device" is a device used to send and receive data via wireless technology.

[0765] "Emotional state" refers to the user's psychological and emotional state, and is information inferred from voice tone, facial expressions, and text.

[0766] "Environmental sensor data" refers to data that measures the conditions of the environment in which plants are placed, and includes information such as temperature, humidity, and light intensity.

[0767] A "control mechanism" is a mechanism for automatically adjusting and executing plant care.

[0768] This invention is a system that streamlines plant management and provides personalized care based on the user's emotions. The system mainly consists of an image acquisition device, an image analysis device, a server using a generative AI model, a terminal equipped with a user interface, and a wireless communication device.

[0769] The user first uses a terminal to take images of plants using an image acquisition device. The terminal transmits this image data to a server via wireless communication. The server uses an image analysis device to analyze the type and health status of the plants. At this stage, machine learning algorithms are used to evaluate the health status based on the color and shape of the leaves.

[0770] Next, the server uses a generative AI model to analyze the user's emotional state. The device collects the user's voice and facial expression data and sends it to the server. The server recognizes the emotional state from this data and reflects it in the plant care plan. For example, if the user is feeling stressed, the server generates a care plan that suggests stress-relieving plants and environmental settings.

[0771] The generated care plan is presented to the user via the terminal, allowing them to review and modify it as needed. To facilitate this interaction, the user interface is designed to be intuitive and easy to use. This enables users to provide optimized plant care.

[0772] A concrete example is a user who grows houseplants in their office. If this user is busy and stressed, the server will sense this and suggest plant placement and lighting adjustments that promote relaxation. An example of a prompt would be, "Please suggest plant placement that suits my current mood." This allows the user to quickly implement care that is appropriate for their emotions.

[0773] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0774] Step 1:

[0775] The user uses a terminal to capture images of plants with an image acquisition device. These images are stored in the terminal's memory and transmitted to a server using a wireless communication device. The input is a raw image of the plant, and the output is image data sent to the server.

[0776] Step 2:

[0777] The server takes the received images into an image analysis device and analyzes the plant type and health status. In this step, machine learning algorithms are used to process and calculate data based on leaf color and shape. The input is image data, and the output is the analyzed plant information.

[0778] Step 3:

[0779] The server receives user voice and facial expression data transmitted from the terminal and processes it to analyze the emotional state. The input is the user's emotion-related data, and the output is the inferred emotional state. This utilizes speech recognition and facial recognition technologies.

[0780] Step 4:

[0781] The server uses a generative AI model to generate a care plan based on the plant type, health status, and the user's emotional state. The input is pre-collected plant information and emotional state, and the output is a personalized care plan.

[0782] Step 5:

[0783] The generated care plan is presented to the user through the user interface. The user can review this plan and make modifications as needed. The input is care plan information, and the output is the user-approved care plan.

[0784] Step 6:

[0785] The server controls IoT devices and performs plant care according to the approved care plan. The input in this step is the finalized care plan, and the output is the actual plant care actions taken, such as watering or adjusting the light.

[0786] (Application Example 2)

[0787] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0788] In recent years, technologies that support plant growth have attracted attention, but these technologies typically only generate care plans based on plant physiological data and do not take into account the user's emotional state. As a result, general care is provided regardless of the user's mental stress or comfort, leading to the problem that the optimal plant growing environment for the user is not provided.

[0789] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0790] In this invention, the server includes means for taking photographs of plants using an image acquisition device, means for analyzing the user's emotions, and means for generating a care plan based on identified plant information and emotion data. This makes it possible to provide care that is tailored not only to the plant's health but also to the user's emotional state.

[0791] An "image acquisition device" is a device that has the function of taking images of plants and collects basic data for analyzing the type and health status of the plants.

[0792] An "image analysis device" is a device that processes acquired image data to recognize the type and health status of plants.

[0793] "Means of analyzing emotions" refers to methods for collecting user emotional data and analyzing that data to estimate the user's emotional state.

[0794] "Means for generating care plans" refers to means for creating personalized plant care plans based on identified plant information and user emotional data.

[0795] A "wireless communication device" is a device that receives various sensor data from a remote location and communicates to transmit that data to a server.

[0796] "Control means" refers to means of performing control to execute optimal automated care for plants based on environmental data and emotional data.

[0797] The system for implementing this invention primarily uses a terminal, a server, and an emotion engine.

[0798] The terminal is equipped with an image acquisition device and transmits images of plants taken by the user to a server. The server receives this image data using an image analysis device, which identifies the plant species and its health condition. The plant information obtained through this analysis is used as basic data for care plans.

[0799] In parallel, the emotion engine collects emotional data from the user's voice and text input, and analyzes this data to estimate the user's emotional state. This emotional data is also reflected in the care plan.

[0800] Based on this plant information and emotional data, the server generates an individualized care plan for the plants. This plan includes, for example, the timing of automatic watering and fertilizer distribution, and the adjustment of appropriate lighting. The generated care plan is sent back to the terminal and presented to the user. The user can approve or adjust this care plan. Once the final care plan is confirmed based on the user's actions, the server controls IoT devices via wireless communication to perform automated care. This control includes, for example, lighting operation and ambient sound adjustment.

[0801] As a concrete example, imagine a user who grows houseplants in their office. When the user says, "I want to relieve today's stress," the emotion engine analyzes their voice and determines their stress level. Based on this, it suggests plant placement and lighting adjustments. An example of this prompt is, "Please tell me how to arrange plants and set up lighting in a way that has a relaxing effect, suitable for when the user is feeling stressed."

[0802] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0803] Step 1:

[0804] The user takes a picture of a plant using their device. This image data is transmitted to the server using wireless communication. The input is the image of the plant, and the output is the transfer of the image data to the server.

[0805] Step 2:

[0806] The server processes the received image data using an image analysis device. Specifically, it applies an image analysis algorithm to recognize the plant species and health status. The input is image data of the plant, and the output is data representing the plant species and health status.

[0807] Step 3:

[0808] The user inputs emotion-reflecting data into the device. This data can be voice or text, and the device sends this information to the emotion engine. The input is voice or text data, and the output is emotion data.

[0809] Step 4:

[0810] The server analyzes the user's emotional data using an emotion engine. Specifically, it estimates the user's emotional state using a generative AI model. In this process, emotional parameters such as stress levels and the need for relaxation are generated. The output is the analyzed emotional parameters.

[0811] Step 5:

[0812] The server generates a care plan based on plant information and user emotion data. Utilizing a generation AI model, specific plant care methods tailored to the user's emotions are proposed. The output is a care plan, which includes suggestions for automatic watering timing and lighting adjustments.

[0813] Step 6:

[0814] The generated care plan is transmitted wirelessly to the terminal and presented to the user. The user can review this plan on the screen and make modifications as needed. The input is the care plan presented to the user, and the output is the user-approved or modified care plan.

[0815] Step 7:

[0816] Once the user approves the care plan, the server controls IoT devices and implements specific care based on it. For example, it might rearrange plants or adjust lighting. The output is the implementation of automated care tailored to the user's environment.

[0817] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0818] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0819] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0820] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0821] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0822] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0823] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0824] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0825] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0826] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0827] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0828] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0829] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0830] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0831] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0832] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0833] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0834] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0835] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0836] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0837] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0838] The following is further disclosed regarding the embodiments described above.

[0839] (Claim 1)

[0840] A means of photographing plants using an image acquisition device,

[0841] A means for recognizing the type and health status of plants using an image analysis device,

[0842] A means for generating a plant care plan based on identified plant information,

[0843] A means of presenting the generated care plan to the user,

[0844] A means for receiving and analyzing environmental sensor data of plants using a wireless communication device,

[0845] A system including control means that performs optimal automated care for plants based on environmental data.

[0846] (Claim 2)

[0847] The system according to claim 1, characterized by generating and providing different care plans for each type of plant based on the results of plant image analysis.

[0848] (Claim 3)

[0849] The system according to claim 1, characterized in that it has a user interface that allows the user to review and modify the generated care plan.

[0850] "Example 1"

[0851] (Claim 1)

[0852] A means of photographing plants using an image acquisition device,

[0853] A means for identifying the characteristics and health status of plants using image analysis methods,

[0854] A means for automatically generating a plant management plan based on information about identified plants,

[0855] A means of notifying users of the generated management plan,

[0856] A means of acquiring and analyzing environmental data of plants using wireless communication devices,

[0857] A system including control means that automatically performs plant management operations based on surrounding environmental data.

[0858] (Claim 2)

[0859] The system according to claim 1, characterized by generating and providing a management plan tailored to different plant species based on the results of plant image analysis.

[0860] (Claim 3)

[0861] The system according to claim 1, characterized by having a user interface that allows the user to review and modify the generated management plan.

[0862] "Application Example 1"

[0863] (Claim 1)

[0864] A means of photographing plants using an image acquisition device,

[0865] A means for recognizing the type and health status of plants using an image analysis device,

[0866] A means for generating a plant care plan based on identified plant information,

[0867] A means of presenting the generated care plan to the user,

[0868] A means for receiving and analyzing environmental sensor data of plants using a wireless communication device,

[0869] A control means that performs optimal automated care for plants based on environmental data,

[0870] A means of checking the health status of plants in real time and receiving notifications using smart devices,

[0871] A system that includes this.

[0872] (Claim 2)

[0873] The system according to claim 1, characterized by generating and providing different care plans for each plant species based on the results of plant image analysis.

[0874] (Claim 3)

[0875] The system according to claim 1, characterized by having a user interface that allows the user to review and modify the generated care plan and receive re-notifications via a smart device.

[0876] "Example 2 of combining an emotion engine"

[0877] (Claim 1)

[0878] A means of photographing plants using an image acquisition device,

[0879] A means for recognizing the type and health status of plants using an image analysis device,

[0880] A means for generating a plant care plan using a generative AI model based on identified plant information,

[0881] A means for analyzing the user's emotional state and generating a care plan that includes adjustments that take the emotional state into account,

[0882] A means of presenting the generated care plan to the user through a user interface,

[0883] A means for users to review and modify their emotionally-based care plans,

[0884] A means for receiving and analyzing environmental sensor data of plants using a wireless communication device,

[0885] A system including control means to perform optimal automated care for plants based on environmental data and emotional state.

[0886] (Claim 2)

[0887] The system according to claim 1, characterized in that it generates and provides different care plans for each type of plant based on the image analysis results of the plant and the user's emotional state.

[0888] (Claim 3)

[0889] The system according to claim 1, characterized in that it uses a generative AI model to generate prompt messages for the user based on user sentiment analysis data.

[0890] "Application example 2 when combining with an emotional engine"

[0891] (Claim 1)

[0892] A means of photographing plants using an image acquisition device,

[0893] A means for recognizing the type and health status of plants using an image analysis device,

[0894] A means of analyzing user emotions and collecting emotional data,

[0895] A means for generating a plant care plan based on identified plant information and emotional data,

[0896] A means of presenting the generated care plan to the user,

[0897] A means for receiving and analyzing environmental sensor data of plants using a wireless communication device,

[0898] A system including control means that performs optimal automated care for plants based on environmental data and emotional data.

[0899] (Claim 2)

[0900] The system according to claim 1, characterized by generating and providing different care plans for each type of plant based on the results of plant image analysis and user sentiment data.

[0901] (Claim 3)

[0902] The system according to claim 1, characterized in that it has a user interface that allows the user to review and modify the generated care plan based on their emotions. [Explanation of Symbols]

[0903] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of photographing plants using an image acquisition device, A means for recognizing the type and health status of plants using an image analysis device, A means for generating a plant care plan based on identified plant information, A means of presenting the generated care plan to the user, A means for receiving and analyzing environmental sensor data of plants using a wireless communication device, A control means that performs optimal automated care for plants based on environmental data, A means of checking the health status of plants in real time and receiving notifications using smart devices, A system that includes this.

2. The system according to claim 1, characterized in that it generates and provides different care plans for each plant species based on the results of plant image analysis.

3. The system according to claim 1, characterized in that it has a user interface, allows the user to review and modify the generated care plan, and receives further notifications via a smart device.

Images