System

Abstract

A system is provided.SOLUTION: A means for a user to post an image of a plant, a means for a terminal to transmit the posted image of the plant and metadata to a server, a means for the server to store the received image and metadata, a means for the server to identify a type of the plant using an image recognition model for the received image, a means for the server to extract a growth condition from a database based on the identified type of the plant, and a means for the server to integrate the extracted growth condition; A system comprising: means for generating an optimal growing plan for a user; means for transmitting the growing plan generated by a server to a terminal; and means for presenting the growing plan received by the terminal to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Many people are interested in decorating their homes with houseplants or bouquets, but they often face the problem of these plants quickly dying and not knowing how to properly care for them. In particular, when bouquets contain multiple types of plants, each plant requires different growing conditions, making it extremely difficult to determine which method to use. This can result in excessive effort and time spent maintaining the health of the plants, which can increase the risk of the plants dying. [Means for solving the problem]

[0005] The present invention provides a system that automatically provides optimal cultivation methods for plants simply by users posting images of the plants they are growing. The system includes a means for users to post images of plants and a means for transmitting the images and metadata to a server. The server standardizes the received images, resizes them if necessary, and identifies the plant type using an image recognition model. Based on the identified plant type, cultivation conditions are extracted from a database and integrated to generate an optimal cultivation plan. The cultivation plan is then sent to a terminal and presented to the user. In this way, users can grasp the cultivation conditions for multiple plants at once and manage them appropriately. This system significantly reduces the effort and time required to keep plants healthy and long-lasting.

[0006] "User" refers to any individual or organization that uses the System to post images of plants.

[0007] "Terminal" refers to the device operated by the user (smartphone, tablet, PC, etc.), which is responsible for sending images of plants to the server.

[0008] "Server" refers to the central computer system that processes images and metadata received from users and generates development plans.

[0009] An "image recognition model" refers to an algorithm that uses machine learning and deep learning techniques to identify plant species from images.

[0010] "Growing conditions" refers to the elements necessary for plants to grow healthily (such as watering frequency, sunlight, water quality, fertilizer, etc.).

[0011] "Database" refers to a storage device or system that stores the growing conditions and characteristics of various plants.

[0012] "Growth plan" refers to instructions for users that are created by combining optimal growing conditions for the recognized plants.

[0013] "Metadata" refers to additional information associated with an image (e.g., date and time of capture, location information) that is used to improve the accuracy of image recognition. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] Users use the "My Flowerbed" application to manage the plants they grow at home or in the office. The system utilizes a server, user devices, and image recognition models as part of the platform.

[0036] Program processing flow and explanation

[0037] 1. User-submitted images

[0038] Users open the app and use their device's camera to take a picture of a plant or bouquet. In some cases, they can also select an existing image from their device's gallery. The image is displayed in the app, and users can then click the "Submit" button to send it to the server.

[0039] 2. Submitting images and metadata

[0040] The device collects the images selected by the user along with metadata such as the date and time of the photo and location, and transmits this data to a server over a secure protocol.

[0041] 3. Image Reception and Preprocessing

[0042] The server receives the images and metadata sent from the device, standardizes the image format, and resizes them if necessary, improving the processing efficiency of the image recognition model.

[0043] 4. Image Recognition Processing

[0044] The server then uses an image recognition model to identify the plant species from the received image. This model has been trained on many plant images in advance, allowing it to identify plant species with high accuracy.

[0045] 5. Obtaining breeding conditions from a database

[0046] Based on the identified plant type, the server extracts from a database the optimal growing conditions for each plant, including watering frequency, light requirements, and the appropriate type of fertilizer.

[0047] 6. Integration of multiple plant conditions

[0048] If an image contains multiple plants, the server integrates their respective growing conditions and generates a consistent growing plan. For example, if a Monstera and a rose are identified together, the server creates a plan based on their respective characteristics.

[0049] 7. Generate and submit a training plan

[0050] The generated cultivation plan is sent from the server to the device, and includes specific advice such as how often to water the plant, the amount of sunlight, and when to use fertilizer.

[0051] 8. User Notification and Execution

[0052] The device will notify the user of the received care plan, and the user can check detailed care instructions in the app and follow the instructions to care for their plants. The app also provides regular reminders to help users continue to care for their plants properly.

[0053] Specific examples

[0054] For example, if a user posts a photo of a Monstera plant in their living room and a vase of roses to the app, the following process occurs:

[0055] 1. The user takes a photo using the app and presses the post button.

[0056] 2. The device sends the image along with its metadata to the server.

[0057] 3. The server receives the image, normalizes it, and resizes it.

[0058] 4. The server uses an image recognition model to distinguish between a Monstera and a rose.

[0059] 5. The server retrieves the optimal growing conditions for each plant from the database.

[0060] 6. The server integrates the conditions for the Monstera and roses and generates a consistent cultivation plan.

[0061] 7. The server sends the development plan to the device.

[0062] 8. The device will notify the user of the development plan and provide specific care methods.

[0063] In this way, the system helps users easily care for multiple plants and provides optimal methods to ensure their longevity.

[0064] The processing flow will be explained below.

[0065] Program processing flow

[0066] Step 1:

[0067] Users open the "MY Flower Bed" app and use their device's camera to take a picture of the plants or bouquet they are growing, or select an existing image from their photo gallery.

[0068] Step 2:

[0069] The user presses the submit button to send the selected image to the server, optionally adding comments and metadata about the plant's condition.

[0070] Step 3:

[0071] The device acquires the image selected by the user and the associated metadata (e.g., shooting date and time, location information), and makes a request to the server to send the data.

[0072] Step 4:

[0073] The server receives the images and metadata sent, and first converts the file format and adjusts the image size to standardize the received images.

[0074] Step 5:

[0075] The server passes the standardized images to an image recognition algorithm, which uses a pre-trained convolutional neural network (CNN) or similar to identify the type of plant in the image.

[0076] Step 6:

[0077] The server analyzes multiple segments within an image and individually identifies the plants contained within each segment, allowing for accurate species identification even when multiple plants are present in a photo.

[0078] Step 7:

[0079] The server extracts the optimal growing conditions (such as watering frequency, amount of sunlight, and type of fertilizer) from a database based on the identified plant type.

[0080] Step 8:

[0081] The server comprehensively assesses the growing conditions of multiple plants and creates a consistent growing plan for the user, which is adjusted to take into account the characteristics of each plant.

[0082] Step 9:

[0083] The server formats the plan into specific instructions and sends them to the device, including instructions on when to water the plant, how much sunlight it needs, and how to properly fertilize it.

[0084] Step 10:

[0085] The device receives the training plan from the server and displays it on the app's user interface. Push notifications and alarms can also be used to set reminders to encourage timely care.

[0086] Step 11:

[0087] Users can care for their plants according to the provided care plan, and can also post images again to receive updated care plans as the situation evolves.

[0088] Example 1

[0089] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0090] Conventional plant growth management systems make it difficult for users to integrate individual conditions and generate consistent growth plans when managing multiple plants. They also have limited functionality for collecting detailed metadata along with plant images and providing regular reminders to users. This makes it difficult for users to effectively care for their plants, and it takes a lot of time and effort.

[0091] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0092] In this invention, the server includes means for standardizing and resizing the image format, means for integrating the growing conditions of the identified multiple plants to generate a consistent growing plan, and means for providing regular reminders to support the user in continuing appropriate care. This makes it easier for the user to manage multiple plants, enables the generation of a growing plan that integrates the optimal growing conditions for each plant, and enables the user to care for the plants efficiently and effectively.

[0093] "User" refers to a person who uses the system or application to manage and cultivate plants.

[0094] "Device" refers to the device a user uses to capture images of plants and collect and transmit metadata, including smartphones and tablets.

[0095] "Server" refers to a computer system that receives, stores, and processes images and metadata submitted by users.

[0096] "Image recognition model" refers to the machine learning or deep learning algorithm used to identify plant species from received images.

[0097] "Metadata" is data that includes information related to an image, specifically including the date and time of the image capture, location information, camera settings, and the like.

[0098] "Growing conditions" refers to the environmental factors that a particular plant needs to grow optimally, including watering frequency, light requirements, and type of fertilizer.

[0099] A "growth plan" refers to specific care methods created based on the optimal growing conditions for each type of plant.

[0100] "Periodic reminders" refers to a feature that sends regular notifications to users to encourage them to continue proper development.

[0101] "Normalization" refers to the process of converting received images into a consistent format so that subsequent processing can be done efficiently.

[0102] "Resizing" refers to the process of changing the size of an image, which improves the efficiency of image recognition models.

[0103] The present invention provides a system that allows users to easily and effectively manage and cultivate plants, and is implemented using a server, a terminal, and an image recognition model, allowing users to efficiently obtain information necessary for plant care.

[0104] First, a user installs an application for managing plants. When a user wants to post an image of a plant, they take or select an image using the device's camera or gallery. The device then collects the image along with metadata such as the date and time of the photo and its location, and sends it to a server via a secure protocol (e.g., HTTPS).

[0105] The server first stores the received images and metadata. Next, it standardizes the image format and resizes it as necessary. This allows for efficient processing by an image recognition model (e.g., a deep learning model using TensorFlow or PyTorch). The server then uses the image recognition model to identify the plant species from the received images. This model has been trained in advance on many plant images and can identify plant species with high accuracy.

[0106] After identifying the type of plant, the server extracts the optimal growing conditions for each plant from a database (e.g., a relational database such as MySQL). This includes watering frequency, sunlight requirements, and the appropriate type of fertilizer. If multiple plants are identified, their respective growing conditions are integrated to generate a consistent growing plan. For example, if a Monstera and a rose are identified together, a growing plan tailored to the characteristics of each will be created. The generated growing plan is sent from the server to the device, and the device notifies the user. The user can check detailed growing instructions through the app and practice appropriate care. The device also provides regular reminders to help the user continue to provide appropriate care.

[0107] Specific examples

[0108] For example, if a user posts a photo of a monstera plant in their living room and a vase of roses, the following process occurs:

[0109] 1. The user takes a photo using the app and presses the post button.

[0110] 2. The device sends the image along with its metadata to the server.

[0111] 3. The server receives the image, normalizes it, and resizes it.

[0112] 4. The server uses an image recognition model to distinguish between a Monstera and a rose.

[0113] 5. The server retrieves the optimal growing conditions for each plant from the database.

[0114] 6. The server integrates the conditions for the Monstera and roses and generates a consistent cultivation plan.

[0115] 7. The server sends the development plan to the device.

[0116] 8. The device will notify the user of the development plan and provide specific care methods.

[0117] In this way, the system helps users easily care for multiple plants and provides optimal methods to ensure their longevity.

[0118] Prompt Sentence Examples

[0119] "I would like to know how to care for the Monstera and roses I am growing at home. Please tell me the care plan for growing Monstera and roses at the same time."

[0120] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0121] Step 1: User submits image

[0122] 1. The user launches the app and the main screen displays the camera and gallery icons.

[0123] 2. The user taps the camera icon to launch the device's camera app and take a picture of the plant or bouquet.

[0124] 3. Input: An image of a plant taken with the device's camera.

[0125] 4. The user presses the shutter button to take a picture, which is then displayed on the preview screen.

[0126] 5. The user presses the "Post" button on the preview screen and the image is saved within the app.

[0127] 6. Output: Image data captured by the user.

[0128] Step 2: Submit your images and metadata

[0129] 1. The device obtains metadata, such as the date and time of the image taken and its location, for the image selected or taken by the user.

[0130] 2. Input: Images taken by the user, along with metadata such as the date and time of the image and its location.

[0131] 3. The device sends the image and metadata to the server over a secure protocol such as HTTPS.

[0132] 4. Output: Image data and metadata sent to the server.

[0133] Step 3: Receiving and preprocessing images

[0134] 1. The server receives the image and metadata sent from the device.

[0135] 2. Input: Image data and metadata sent from the device.

[0136] 3. The server standardizes the image format and resizes it as needed, for example converting the image width and height to a consistent number of pixels.

[0137] 4. Output: Resized and standardized image data.

[0138] Step 4: Image Recognition Processing

[0139] 1. The server inputs the preprocessed image into an image recognition model that has been trained on many plant images in advance.

[0140] 2. Input: Preprocessed image data.

[0141] 3. The server uses an image recognition model to identify the plant type.

[0142] 4. Output: Identified plant species data (e.g. Monstera, Rose).

[0143] Step 5: Obtaining breeding conditions from the database

[0144] 1. The server searches the database for growing conditions based on the identified plant type.

[0145] 2. Input: Identified plant species data.

[0146] 3. The server obtains growing conditions, including watering frequency, sunlight requirements, and appropriate type of fertilizer.

[0147] 4. Output: Growing condition data for each plant.

[0148] Step 6: Integrating multiple plant conditions

[0149] 1. When multiple plants are identified, the server integrates their respective growing conditions.

[0150] 2. Input: Individual growing condition data for multiple plants.

[0151] 3. The server analyzes the similarities and differences between each breeding condition and generates a consistent breeding plan.

[0152] 4. Output: Integrated development plan.

[0153] Step 7: Generate and submit your development plan

[0154] 1. The server generates specific advice based on an integrated growing plan, including watering frequency, amount of sunlight, and timing of fertilizer application.

[0155] 2. Input: Integrated Development Plan.

[0156] 3. The server sends this development plan to the terminal.

[0157] 4. Output: Development plan data sent to the terminal.

[0158] Step 8: Notify users and take action

[0159] 1. The terminal notifies the user of the development plan received from the server.

[0160] 2. Input: Development plan data from the server.

[0161] 3. The user views the notification and checks detailed training instructions within the app.

[0162] 4. The user follows the plant care instructions, such as watering a Monstera twice a week or placing a rose in a sunny spot.

[0163] 5. The device will provide regular reminders to help users continue to take proper care.

[0164] (Application example 1)

[0165] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0166] Conventional plant growth management systems are primarily used to manage plants in indoor environments, making it difficult to manage plants that adapt to the unique environmental conditions inside autonomous vehicles. Furthermore, there is a lack of technology that provides growth plans that take into account environmental factors such as temperature, humidity, and CO2 concentration inside the vehicle. This makes it difficult for users who want to grow plants inside their vehicles to provide an optimal growing environment, making it difficult to maintain the health of their plants.

[0167] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0168] In this invention, the server includes means for users to post plant images, means for a terminal to transmit the posted plant images and metadata to the server, means for the server to store the received images and metadata, means for the server to identify the type of plant using an image recognition model for the received images, means for the server to extract cultivation conditions from a database based on the identified plant type, means for the server to integrate the extracted cultivation conditions and generate an optimal cultivation plan for the user, means for transmitting the generated cultivation plan to the terminal, means for the terminal to present the received cultivation plan to the user, means for acquiring vehicle sensor data, and means for combining the acquired sensor data with plant cultivation conditions to generate an optimal cultivation plan, thereby enabling optimal plant cultivation management tailored to the in-vehicle environment of an autonomous vehicle.

[0169] "User" refers to an entity that uses this system to manage plants.

[0170] A "terminal" is a device used by a user that is equipped with means for taking images of plants and transmitting data.

[0171] The "server" is a device that receives, stores, and processes images and data sent from the terminal, and runs an image recognition model to identify plants and generate cultivation plans.

[0172] "Metadata" is additional information related to the main data, such as the date and time an image was taken and location information.

[0173] An "image recognition model" is a software model that runs on a server and is a technology used to identify plant species from submitted images.

[0174] A "database" is a collection of data that stores information about plant growing conditions.

[0175] A "growth plan" is a systematic compilation of the various conditions necessary for healthy plant growth (e.g., frequency of watering, sunlight conditions, timing of fertilizer application, etc.).

[0176] "Sensor data" refers to data obtained from sensors used to detect environmental information inside a vehicle, such as temperature, humidity, and CO2 concentration.

[0177] The "vehicle" is a means of transportation that utilizes autonomous driving technology and is equipped with environmental sensors to support plant growth.

[0178] "Standardization" is the process of converting incoming image data into a consistent format.

[0179] "Resizing" is the process of changing the size of image data to an appropriate size.

[0180] In the embodiment of the present invention, a system for managing plant growth inside an autonomous vehicle will be specifically described.

[0181] Users use a smartphone application called "CAR BOTANICA" to manage the plants installed in their vehicle. The user launches the application and takes a picture of the plant using the vehicle's camera. The image is then sent to a server via the device.

[0182] The server stores the received images and metadata (date and time of capture, location, sensor data), then uses an image recognition model to identify the plant species from the submitted image, using a machine learning framework such as TensorFlow.

[0183] Based on the identified plant type, the server extracts the growing conditions from a database. The database stores the growing conditions for various plants (watering frequency, sunlight conditions, fertilizer application timing, etc.). Furthermore, by combining this with sensor data (temperature, humidity, CO2 concentration), the server generates an optimal growing plan. This provides a growing plan that adapts to the unique environmental conditions inside the vehicle.

[0184] The development plan generated by the server is sent to the device (smartphone application). The user can check the development plan through the app and take appropriate care. The application also uses notification services such as Firebase to send reminders to the user and support them in taking appropriate care.

[0185] For example, if a user takes a picture of a Monstera plant in their car, the image is sent to a server where it is normalized and resized. An image recognition model then identifies the Monstera plant and extracts its optimal growing conditions from a database. This, combined with sensor data, generates a growing plan that, for example, notifies the user to open a window to ensure optimal temperature and humidity control.

[0186] Example prompt sentence:

[0187] Input plant images taken inside the car into a TensorFlow model to identify the plant type. Retrieve the identified plant's growing conditions from a database and combine them with sensor data to generate an optimal growing plan. Notify the driver of the generated growing plan using the Firebase notification service.

[0188] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0189] Step 1:

[0190] The user launches the smartphone application and takes a picture of a plant using the in-car camera. The application displays the image the user has taken and provides a posting button. The input is the image data captured by the camera, and the output is the image that the user can review and post.

[0191] Step 2:

[0192] The device sends images posted by users to the server, along with metadata such as the date and time of the photo, location information, and sensor data from inside the vehicle (temperature, humidity, CO2 concentration). The inputs are image data and metadata, and the output is this data sent to the server.

[0193] Step 3:

[0194] The server stores the received images and metadata. The server standardizes the image data and resizes it as necessary, improving the processing efficiency of the image recognition model. The inputs are the received images and metadata, and the standardized and resized image data is stored as the output.

[0195] Step 4:

[0196] The server uses an image recognition model to identify the plant species from the received image. It uses a machine learning framework such as TensorFlow to identify the plant species from the image. The input is the normalized and resized image data, and the output is the identified plant species.

[0197] Step 5:

[0198] The server extracts the growing conditions from a database based on the identified plant type. The database stores the growing conditions for various plants (watering frequency, sunlight conditions, fertilizer application timing, etc.). The input is the identified plant type, and the output is the extracted growing conditions.

[0199] Step 6:

[0200] The server combines the sensor data acquired with the plant's growing conditions to generate an optimal growing plan. For example, if the temperature and humidity inside the car do not meet certain conditions, it creates a growing plan that includes suggestions for providing an environment suitable for the plant (opening windows, adjusting temperature settings, etc.). The inputs are sensor data and growing conditions, and the output is an optimized growing plan.

[0201] Step 7:

[0202] The server sends the generated training plan to the device. The device receives this training plan and notifies the user. Services such as Firebase are used for notification. The input is the generated training plan, and the output is the training plan sent to the device.

[0203] Step 8:

[0204] The device receives the training plan and presents it to the user, informing them of specific care methods. The user can then check the detailed instructions through the app and provide appropriate care. The input is the received training plan, and the output is the training plan and care methods presented to the user.

[0205] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0206] This invention is a system that recognizes the user's emotional state by combining an emotion engine with the "MY Flower Bed" application, and provides care methods and encouraging messages based on that. This system consists of a server, a terminal, an image recognition model, an emotion engine, and a database.

[0207] Program processing flow and explanation

[0208] 1. User-submitted images

[0209] The user opens the application, takes a picture of a plant or bouquet using the device's camera, or selects an existing image from the photo gallery. The user then checks the image they have taken or selected within the application and presses the submit button to send the image to the server.

[0210] 2. Submitting images and metadata

[0211] The device acquires the image selected by the user along with the date and time of the photo, location information, and the user's emotional data, and sends them to the server.

[0212] 3. Image Reception and Preprocessing

[0213] The server receives the image, metadata, and emotion data sent from the device. First, it normalizes the image and resizes it if necessary.

[0214] 4. Image Recognition Processing

[0215] The server then uses an image recognition model to identify the plant species from the received image. This model has been trained on many plant images in advance, allowing it to identify plant species with high accuracy.

[0216] 5. Obtaining breeding conditions from a database

[0217] Based on the identified plant type, the server extracts from a database the optimal growing conditions for each plant, including watering frequency, light requirements, and the appropriate type of fertilizer.

[0218] 6. Integration of multiple plant conditions

[0219] If an image contains multiple plants, the server will integrate their respective growing conditions and generate a consistent growing plan. For example, if a Monstera and a rose are identified together, a growing plan will be created based on their respective characteristics.

[0220] 7. Emotion Data Analysis

[0221] The server uses an emotion engine to analyze the user's emotional data, identify the user's emotional state (e.g., joy, sadness, stress, etc.), and generate appropriate advice or encouraging messages based on this.

[0222] 8. Creation and integration of development plans

[0223] The server further customizes the planting plan by taking into account the user's emotional state, as analyzed by the emotion engine. For example, if the user is feeling stressed, the plan will include advice on plant care that has a relaxing effect.

[0224] 9. Development plan and message sending

[0225] The server generates a training plan and sends messages based on the emotions to the device, including specific care methods and encouraging messages.

[0226] 10. User Notice and Execution

[0227] The device displays the plant care plan and messages received on the app's user interface. Push notifications and reminder functions are used to prompt the user to care for their plants in a timely manner. The user then follows the plant care plan and emotion-based advice provided. Additional feedback can also be input into the app, allowing the emotion engine to continuously track changes in the user's emotional state.

[0228] Specific examples

[0229] For example, if a user posts a photo of a Monstera plant in their living room and a vase of roses to the app, the following process occurs:

[0230] 1. The user takes a picture using the application and presses the post button.

[0231] 2. The device sends the image, metadata, and user emotion data to the server.

[0232] 3. The server receives the image, normalizes it, and resizes it.

[0233] 4. The server uses an image recognition model to distinguish between a Monstera and a rose.

[0234] 5. The server retrieves the growing conditions for each plant from the database.

[0235] 6. The server integrates the growing conditions for Monstera and roses.

[0236] 7. The server analyzes the user's emotional data using an emotion engine and determines that the user is feeling stressed.

[0237] 8. The server creates a development plan by adding development advice that helps reduce stress.

[0238] 9. The server sends the training plan and encouraging messages to the device.

[0239] 10. The device notifies the user of the received development plan and message and displays it.

[0240] 11. The user will care for the plant in accordance with the growing plan and advice.

[0241] In this way, the system can provide optimal plant cultivation methods taking into account the user's emotional state, while also supporting the user's physical and mental care.

[0242] The processing flow will be explained below.

[0243] Program processing flow

[0244] Step 1:

[0245] The user opens the "MY Flower Bed" app and uses the device's camera to take a picture of a plant or bouquet, or select an existing image from the photo gallery.

[0246] Step 2:

[0247] The user presses the submit button to send the selected image to the server, optionally including emotional data (e.g., emotion estimation via an in-app survey or facial recognition).

[0248] Step 3:

[0249] The device acquires the image selected by the user along with metadata such as the date and time of shooting, location information, and emotional data, and makes a request to the server to send the data.

[0250] Step 4:

[0251] The server receives the images, metadata, and emotion data, normalizes the received images, and resizes them if necessary.

[0252] Step 5:

[0253] The server passes the standardized images to an image recognition algorithm, which uses a pre-trained convolutional neural network (CNN) to identify the type of plant in the image.

[0254] Step 6:

[0255] The server analyzes multiple segments within an image and individually identifies the plants contained within each segment, allowing for accurate species identification even when multiple plants are present in a photo.

[0256] Step 7:

[0257] The server extracts the optimal growing conditions (such as watering frequency, amount of sunlight, and type of fertilizer) from a database based on the identified plant type.

[0258] Step 8:

[0259] The server comprehensively assesses the growing conditions of multiple plants and creates a consistent growing plan for the user, which is adjusted to take into account the characteristics of each plant.

[0260] Step 9:

[0261] The server analyzes the user-provided emotion data using an emotion engine, which identifies the user's emotional state (e.g., joy, sadness, stress, etc.).

[0262] Step 10:

[0263] The server then customizes the plant care plan based on information gleaned from the emotional data, adding relaxing plant care advice if the user is feeling stressed, for example.

[0264] Step 11:

[0265] The server generates emotion-based messages (encouragement, positive feedback, etc.) and includes them in the development plan.

[0266] Step 12:

[0267] The server sends a customized development plan and a message based on the emotion to the terminal.

[0268] Step 13:

[0269] The device displays the received training plan and message on the user interface, and uses push notifications and reminder functions to encourage users to take timely care.

[0270] Step 14:

[0271] Users follow the advice based on the presented plant care plan and their emotions. If necessary, they can resubmit images and receive updated care plans. This allows users to not only manage the health of their plants, but also their own emotions at the same time.

[0272] Example 2

[0273] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0274] Conventional plant cultivation support systems simply provide plant cultivation conditions without considering the user's emotional state, and do not contribute to the user's mental care or stress reduction. Therefore, there is a need for a system that takes the user's emotional state into consideration and provides individually customized cultivation plans and encouraging messages to support the user's mental and physical health.

[0275] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0276] In this invention, the server includes means for users to post plant images, means for a terminal to transmit the posted plant images and additional information to the server, means for the server to store the received images and additional information, means for the server to identify the type of plant using an image recognition model for the received images, means for the server to extract cultivation conditions from a database based on the identified plant type, means for the server to integrate the extracted cultivation conditions and generate an optimal cultivation plan for the user, means for the server to transmit the generated cultivation plan to the terminal, means for the terminal to present the cultivation plan received by the terminal to the user, means for the server to analyze the user's emotional state using an emotion analysis device, and means for the server to customize the cultivation plan based on the analyzed emotional state and generate advice and encouraging messages associated with the emotions. This enables the provision of a plant cultivation plan that takes the user's emotional state into consideration and emotional care.

[0277] A "user" is someone who posts images of plants and receives advice from the system, including growing plans and emotionally relevant advice.

[0278] "Terminal" refers to the device used by the user to take images of plants, send the data to the server, and receive and display cultivation plans and advice.

[0279] "Server" refers to a central device that receives, stores, processes images and data sent from a terminal, and generates and sends development plans and advice to the terminal.

[0280] "Additional information" refers to additional metadata sent with an image, such as the date and time of the image capture, location information, or user emotional data.

[0281] "Image Recognition Model" refers to a model that uses machine learning algorithms to identify the type of plant contained in a received image.

[0282] "Database" refers to a system for storing and managing plant growing conditions and other related information.

[0283] "Growth conditions" refers to information including watering frequency, sunlight conditions, type of fertilizer, etc., as conditions for optimal growth of a particular plant.

[0284] "Growth plan" refers to a plan that integrates growing conditions extracted from the database and provides specific plant care methods to the user.

[0285] "Emotion analysis device" refers to hardware or software used to analyze a user's emotional state.

[0286] "Messages of advice and encouragement" refer to advice and support messages that are generated based on the user's emotional state and provided to the user.

[0287] This invention is a plant-growing support system that takes into account the user's emotional state. This system is composed of a server, a terminal, an image recognition model, an emotion analysis device, and a database, and provides users with individually customized plant-growing plans and advice.

[0288] Hardware and Software Configuration

[0289] Terminal

[0290] The terminal is a smartphone or tablet that allows users to take pictures of plants, send the data to the server, and receive and display cultivation plans and advice. The terminal is equipped with a camera, GPS module, and internet connection.

[0291] server

[0292] The server is a central device for storing and processing the received images and additional information. The server is equipped with an image recognition model, an emotion analysis device, and a database.

[0293] Image recognition model: Uses machine learning algorithms such as TensorFlow and PyTorch to identify the type of plant from the received image.

[0294] Emotion analysis device: A device that uses natural language processing (NLP) and facial expression recognition algorithms to analyze a user's emotional state.

[0295] Database: A system that stores and manages plant growth conditions and related information.

[0296] Data processing and calculation flow

[0297] 1. Image Reception and Preprocessing

[0298] The device sends the plant image and additional information (date and time of the photo, location information, and emotion data) to the server, which then receives the image and standardizes it by converting its format and resizing it.

[0299] 2. Image Recognition Processing

[0300] The server uses image recognition models to identify plant species, which identifies the names and genus species of plants contained in the image.

[0301] 3. Extraction of growth conditions

[0302] Based on the identified plant, the server extracts growing conditions (such as watering frequency, sunlight conditions, and type of fertilizer) from the database.

[0303] 5. Emotion Data Analysis

[0304] The server uses an emotion analyzer to identify the user's emotional state from their input data and facial expressions. For example, it may determine that the user is "stressed" based on their text input.

[0305] 6. Creating and customizing training plans

[0306] The server then combines the extracted training conditions with the emotional data to generate an optimal training plan for the user, adding customized advice and encouraging messages according to the user's emotional state.

[0307] 7. Development plan and message sending

[0308] The server sends the generated training plan and message to the terminal, which receives it and notifies and displays it to the user.

[0309] Specific examples

[0310] For example, if a user posts a photo of a Monstera plant in their living room and a vase of roses, the following happens:

[0311] 1. The user takes a photo of a plant using the in-app camera, then presses the "Post" button to send the image.

[0312] 2. The device sends the captured image and additional information (date and time of capture, location information, and emotion data) to the server.

[0313] 3. The server receives the image, normalizes it, and resizes it.

[0314] 4. The server uses image recognition models to identify the Monstera and the rose, and their plant names are identified.

[0315] 5. The server retrieves the growing conditions for the identified plants from the database. For example, a Monstera should be watered every two weeks and indirect sunlight, while a rose should be watered once a week and in direct sunlight.

[0316] 6. The server uses an emotion analyzer to analyze whether the user is feeling "stressed."

[0317] 7. The server generates a care plan by adding advice to help reduce stress. For example, it adds an encouraging message such as, "Watering this Monstera every morning will help you start your day relaxing."

[0318] 8. The server sends the final training plan and encouraging messages to the device.

[0319] 9. The device displays the training plan and message received within the app and notifies the user.

[0320] Prompt Sentence Examples

[0321] Use a prompt like this for the emotion engine:

[0322] "The user's current emotional state is stressed. Generate advice suggesting how to care for plants to reduce stress."

[0323] As described above, the present invention is a system that contributes to the care of the user's mind and body by providing a plant cultivation plan that takes into account the user's emotional state.

[0324] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0325] Step 1:

[0326] The user opens the application and takes a picture of a plant or bouquet using the device's camera, or selects an existing image from the photo gallery. When the user presses the "Post" button, the selected or taken image is temporarily saved on the device. The input is the taken or selected image, and the output is the temporarily saved image.

[0327] Step 2:

[0328] Based on the user's operation, the device sends the temporarily saved image and additional information (date and time of shooting, location information, emotional data) to the server. Emotional data is extracted from direct input from the user or facial expressions captured by the device's camera. The device then sends the acquired data to the server. The input is the image and additional information, and the output is the data sent to the server.

[0329] Step 3:

[0330] The server receives the image and additional information sent from the device. It then standardizes the image format and resizes it as necessary. Image standardization includes unifying the image format and adjusting the resolution. The input is the data received from the device, and the output is the standardized image and metadata.

[0331] Step 4:

[0332] The server uses an image recognition model on the standardized images to identify the plant type. This process is performed using machine learning models such as TensorFlow and PyTorch. The model identifies plant types with high accuracy based on the training dataset. The input is the standardized image, and the output is the plant type identification result.

[0333] Step 5:

[0334] Based on the identified plant type, the server extracts the growing conditions from the database. Growing conditions include watering frequency, sunlight requirements, type of fertilizer, etc. This data extraction is performed using SQL queries. The input is the plant identification results, and the output is the growing conditions for each plant.

[0335] Step 6:

[0336] When the server identifies multiple plants in an image, it integrates their respective growing conditions to generate a consistent growing plan. It compares the multiple growing conditions and adjusts and integrates them to find common and conflicting parts. The input is multiple growing conditions, and the output is an integrated growing plan.

[0337] Step 7:

[0338] The server uses an emotion analyzer to analyze the transmitted emotional data. It uses natural language processing and facial expression recognition algorithms to identify the user's emotional state (e.g., stress, joy, sadness). The input is the emotional data, and the output is the analyzed emotional state.

[0339] Step 8:

[0340] The server further customizes the integrated training plan based on the analyzed emotional state. For example, if the user is feeling stressed, it adds relaxing training advice or encouraging messages. The input is the training plan and the user's emotional state, and the output is a customized training plan.

[0341] Step 9:

[0342] The server transmits the generated customized development plan and encouraging message to the terminal. The input is the customized development plan and message, and the output is the data transmitted to the terminal.

[0343] Step 10:

[0344] The device notifies the user of the plant care plan and messages received and displays them on the app's user interface. Push notifications and reminder functions are used to encourage timely care. Users care for their plants according to the provided plant care plan and advice, and provide additional feedback to the app, allowing the emotion engine to track changes in the user's emotional state. The input is data received from the server, and the output is notification and display to the user, and feedback collection.

[0345] Through these steps, the system takes into account the user's emotional state, provides an optimal plant cultivation plan, and supports the user's physical and mental health.

[0346] (Application example 2)

[0347] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0348] Conventional plant-growing applications were able to identify plant species from images posted by users and provide information on growing conditions. However, these applications were unable to customize growing plans that took into account the user's emotional state, making it difficult to improve user satisfaction and care for their mind and body. Furthermore, when selecting plants in physical stores, there was a lack of support for users to choose plants that match the plant's characteristics and their own mental and physical state. This has created a need for applications that can provide both appropriate growing plans and care for the user's mind and body.

[0349] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0350] In this invention, the server includes a means for generating a customized plant growing plan based on the user's emotional state, a means for transmitting the generated plant growing plan and a message based on the user's emotion to the terminal, and a means for presenting the plant growing plan and message received by the terminal to the user. This makes it possible to provide a plant growing plan that takes the user's emotional state into consideration, and realizes a system that supports users in selecting appropriate plants in a physical store while also providing physical and mental care.

[0351] "User" refers to an individual who posts an image of a plant and receives a growing plan.

[0352] "Terminal" refers to an electronic device used by a User, including a smartphone, tablet, or computer.

[0353] "Image" means a visual representation of a plant photographed or selected by a User using a Device.

[0354] "Metadata" refers to additional information that accompanies an image, including the date and time the image was taken, location information, and the user's emotional data.

[0355] "Server" refers to a remote computer that receives, stores, and analyzes data sent from the terminal and generates training plans and messages.

[0356] "Image recognition model" refers to a program that implements a machine learning algorithm used to analyze images and identify plant species.

[0357] "Plant type" refers to the classification of a particular plant as identified by the image recognition model.

[0358] "Growth conditions" refers to information that indicates the optimal cultivation method based on the type of plant, and refers to data including watering frequency, sunlight conditions, and appropriate type of fertilizer.

[0359] A "growing plan" refers to a plan that integrates the growing conditions of the identified plants and indicates specific growing methods customized based on the user's emotional state.

[0360] "Emotional State" refers to the user's current mental state as analyzed using the user's emotion engine.

[0361] "Emotion-based messages" refer to words of encouragement or advice that are generated based on the user's emotional state.

[0362] A system for implementing the present invention includes an application for supporting plant growth and a configuration for providing customized advice based on emotional states. Specific embodiments of this system will be described below.

[0363] Hardware and software used

[0364] Device: A smartphone, tablet, or computer device through which users can post plant images.

[0365] Server: A remote computer that receives, stores, and analyzes data sent from the device.

[0366] Image recognition models: Use algorithms built using machine learning frameworks such as TensorFlow or PyTorch.

[0367] Sentiment engine: Uses natural language processing services such as IBM Watson and Google Cloud Natural Language API.

[0368] Database: Use a relational database such as PostgreSQL or MySQL.

[0369] Front-end: Building the application's user interface using React Native or Flutter.

[0370] Specific details of data processing and calculation

[0371] 1. User-submitted images

[0372] The device receives input from the user and sends the plant image and metadata (date and time of capture, location, and emotional data) to the server, where the user is prompted to enter their real-time emotional state.

[0373] 2. Data processing by the server

[0374] The server standardizes and resizes the received image data, identifies the plant type using an image recognition model, extracts growing conditions from a database based on the identified plant type, and then integrates the extracted growing conditions to generate a customized growing plan and message based on the user's emotional state.

[0375] 3. Notice to Users

[0376] The server sends the generated plant care plan and emotion-based messages to the device, which then presents them to the user, allowing the user to learn the optimal plant care method based on their emotional state.

[0377] Specific examples

[0378] For example, consider a case where a user takes a photo of a plant and then inputs their emotional state according to the following prompt:

[0379] Example prompt sentence:

[0380] "Please tell us about your recent stressful situations. Please talk about your feelings and worries."

[0381] If the user types "I'm tired," the server processing will generate a message like this:

[0382] Sample message:

[0383] "The plant chosen this time is the Monstera, which helps relieve stress. Make sure to water it regularly and provide it with adequate sunlight. When you're busy, just looking at the greenery of this plant can have a relaxing effect."

[0384] This system allows users to take their emotional state into consideration and implement optimal plant care methods, as well as receive advice that can help relieve daily stress.

[0385] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0386] Step 1:

[0387] User-submitted images

[0388] The user takes a photo of the plant using their device or selects an existing image, then reviews the image within the application and presses the submit button to send the image to the server.

[0389] Input: A user-taken or selected image of a plant.

[0390] Output: Image data is stored on the device and a request is generated to be sent to the server.

[0391] Step 2:

[0392] Sending images and metadata

[0393] The device sends the image selected by the user, along with the date and time of the photo, location information, and emotion data collected based on the prompt text, to the server.

[0394] Input: Plant image, photo date and time, location information, and user emotion data.

[0395] Output: The request is processed and the submitted data is saved on the server.

[0396] Step 3:

[0397] Image reception and preprocessing

[0398] The server receives the image and metadata sent from the device, normalizes the image data, and resizes it if necessary, so that the image is in a format suitable for the image recognition model.

[0399] Input: Received plant images and metadata.

[0400] Output: Normalized and resized image data.

[0401] Step 4:

[0402] Image Recognition Processing

[0403] The server uses an image recognition model to identify the type of plant from the received image. This model has been trained on many plant images in advance, allowing it to identify plant species with high accuracy.

[0404] Input: Standardized plant image data.

[0405] Output: The plant species identified.

[0406] Step 5:

[0407] Obtaining breeding conditions from a database

[0408] Based on the identified plant type, the server extracts from a database the optimal growing conditions for each plant, including watering frequency, light requirements, and the appropriate type of fertilizer.

[0409] Input: Identified plant species.

[0410] Output: Extracted growing condition data.

[0411] Step 6:

[0412] Integration of multiple plant conditions

[0413] If the image contains multiple plants, the server integrates the growing conditions of each plant and generates a consistent growing plan.

[0414] Input: Growing conditions for multiple plants.

[0415] Output: Integrated development plan.

[0416] Step 7:

[0417] Emotional Data Analysis

[0418] The server uses an emotion engine to analyze the user's emotional data, identify the user's emotional state (e.g., joy, sadness, stress, etc.), and generate appropriate advice or encouraging messages based on the data.

[0419] Input: User emotion data.

[0420] Output: User's emotional state data and an encouraging message.

[0421] Step 8:

[0422] Creation and integration of development plans

[0423] The server further customizes the planting plan by taking into account the user's emotional state, as analyzed by the emotion engine. For example, if the user is feeling stressed, the plan will include advice on plant care that has a relaxing effect.

[0424] Input: User emotional state data.

[0425] Output: A customized development plan.

[0426] Step 9:

[0427] Development plan and message sending

[0428] The server sends a message based on the generated development plan and emotion to the terminal.

[0429] Input: A customized development plan and encouraging message.

[0430] Output: The development plan and message sent.

[0431] Step 10:

[0432] User notification and action

[0433] The device displays the plant care plan and messages received on the app's user interface. Push notifications and reminder functions are used to prompt the user to provide timely care. The user can then care for their plants according to the provided care plan and emotional advice.

[0434] Input: Received development plans and messages.

[0435] Output: The development plan and messages displayed in the user interface.

[0436] In this way, each processing step is designed to perform appropriate calculations on data inputs and obtain customized outputs based on the user's emotional state.

[0437] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0438] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0439] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0440] [Second embodiment]

[0441] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0442] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0443] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0444] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0445] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0446] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0447] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0448] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0449] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0450] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0451] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0452] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0453] Users use the "My Flowerbed" application to manage the plants they grow at home or in the office. The system utilizes a server, user devices, and image recognition models as part of the platform.

[0454] Program processing flow and explanation

[0455] 1. User-submitted images

[0456] Users open the app and use their device's camera to take a picture of a plant or bouquet. In some cases, they can also select an existing image from their device's gallery. The image is displayed in the app, and users can then click the "Submit" button to send it to the server.

[0457] 2. Submitting images and metadata

[0458] The device collects the images selected by the user along with metadata such as the date and time of the photo and location, and transmits this data to a server over a secure protocol.

[0459] 3. Image Reception and Preprocessing

[0460] The server receives the images and metadata sent from the device, standardizes the image format, and resizes them if necessary, improving the processing efficiency of the image recognition model.

[0461] 4. Image Recognition Processing

[0462] The server then uses an image recognition model to identify the plant species from the received image. This model has been trained on many plant images in advance, allowing it to identify plant species with high accuracy.

[0463] 5. Obtaining breeding conditions from a database

[0464] Based on the identified plant type, the server extracts from a database the optimal growing conditions for each plant, including watering frequency, light requirements, and the appropriate type of fertilizer.

[0465] 6. Integration of multiple plant conditions

[0466] If an image contains multiple plants, the server integrates their respective growing conditions and generates a consistent growing plan. For example, if a Monstera and a rose are identified together, the server creates a plan based on their respective characteristics.

[0467] 7. Generate and submit a training plan

[0468] The generated cultivation plan is sent from the server to the device, and includes specific advice such as how often to water the plant, the amount of sunlight, and when to use fertilizer.

[0469] 8. User Notification and Execution

[0470] The device will notify the user of the received care plan, and the user can check detailed care instructions in the app and follow the instructions to care for their plants. The app also provides regular reminders to help users continue to care for their plants properly.

[0471] Specific examples

[0472] For example, if a user posts a photo of a Monstera plant in their living room and a vase of roses to the app, the following process occurs:

[0473] 1. The user takes a photo using the app and presses the post button.

[0474] 2. The device sends the image along with its metadata to the server.

[0475] 3. The server receives the image, normalizes it, and resizes it.

[0476] 4. The server uses an image recognition model to distinguish between a Monstera and a rose.

[0477] 5. The server retrieves the optimal growing conditions for each plant from the database.

[0478] 6. The server integrates the conditions for the Monstera and roses and generates a consistent cultivation plan.

[0479] 7. The server sends the development plan to the device.

[0480] 8. The device will notify the user of the development plan and provide specific care methods.

[0481] In this way, the system helps users easily care for multiple plants and provides optimal methods to ensure their longevity.

[0482] The processing flow will be explained below.

[0483] Program processing flow

[0484] Step 1:

[0485] Users open the "MY Flower Bed" app and use their device's camera to take a picture of the plants or bouquet they are growing, or select an existing image from their photo gallery.

[0486] Step 2:

[0487] The user presses the submit button to send the selected image to the server, optionally adding comments and metadata about the plant's condition.

[0488] Step 3:

[0489] The device acquires the image selected by the user and the associated metadata (e.g., shooting date and time, location information), and makes a request to the server to send the data.

[0490] Step 4:

[0491] The server receives the images and metadata sent, and first converts the file format and adjusts the image size to standardize the received images.

[0492] Step 5:

[0493] The server passes the standardized images to an image recognition algorithm, which uses a pre-trained convolutional neural network (CNN) or similar to identify the type of plant in the image.

[0494] Step 6:

[0495] The server analyzes multiple segments within an image and individually identifies the plants contained within each segment, allowing for accurate species identification even when multiple plants are present in a photo.

[0496] Step 7:

[0497] The server extracts the optimal growing conditions (such as watering frequency, amount of sunlight, and type of fertilizer) from a database based on the identified plant type.

[0498] Step 8:

[0499] The server comprehensively assesses the growing conditions of multiple plants and creates a consistent growing plan for the user, which is adjusted to take into account the characteristics of each plant.

[0500] Step 9:

[0501] The server formats the plan into specific instructions and sends them to the device, including instructions on when to water the plant, how much sunlight it needs, and how to properly fertilize it.

[0502] Step 10:

[0503] The device receives the training plan from the server and displays it on the app's user interface. Push notifications and alarms can also be used to set reminders to encourage timely care.

[0504] Step 11:

[0505] Users can care for their plants according to the provided care plan, and can also post images again to receive updated care plans as the situation evolves.

[0506] Example 1

[0507] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0508] Conventional plant growth management systems make it difficult for users to integrate individual conditions and generate consistent growth plans when managing multiple plants. They also have limited functionality for collecting detailed metadata along with plant images and providing regular reminders to users. This makes it difficult for users to effectively care for their plants, and it takes a lot of time and effort.

[0509] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0510] In this invention, the server includes means for standardizing and resizing the image format, means for integrating the growing conditions of the identified multiple plants to generate a consistent growing plan, and means for providing regular reminders to support the user in continuing appropriate care. This makes it easier for the user to manage multiple plants, enables the generation of a growing plan that integrates the optimal growing conditions for each plant, and enables the user to care for the plants efficiently and effectively.

[0511] "User" refers to a person who uses the system or application to manage and cultivate plants.

[0512] "Device" refers to the device a user uses to capture images of plants and collect and transmit metadata, including smartphones and tablets.

[0513] "Server" refers to a computer system that receives, stores, and processes images and metadata submitted by users.

[0514] "Image recognition model" refers to the machine learning or deep learning algorithm used to identify plant species from received images.

[0515] "Metadata" is data that includes information related to an image, specifically including the date and time of the image capture, location information, camera settings, and the like.

[0516] "Growing conditions" refers to the environmental factors that a particular plant needs to grow optimally, including watering frequency, light requirements, and type of fertilizer.

[0517] A "growth plan" refers to specific care methods created based on the optimal growing conditions for each type of plant.

[0518] "Periodic reminders" refers to a feature that sends regular notifications to users to encourage them to continue proper development.

[0519] "Normalization" refers to the process of converting received images into a consistent format so that subsequent processing can be done efficiently.

[0520] "Resizing" refers to the process of changing the size of an image, which improves the efficiency of image recognition models.

[0521] The present invention provides a system that allows users to easily and effectively manage and cultivate plants, and is implemented using a server, a terminal, and an image recognition model, allowing users to efficiently obtain information necessary for plant care.

[0522] First, a user installs an application for managing plants. When a user wants to post an image of a plant, they take or select an image using the device's camera or gallery. The device then collects the image along with metadata such as the date and time of the photo and its location, and sends it to a server via a secure protocol (e.g., HTTPS).

[0523] The server first stores the received images and metadata. Next, it standardizes the image format and resizes it as necessary. This allows for efficient processing by an image recognition model (e.g., a deep learning model using TensorFlow or PyTorch). The server then uses the image recognition model to identify the plant species from the received images. This model has been trained in advance on many plant images and can identify plant species with high accuracy.

[0524] After identifying the type of plant, the server extracts the optimal growing conditions for each plant from a database (e.g., a relational database such as MySQL). This includes watering frequency, sunlight requirements, and the appropriate type of fertilizer. If multiple plants are identified, their respective growing conditions are integrated to generate a consistent growing plan. For example, if a Monstera and a rose are identified together, a growing plan tailored to the characteristics of each will be created. The generated growing plan is sent from the server to the device, and the device notifies the user. The user can check detailed growing instructions through the app and practice appropriate care. The device also provides regular reminders to help the user continue to provide appropriate care.

[0525] Specific examples

[0526] For example, if a user posts a photo of a monstera plant in their living room and a vase of roses, the following process occurs:

[0527] 1. The user takes a photo using the app and presses the post button.

[0528] 2. The device sends the image along with its metadata to the server.

[0529] 3. The server receives the image, normalizes it, and resizes it.

[0530] 4. The server uses an image recognition model to distinguish between a Monstera and a rose.

[0531] 5. The server retrieves the optimal growing conditions for each plant from the database.

[0532] 6. The server integrates the conditions for the Monstera and roses and generates a consistent cultivation plan.

[0533] 7. The server sends the development plan to the device.

[0534] 8. The device will notify the user of the development plan and provide specific care methods.

[0535] In this way, the system helps users easily care for multiple plants and provides optimal methods to ensure their longevity.

[0536] Prompt Sentence Examples

[0537] "I would like to know how to care for the Monstera and roses I am growing at home. Please tell me the care plan for growing Monstera and roses at the same time."

[0538] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0539] Step 1: User submits image

[0540] 1. The user launches the app and the main screen displays the camera and gallery icons.

[0541] 2. The user taps the camera icon to launch the device's camera app and take a picture of the plant or bouquet.

[0542] 3. Input: An image of a plant taken with the device's camera.

[0543] 4. The user presses the shutter button to take a picture, which is then displayed on the preview screen.

[0544] 5. The user presses the "Post" button on the preview screen and the image is saved within the app.

[0545] 6. Output: Image data captured by the user.

[0546] Step 2: Submit your images and metadata

[0547] 1. The device obtains metadata, such as the date and time of the image taken and its location, for the image selected or taken by the user.

[0548] 2. Input: Images taken by the user, along with metadata such as the date and time of the image and its location.

[0549] 3. The device sends the image and metadata to the server over a secure protocol such as HTTPS.

[0550] 4. Output: Image data and metadata sent to the server.

[0551] Step 3: Receiving and preprocessing images

[0552] 1. The server receives the image and metadata sent from the device.

[0553] 2. Input: Image data and metadata sent from the device.

[0554] 3. The server standardizes the image format and resizes it as needed, for example converting the image width and height to a consistent number of pixels.

[0555] 4. Output: Resized and standardized image data.

[0556] Step 4: Image Recognition Processing

[0557] 1. The server inputs the preprocessed image into an image recognition model that has been trained on many plant images in advance.

[0558] 2. Input: Preprocessed image data.

[0559] 3. The server uses an image recognition model to identify the plant type.

[0560] 4. Output: Identified plant species data (e.g. Monstera, Rose).

[0561] Step 5: Obtaining breeding conditions from the database

[0562] 1. The server searches the database for growing conditions based on the identified plant type.

[0563] 2. Input: Identified plant species data.

[0564] 3. The server obtains growing conditions, including watering frequency, sunlight requirements, and appropriate type of fertilizer.

[0565] 4. Output: Growing condition data for each plant.

[0566] Step 6: Integrating multiple plant conditions

[0567] 1. When multiple plants are identified, the server integrates their respective growing conditions.

[0568] 2. Input: Individual growing condition data for multiple plants.

[0569] 3. The server analyzes the similarities and differences between each breeding condition and generates a consistent breeding plan.

[0570] 4. Output: Integrated development plan.

[0571] Step 7: Generate and submit your development plan

[0572] 1. The server generates specific advice based on an integrated growing plan, including watering frequency, amount of sunlight, and timing of fertilizer application.

[0573] 2. Input: Integrated Development Plan.

[0574] 3. The server sends this development plan to the terminal.

[0575] 4. Output: Development plan data sent to the terminal.

[0576] Step 8: Notify users and take action

[0577] 1. The terminal notifies the user of the development plan received from the server.

[0578] 2. Input: Development plan data from the server.

[0579] 3. The user views the notification and checks detailed training instructions within the app.

[0580] 4. The user follows the plant care instructions, such as watering a Monstera twice a week or placing a rose in a sunny spot.

[0581] 5. The device will provide regular reminders to help users continue to take proper care.

[0582] (Application example 1)

[0583] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0584] Conventional plant growth management systems are primarily used to manage plants in indoor environments, making it difficult to manage plants that adapt to the unique environmental conditions inside autonomous vehicles. Furthermore, there is a lack of technology that provides growth plans that take into account environmental factors such as temperature, humidity, and CO2 concentration inside the vehicle. This makes it difficult for users who want to grow plants inside their vehicles to provide an optimal growing environment, making it difficult to maintain the health of their plants.

[0585] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0586] In this invention, the server includes means for users to post plant images, means for a terminal to transmit the posted plant images and metadata to the server, means for the server to store the received images and metadata, means for the server to identify the type of plant using an image recognition model for the received images, means for the server to extract cultivation conditions from a database based on the identified plant type, means for the server to integrate the extracted cultivation conditions and generate an optimal cultivation plan for the user, means for transmitting the generated cultivation plan to the terminal, means for the terminal to present the received cultivation plan to the user, means for acquiring vehicle sensor data, and means for combining the acquired sensor data with plant cultivation conditions to generate an optimal cultivation plan, thereby enabling optimal plant cultivation management tailored to the in-vehicle environment of an autonomous vehicle.

[0587] "User" refers to an entity that uses this system to manage plants.

[0588] A "terminal" is a device used by a user that is equipped with means for taking images of plants and transmitting data.

[0589] The "server" is a device that receives, stores, and processes images and data sent from the terminal, and runs an image recognition model to identify plants and generate cultivation plans.

[0590] "Metadata" is additional information related to the main data, such as the date and time an image was taken and location information.

[0591] An "image recognition model" is a software model that runs on a server and is a technology used to identify plant species from submitted images.

[0592] A "database" is a collection of data that stores information about plant growing conditions.

[0593] A "growth plan" is a systematic compilation of the various conditions necessary for healthy plant growth (e.g., frequency of watering, sunlight conditions, timing of fertilizer application, etc.).

[0594] "Sensor data" refers to data obtained from sensors used to detect environmental information inside a vehicle, such as temperature, humidity, and CO2 concentration.

[0595] The "vehicle" is a means of transportation that utilizes autonomous driving technology and is equipped with environmental sensors to support plant growth.

[0596] "Standardization" is the process of converting incoming image data into a consistent format.

[0597] "Resizing" is the process of changing the size of image data to an appropriate size.

[0598] In the embodiment of the present invention, a system for managing plant growth inside an autonomous vehicle will be specifically described.

[0599] Users use a smartphone application called "CAR BOTANICA" to manage the plants installed in their vehicle. The user launches the application and takes a picture of the plant using the vehicle's camera. The image is then sent to a server via the device.

[0600] The server stores the received images and metadata (date and time of capture, location, sensor data), then uses an image recognition model to identify the plant species from the submitted image, using a machine learning framework such as TensorFlow.

[0601] Based on the identified plant type, the server extracts the growing conditions from a database. The database stores the growing conditions for various plants (watering frequency, sunlight conditions, fertilizer application timing, etc.). Furthermore, by combining this with sensor data (temperature, humidity, CO2 concentration), the server generates an optimal growing plan. This provides a growing plan that adapts to the unique environmental conditions inside the vehicle.

[0602] The development plan generated by the server is sent to the device (smartphone application). The user can check the development plan through the app and take appropriate care. The application also uses notification services such as Firebase to send reminders to the user and support them in taking appropriate care.

[0603] For example, if a user takes a picture of a Monstera plant in their car, the image is sent to a server where it is normalized and resized. An image recognition model then identifies the Monstera plant and extracts its optimal growing conditions from a database. This, combined with sensor data, generates a growing plan that, for example, notifies the user to open a window to ensure optimal temperature and humidity control.

[0604] Example prompt sentence:

[0605] Input plant images taken inside the car into a TensorFlow model to identify the plant type. Retrieve the identified plant's growing conditions from a database and combine them with sensor data to generate an optimal growing plan. Notify the driver of the generated growing plan using the Firebase notification service.

[0606] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0607] Step 1:

[0608] The user launches the smartphone application and takes a picture of a plant using the in-car camera. The application displays the image the user has taken and provides a posting button. The input is the image data captured by the camera, and the output is the image that the user can review and post.

[0609] Step 2:

[0610] The device sends images posted by users to the server, along with metadata such as the date and time of the photo, location information, and sensor data from inside the vehicle (temperature, humidity, CO2 concentration). The inputs are image data and metadata, and the output is this data sent to the server.

[0611] Step 3:

[0612] The server stores the received images and metadata. The server standardizes the image data and resizes it as necessary, improving the processing efficiency of the image recognition model. The inputs are the received images and metadata, and the standardized and resized image data is stored as the output.

[0613] Step 4:

[0614] The server uses an image recognition model to identify the plant species from the received image. It uses a machine learning framework such as TensorFlow to identify the plant species from the image. The input is the normalized and resized image data, and the output is the identified plant species.

[0615] Step 5:

[0616] The server extracts the growing conditions from a database based on the identified plant type. The database stores the growing conditions for various plants (watering frequency, sunlight conditions, fertilizer application timing, etc.). The input is the identified plant type, and the output is the extracted growing conditions.

[0617] Step 6:

[0618] The server combines the sensor data acquired with the plant's growing conditions to generate an optimal growing plan. For example, if the temperature and humidity inside the car do not meet certain conditions, it creates a growing plan that includes suggestions for providing an environment suitable for the plant (opening windows, adjusting temperature settings, etc.). The inputs are sensor data and growing conditions, and the output is an optimized growing plan.

[0619] Step 7:

[0620] The server sends the generated training plan to the device. The device receives this training plan and notifies the user. Services such as Firebase are used for notification. The input is the generated training plan, and the output is the training plan sent to the device.

[0621] Step 8:

[0622] The device receives the training plan and presents it to the user, informing them of specific care methods. The user can then check the detailed instructions through the app and provide appropriate care. The input is the received training plan, and the output is the training plan and care methods presented to the user.

[0623] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0624] This invention is a system that recognizes the user's emotional state by combining an emotion engine with the "MY Flower Bed" application, and provides care methods and encouraging messages based on that. This system consists of a server, a terminal, an image recognition model, an emotion engine, and a database.

[0625] Program processing flow and explanation

[0626] 1. User-submitted images

[0627] The user opens the application, takes a picture of a plant or bouquet using the device's camera, or selects an existing image from the photo gallery. The user then checks the image they have taken or selected within the application and presses the submit button to send the image to the server.

[0628] 2. Submitting images and metadata

[0629] The device acquires the image selected by the user along with the date and time of the photo, location information, and the user's emotional data, and sends them to the server.

[0630] 3. Image Reception and Preprocessing

[0631] The server receives the image, metadata, and emotion data sent from the device. First, it normalizes the image and resizes it if necessary.

[0632] 4. Image Recognition Processing

[0633] The server then uses an image recognition model to identify the plant species from the received image. This model has been trained on many plant images in advance, allowing it to identify plant species with high accuracy.

[0634] 5. Obtaining breeding conditions from a database

[0635] Based on the identified plant type, the server extracts from a database the optimal growing conditions for each plant, including watering frequency, light requirements, and the appropriate type of fertilizer.

[0636] 6. Integration of multiple plant conditions

[0637] If an image contains multiple plants, the server will integrate their respective growing conditions and generate a consistent growing plan. For example, if a Monstera and a rose are identified together, a growing plan will be created based on their respective characteristics.

[0638] 7. Emotion Data Analysis

[0639] The server uses an emotion engine to analyze the user's emotional data, identify the user's emotional state (e.g., joy, sadness, stress, etc.), and generate appropriate advice or encouraging messages based on this.

[0640] 8. Creation and integration of development plans

[0641] The server further customizes the planting plan by taking into account the user's emotional state, as analyzed by the emotion engine. For example, if the user is feeling stressed, the plan will include advice on plant care that has a relaxing effect.

[0642] 9. Development plan and message sending

[0643] The server generates a training plan and sends messages based on the emotions to the device, including specific care methods and encouraging messages.

[0644] 10. User Notice and Execution

[0645] The device displays the plant care plan and messages received on the app's user interface. Push notifications and reminder functions are used to prompt the user to care for their plants in a timely manner. The user then follows the plant care plan and emotion-based advice provided. Additional feedback can also be input into the app, allowing the emotion engine to continuously track changes in the user's emotional state.

[0646] Specific examples

[0647] For example, if a user posts a photo of a Monstera plant in their living room and a vase of roses to the app, the following process occurs:

[0648] 1. The user takes a picture using the application and presses the post button.

[0649] 2. The device sends the image, metadata, and user emotion data to the server.

[0650] 3. The server receives the image, normalizes it, and resizes it.

[0651] 4. The server uses an image recognition model to distinguish between a Monstera and a rose.

[0652] 5. The server retrieves the growing conditions for each plant from the database.

[0653] 6. The server integrates the growing conditions for Monstera and roses.

[0654] 7. The server analyzes the user's emotional data using an emotion engine and determines that the user is feeling stressed.

[0655] 8. The server creates a development plan by adding development advice that helps reduce stress.

[0656] 9. The server sends the training plan and encouraging messages to the device.

[0657] 10. The device notifies the user of the received development plan and message and displays it.

[0658] 11. The user will care for the plant in accordance with the growing plan and advice.

[0659] In this way, the system can provide optimal plant cultivation methods taking into account the user's emotional state, while also supporting the user's physical and mental care.

[0660] The processing flow will be explained below.

[0661] Program processing flow

[0662] Step 1:

[0663] The user opens the "MY Flower Bed" app and uses the device's camera to take a picture of a plant or bouquet, or select an existing image from the photo gallery.

[0664] Step 2:

[0665] The user presses the submit button to send the selected image to the server, optionally including emotional data (e.g., emotion estimation via an in-app survey or facial recognition).

[0666] Step 3:

[0667] The device acquires the image selected by the user along with metadata such as the date and time of shooting, location information, and emotional data, and makes a request to the server to send the data.

[0668] Step 4:

[0669] The server receives the images, metadata, and emotion data, normalizes the received images, and resizes them if necessary.

[0670] Step 5:

[0671] The server passes the standardized images to an image recognition algorithm, which uses a pre-trained convolutional neural network (CNN) to identify the type of plant in the image.

[0672] Step 6:

[0673] The server analyzes multiple segments within an image and individually identifies the plants contained within each segment, allowing for accurate species identification even when multiple plants are present in a photo.

[0674] Step 7:

[0675] The server extracts the optimal growing conditions (such as watering frequency, amount of sunlight, and type of fertilizer) from a database based on the identified plant type.

[0676] Step 8:

[0677] The server comprehensively assesses the growing conditions of multiple plants and creates a consistent growing plan for the user, which is adjusted to take into account the characteristics of each plant.

[0678] Step 9:

[0679] The server analyzes the user-provided emotion data using an emotion engine, which identifies the user's emotional state (e.g., joy, sadness, stress, etc.).

[0680] Step 10:

[0681] The server then customizes the plant care plan based on information gleaned from the emotional data, adding relaxing plant care advice if the user is feeling stressed, for example.

[0682] Step 11:

[0683] The server generates emotion-based messages (encouragement, positive feedback, etc.) and includes them in the development plan.

[0684] Step 12:

[0685] The server sends a customized development plan and a message based on the emotion to the terminal.

[0686] Step 13:

[0687] The device displays the received training plan and message on the user interface, and uses push notifications and reminder functions to encourage users to take timely care.

[0688] Step 14:

[0689] Users follow the advice based on the presented plant care plan and their emotions. If necessary, they can resubmit images and receive updated care plans. This allows users to not only manage the health of their plants, but also their own emotions at the same time.

[0690] Example 2

[0691] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0692] Conventional plant cultivation support systems simply provide plant cultivation conditions without considering the user's emotional state, and do not contribute to the user's mental care or stress reduction. Therefore, there is a need for a system that takes the user's emotional state into consideration and provides individually customized cultivation plans and encouraging messages to support the user's mental and physical health.

[0693] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0694] In this invention, the server includes means for users to post plant images, means for a terminal to transmit the posted plant images and additional information to the server, means for the server to store the received images and additional information, means for the server to identify the type of plant using an image recognition model for the received images, means for the server to extract cultivation conditions from a database based on the identified plant type, means for the server to integrate the extracted cultivation conditions and generate an optimal cultivation plan for the user, means for the server to transmit the generated cultivation plan to the terminal, means for the terminal to present the cultivation plan received by the terminal to the user, means for the server to analyze the user's emotional state using an emotion analysis device, and means for the server to customize the cultivation plan based on the analyzed emotional state and generate advice and encouraging messages associated with the emotions. This enables the provision of a plant cultivation plan that takes the user's emotional state into consideration and emotional care.

[0695] A "user" is someone who posts images of plants and receives advice from the system, including growing plans and emotionally relevant advice.

[0696] "Terminal" refers to the device used by the user to take images of plants, send the data to the server, and receive and display cultivation plans and advice.

[0697] "Server" refers to a central device that receives, stores, processes images and data sent from a terminal, and generates and sends development plans and advice to the terminal.

[0698] "Additional information" refers to additional metadata sent with an image, such as the date and time of the image capture, location information, or user emotional data.

[0699] "Image Recognition Model" refers to a model that uses machine learning algorithms to identify the type of plant contained in a received image.

[0700] "Database" refers to a system for storing and managing plant growing conditions and other related information.

[0701] "Growth conditions" refers to information including watering frequency, sunlight conditions, type of fertilizer, etc., as conditions for optimal growth of a particular plant.

[0702] "Growth plan" refers to a plan that integrates growing conditions extracted from the database and provides specific plant care methods to the user.

[0703] "Emotion analysis device" refers to hardware or software used to analyze a user's emotional state.

[0704] "Messages of advice and encouragement" refer to advice and support messages that are generated based on the user's emotional state and provided to the user.

[0705] This invention is a plant-growing support system that takes into account the user's emotional state. This system is composed of a server, a terminal, an image recognition model, an emotion analysis device, and a database, and provides users with individually customized plant-growing plans and advice.

[0706] Hardware and Software Configuration

[0707] Terminal

[0708] The terminal is a smartphone or tablet that allows users to take pictures of plants, send the data to the server, and receive and display cultivation plans and advice. The terminal is equipped with a camera, GPS module, and internet connection.

[0709] server

[0710] The server is a central device for storing and processing the received images and additional information. The server is equipped with an image recognition model, an emotion analysis device, and a database.

[0711] Image recognition model: Uses machine learning algorithms such as TensorFlow and PyTorch to identify the type of plant from the received image.

[0712] Emotion analysis device: A device that uses natural language processing (NLP) and facial expression recognition algorithms to analyze a user's emotional state.

[0713] Database: A system that stores and manages plant growth conditions and related information.

[0714] Data processing and calculation flow

[0715] 1. Image Reception and Preprocessing

[0716] The device sends the plant image and additional information (date and time of the photo, location information, and emotion data) to the server, which then receives the image and standardizes it by converting its format and resizing it.

[0717] 2. Image Recognition Processing

[0718] The server uses image recognition models to identify plant species, which identifies the names and genus species of plants contained in the image.

[0719] 3. Extraction of growth conditions

[0720] Based on the identified plant, the server extracts growing conditions (such as watering frequency, sunlight conditions, and type of fertilizer) from the database.

[0721] 5. Emotion Data Analysis

[0722] The server uses an emotion analyzer to identify the user's emotional state from their input data and facial expressions. For example, it may determine that the user is "stressed" based on their text input.

[0723] 6. Creating and customizing training plans

[0724] The server then combines the extracted training conditions with the emotional data to generate an optimal training plan for the user, adding customized advice and encouraging messages according to the user's emotional state.

[0725] 7. Development plan and message sending

[0726] The server sends the generated training plan and message to the terminal, which receives it and notifies and displays it to the user.

[0727] Specific examples

[0728] For example, if a user posts a photo of a Monstera plant in their living room and a vase of roses, the following happens:

[0729] 1. The user takes a photo of a plant using the in-app camera, then presses the "Post" button to send the image.

[0730] 2. The device sends the captured image and additional information (date and time of capture, location information, and emotion data) to the server.

[0731] 3. The server receives the image, normalizes it, and resizes it.

[0732] 4. The server uses image recognition models to identify the Monstera and the rose, and their plant names are identified.

[0733] 5. The server retrieves the growing conditions for the identified plants from the database. For example, a Monstera should be watered every two weeks and indirect sunlight, while a rose should be watered once a week and in direct sunlight.

[0734] 6. The server uses an emotion analyzer to analyze whether the user is feeling "stressed."

[0735] 7. The server generates a care plan by adding advice to help reduce stress. For example, it adds an encouraging message such as, "Watering this Monstera every morning will help you start your day relaxing."

[0736] 8. The server sends the final training plan and encouraging messages to the device.

[0737] 9. The device displays the training plan and message received within the app and notifies the user.

[0738] Prompt Sentence Examples

[0739] Use a prompt like this for the emotion engine:

[0740] "The user's current emotional state is stressed. Generate advice suggesting how to care for plants to reduce stress."

[0741] As described above, the present invention is a system that contributes to the care of the user's mind and body by providing a plant cultivation plan that takes into account the user's emotional state.

[0742] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0743] Step 1:

[0744] The user opens the application and takes a picture of a plant or bouquet using the device's camera, or selects an existing image from the photo gallery. When the user presses the "Post" button, the selected or taken image is temporarily saved on the device. The input is the taken or selected image, and the output is the temporarily saved image.

[0745] Step 2:

[0746] Based on the user's operation, the device sends the temporarily saved image and additional information (date and time of shooting, location information, emotional data) to the server. Emotional data is extracted from direct input from the user or facial expressions captured by the device's camera. The device then sends the acquired data to the server. The input is the image and additional information, and the output is the data sent to the server.

[0747] Step 3:

[0748] The server receives the image and additional information sent from the device. It then standardizes the image format and resizes it as necessary. Image standardization includes unifying the image format and adjusting the resolution. The input is the data received from the device, and the output is the standardized image and metadata.

[0749] Step 4:

[0750] The server uses an image recognition model on the standardized images to identify the plant type. This process is performed using machine learning models such as TensorFlow and PyTorch. The model identifies plant types with high accuracy based on the training dataset. The input is the standardized image, and the output is the plant type identification result.

[0751] Step 5:

[0752] Based on the identified plant type, the server extracts the growing conditions from the database. Growing conditions include watering frequency, sunlight requirements, type of fertilizer, etc. This data extraction is performed using SQL queries. The input is the plant identification results, and the output is the growing conditions for each plant.

[0753] Step 6:

[0754] When the server identifies multiple plants in an image, it integrates their respective growing conditions to generate a consistent growing plan. It compares the multiple growing conditions and adjusts and integrates them to find common and conflicting parts. The input is multiple growing conditions, and the output is an integrated growing plan.

[0755] Step 7:

[0756] The server uses an emotion analyzer to analyze the transmitted emotional data. It uses natural language processing and facial expression recognition algorithms to identify the user's emotional state (e.g., stress, joy, sadness). The input is the emotional data, and the output is the analyzed emotional state.

[0757] Step 8:

[0758] The server further customizes the integrated training plan based on the analyzed emotional state. For example, if the user is feeling stressed, it adds relaxing training advice or encouraging messages. The input is the training plan and the user's emotional state, and the output is a customized training plan.

[0759] Step 9:

[0760] The server transmits the generated customized development plan and encouraging message to the terminal. The input is the customized development plan and message, and the output is the data transmitted to the terminal.

[0761] Step 10:

[0762] The device notifies the user of the plant care plan and messages received and displays them on the app's user interface. Push notifications and reminder functions are used to encourage timely care. Users care for their plants according to the provided plant care plan and advice, and provide additional feedback to the app, allowing the emotion engine to track changes in the user's emotional state. The input is data received from the server, and the output is notification and display to the user, and feedback collection.

[0763] Through these steps, the system takes into account the user's emotional state, provides an optimal plant cultivation plan, and supports the user's physical and mental health.

[0764] (Application example 2)

[0765] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0766] Conventional plant-growing applications were able to identify plant species from images posted by users and provide information on growing conditions. However, these applications were unable to customize growing plans that took into account the user's emotional state, making it difficult to improve user satisfaction and care for their mind and body. Furthermore, when selecting plants in physical stores, there was a lack of support for users to choose plants that match the plant's characteristics and their own mental and physical state. This has created a need for applications that can provide both appropriate growing plans and care for the user's mind and body.

[0767] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0768] In this invention, the server includes a means for generating a customized plant growing plan based on the user's emotional state, a means for transmitting the generated plant growing plan and a message based on the user's emotion to the terminal, and a means for presenting the plant growing plan and message received by the terminal to the user. This makes it possible to provide a plant growing plan that takes the user's emotional state into consideration, and realizes a system that supports users in selecting appropriate plants in a physical store while also providing physical and mental care.

[0769] "User" refers to an individual who posts an image of a plant and receives a growing plan.

[0770] "Terminal" refers to an electronic device used by a User, including a smartphone, tablet, or computer.

[0771] "Image" means a visual representation of a plant photographed or selected by a User using a Device.

[0772] "Metadata" refers to additional information that accompanies an image, including the date and time the image was taken, location information, and the user's emotional data.

[0773] "Server" refers to a remote computer that receives, stores, and analyzes data sent from the terminal and generates training plans and messages.

[0774] "Image recognition model" refers to a program that implements a machine learning algorithm used to analyze images and identify plant species.

[0775] "Plant type" refers to the classification of a particular plant as identified by the image recognition model.

[0776] "Growth conditions" refers to information that indicates the optimal cultivation method based on the type of plant, and refers to data including watering frequency, sunlight conditions, and appropriate type of fertilizer.

[0777] A "growing plan" refers to a plan that integrates the growing conditions of the identified plants and indicates specific growing methods customized based on the user's emotional state.

[0778] "Emotional State" refers to the user's current mental state as analyzed using the user's emotion engine.

[0779] "Emotion-based messages" refer to words of encouragement or advice that are generated based on the user's emotional state.

[0780] A system for implementing the present invention includes an application for supporting plant growth and a configuration for providing customized advice based on emotional states. Specific embodiments of this system will be described below.

[0781] Hardware and software used

[0782] Device: A smartphone, tablet, or computer device through which users can post plant images.

[0783] Server: A remote computer that receives, stores, and analyzes data sent from the device.

[0784] Image recognition models: Use algorithms built using machine learning frameworks such as TensorFlow or PyTorch.

[0785] Sentiment engine: Uses natural language processing services such as IBM Watson and Google Cloud Natural Language API.

[0786] Database: Use a relational database such as PostgreSQL or MySQL.

[0787] Front-end: Building the application's user interface using React Native or Flutter.

[0788] Specific details of data processing and calculation

[0789] 1. User-submitted images

[0790] The device receives input from the user and sends the plant image and metadata (date and time of capture, location, and emotional data) to the server, where the user is prompted to enter their real-time emotional state.

[0791] 2. Data processing by the server

[0792] The server standardizes and resizes the received image data, identifies the plant type using an image recognition model, extracts growing conditions from a database based on the identified plant type, and then integrates the extracted growing conditions to generate a customized growing plan and message based on the user's emotional state.

[0793] 3. Notice to Users

[0794] The server sends the generated plant care plan and emotion-based messages to the device, which then presents them to the user, allowing the user to learn the optimal plant care method based on their emotional state.

[0795] Specific examples

[0796] For example, consider a case where a user takes a photo of a plant and then inputs their emotional state according to the following prompt:

[0797] Example prompt sentence:

[0798] "Please tell us about your recent stressful situations. Please talk about your feelings and worries."

[0799] If the user types "I'm tired," the server processing will generate a message like this:

[0800] Sample message:

[0801] "The plant chosen this time is the Monstera, which helps relieve stress. Make sure to water it regularly and provide it with adequate sunlight. When you're busy, just looking at the greenery of this plant can have a relaxing effect."

[0802] This system allows users to take their emotional state into consideration and implement optimal plant care methods, as well as receive advice that can help relieve daily stress.

[0803] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0804] Step 1:

[0805] User-submitted images

[0806] The user takes a photo of the plant using their device or selects an existing image, then reviews the image within the application and presses the submit button to send the image to the server.

[0807] Input: A user-taken or selected image of a plant.

[0808] Output: Image data is stored on the device and a request is generated to be sent to the server.

[0809] Step 2:

[0810] Sending images and metadata

[0811] The device sends the image selected by the user, along with the date and time of the photo, location information, and emotion data collected based on the prompt text, to the server.

[0812] Input: Plant image, photo date and time, location information, and user emotion data.

[0813] Output: The request is processed and the submitted data is saved on the server.

[0814] Step 3:

[0815] Image reception and preprocessing

[0816] The server receives the image and metadata sent from the device, normalizes the image data, and resizes it if necessary, so that the image is in a format suitable for the image recognition model.

[0817] Input: Received plant images and metadata.

[0818] Output: Normalized and resized image data.

[0819] Step 4:

[0820] Image Recognition Processing

[0821] The server uses an image recognition model to identify the type of plant from the received image. This model has been trained on many plant images in advance, allowing it to identify plant species with high accuracy.

[0822] Input: Standardized plant image data.

[0823] Output: The plant species identified.

[0824] Step 5:

[0825] Obtaining breeding conditions from a database

[0826] Based on the identified plant type, the server extracts from a database the optimal growing conditions for each plant, including watering frequency, light requirements, and the appropriate type of fertilizer.

[0827] Input: Identified plant species.

[0828] Output: Extracted growing condition data.

[0829] Step 6:

[0830] Integration of multiple plant conditions

[0831] If the image contains multiple plants, the server integrates the growing conditions of each plant and generates a consistent growing plan.

[0832] Input: Growing conditions for multiple plants.

[0833] Output: Integrated development plan.

[0834] Step 7:

[0835] Emotional Data Analysis

[0836] The server uses an emotion engine to analyze the user's emotional data, identify the user's emotional state (e.g., joy, sadness, stress, etc.), and generate appropriate advice or encouraging messages based on the data.

[0837] Input: User emotion data.

[0838] Output: User's emotional state data and an encouraging message.

[0839] Step 8:

[0840] Creation and integration of development plans

[0841] The server further customizes the planting plan by taking into account the user's emotional state, as analyzed by the emotion engine. For example, if the user is feeling stressed, the plan will include advice on plant care that has a relaxing effect.

[0842] Input: User emotional state data.

[0843] Output: A customized development plan.

[0844] Step 9:

[0845] Development plan and message sending

[0846] The server sends a message based on the generated development plan and emotion to the terminal.

[0847] Input: A customized development plan and encouraging message.

[0848] Output: The development plan and message sent.

[0849] Step 10:

[0850] User notification and action

[0851] The device displays the plant care plan and messages received on the app's user interface. Push notifications and reminder functions are used to prompt the user to provide timely care. The user can then care for their plants according to the provided care plan and emotional advice.

[0852] Input: Received development plans and messages.

[0853] Output: The development plan and messages displayed in the user interface.

[0854] In this way, each processing step is designed to perform appropriate calculations on data inputs and obtain customized outputs based on the user's emotional state.

[0855] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0856] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0857] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0858] [Third embodiment]

[0859] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0860] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0861] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0862] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0863] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0864] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0865] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0866] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0867] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0868] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0869] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0870] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0871] Users use the "My Flowerbed" application to manage the plants they grow at home or in the office. The system utilizes a server, user devices, and image recognition models as part of the platform.

[0872] Program processing flow and explanation

[0873] 1. User-submitted images

[0874] Users open the app and use their device's camera to take a picture of a plant or bouquet. In some cases, they can also select an existing image from their device's gallery. The image is displayed in the app, and users can then click the "Submit" button to send it to the server.

[0875] 2. Submitting images and metadata

[0876] The device collects the images selected by the user along with metadata such as the date and time of the photo and location, and transmits this data to a server over a secure protocol.

[0877] 3. Image Reception and Preprocessing

[0878] The server receives the images and metadata sent from the device, standardizes the image format, and resizes them if necessary, improving the processing efficiency of the image recognition model.

[0879] 4. Image Recognition Processing

[0880] The server then uses an image recognition model to identify the plant species from the received image. This model has been trained on many plant images in advance, allowing it to identify plant species with high accuracy.

[0881] 5. Obtaining breeding conditions from a database

[0882] Based on the identified plant type, the server extracts from a database the optimal growing conditions for each plant, including watering frequency, light requirements, and the appropriate type of fertilizer.

[0883] 6. Integration of multiple plant conditions

[0884] If an image contains multiple plants, the server integrates their respective growing conditions and generates a consistent growing plan. For example, if a Monstera and a rose are identified together, the server creates a plan based on their respective characteristics.

[0885] 7. Generate and submit a training plan

[0886] The generated cultivation plan is sent from the server to the device, and includes specific advice such as how often to water the plant, the amount of sunlight, and when to use fertilizer.

[0887] 8. User Notification and Execution

[0888] The device will notify the user of the received care plan, and the user can check detailed care instructions in the app and follow the instructions to care for their plants. The app also provides regular reminders to help users continue to care for their plants properly.

[0889] Specific examples

[0890] For example, if a user posts a photo of a Monstera plant in their living room and a vase of roses to the app, the following process occurs:

[0891] 1. The user takes a photo using the app and presses the post button.

[0892] 2. The device sends the image along with its metadata to the server.

[0893] 3. The server receives the image, normalizes it, and resizes it.

[0894] 4. The server uses an image recognition model to distinguish between a Monstera and a rose.

[0895] 5. The server retrieves the optimal growing conditions for each plant from the database.

[0896] 6. The server integrates the conditions for the Monstera and roses and generates a consistent cultivation plan.

[0897] 7. The server sends the development plan to the device.

[0898] 8. The device will notify the user of the development plan and provide specific care methods.

[0899] In this way, the system helps users easily care for multiple plants and provides optimal methods to ensure their longevity.

[0900] The processing flow will be explained below.

[0901] Program processing flow

[0902] Step 1:

[0903] Users open the "MY Flower Bed" app and use their device's camera to take a picture of the plants or bouquet they are growing, or select an existing image from their photo gallery.

[0904] Step 2:

[0905] The user presses the submit button to send the selected image to the server, optionally adding comments and metadata about the plant's condition.

[0906] Step 3:

[0907] The device acquires the image selected by the user and the associated metadata (e.g., shooting date and time, location information), and makes a request to the server to send the data.

[0908] Step 4:

[0909] The server receives the images and metadata sent, and first converts the file format and adjusts the image size to standardize the received images.

[0910] Step 5:

[0911] The server passes the standardized images to an image recognition algorithm, which uses a pre-trained convolutional neural network (CNN) or similar to identify the type of plant in the image.

[0912] Step 6:

[0913] The server analyzes multiple segments within an image and individually identifies the plants contained within each segment, allowing for accurate species identification even when multiple plants are present in a photo.

[0914] Step 7:

[0915] The server extracts the optimal growing conditions (such as watering frequency, amount of sunlight, and type of fertilizer) from a database based on the identified plant type.

[0916] Step 8:

[0917] The server comprehensively assesses the growing conditions of multiple plants and creates a consistent growing plan for the user, which is adjusted to take into account the characteristics of each plant.

[0918] Step 9:

[0919] The server formats the plan into specific instructions and sends them to the device, including instructions on when to water the plant, how much sunlight it needs, and how to properly fertilize it.

[0920] Step 10:

[0921] The device receives the training plan from the server and displays it on the app's user interface. Push notifications and alarms can also be used to set reminders to encourage timely care.

[0922] Step 11:

[0923] Users can care for their plants according to the provided care plan, and can also post images again to receive updated care plans as the situation evolves.

[0924] Example 1

[0925] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0926] Conventional plant growth management systems make it difficult for users to integrate individual conditions and generate consistent growth plans when managing multiple plants. They also have limited functionality for collecting detailed metadata along with plant images and providing regular reminders to users. This makes it difficult for users to effectively care for their plants, and it takes a lot of time and effort.

[0927] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0928] In this invention, the server includes means for standardizing and resizing the image format, means for integrating the growing conditions of the identified multiple plants to generate a consistent growing plan, and means for providing regular reminders to support the user in continuing appropriate care. This makes it easier for the user to manage multiple plants, enables the generation of a growing plan that integrates the optimal growing conditions for each plant, and enables the user to care for the plants efficiently and effectively.

[0929] "User" refers to a person who uses the system or application to manage and cultivate plants.

[0930] "Device" refers to the device a user uses to capture images of plants and collect and transmit metadata, including smartphones and tablets.

[0931] "Server" refers to a computer system that receives, stores, and processes images and metadata submitted by users.

[0932] "Image recognition model" refers to the machine learning or deep learning algorithm used to identify plant species from received images.

[0933] "Metadata" is data that includes information related to an image, specifically including the date and time of the image capture, location information, camera settings, and the like.

[0934] "Growing conditions" refers to the environmental factors that a particular plant needs to grow optimally, including watering frequency, light requirements, and type of fertilizer.

[0935] A "growth plan" refers to specific care methods created based on the optimal growing conditions for each type of plant.

[0936] "Periodic reminders" refers to a feature that sends regular notifications to users to encourage them to continue proper development.

[0937] "Normalization" refers to the process of converting received images into a consistent format so that subsequent processing can be done efficiently.

[0938] "Resizing" refers to the process of changing the size of an image, which improves the efficiency of image recognition models.

[0939] The present invention provides a system that allows users to easily and effectively manage and cultivate plants, and is implemented using a server, a terminal, and an image recognition model, allowing users to efficiently obtain information necessary for plant care.

[0940] First, a user installs an application for managing plants. When a user wants to post an image of a plant, they take or select an image using the device's camera or gallery. The device then collects the image along with metadata such as the date and time of the photo and its location, and sends it to a server via a secure protocol (e.g., HTTPS).

[0941] The server first stores the received images and metadata. Next, it standardizes the image format and resizes it as necessary. This allows for efficient processing by an image recognition model (e.g., a deep learning model using TensorFlow or PyTorch). The server then uses the image recognition model to identify the plant species from the received images. This model has been trained in advance on many plant images and can identify plant species with high accuracy.

[0942] After identifying the type of plant, the server extracts the optimal growing conditions for each plant from a database (e.g., a relational database such as MySQL). This includes watering frequency, sunlight requirements, and the appropriate type of fertilizer. If multiple plants are identified, their respective growing conditions are integrated to generate a consistent growing plan. For example, if a Monstera and a rose are identified together, a growing plan tailored to the characteristics of each will be created. The generated growing plan is sent from the server to the device, and the device notifies the user. The user can check detailed growing instructions through the app and practice appropriate care. The device also provides regular reminders to help the user continue to provide appropriate care.

[0943] Specific examples

[0944] For example, if a user posts a photo of a monstera plant in their living room and a vase of roses, the following process occurs:

[0945] 1. The user takes a photo using the app and presses the post button.

[0946] 2. The device sends the image along with its metadata to the server.

[0947] 3. The server receives the image, normalizes it, and resizes it.

[0948] 4. The server uses an image recognition model to distinguish between a Monstera and a rose.

[0949] 5. The server retrieves the optimal growing conditions for each plant from the database.

[0950] 6. The server integrates the conditions for the Monstera and roses and generates a consistent cultivation plan.

[0951] 7. The server sends the development plan to the device.

[0952] 8. The device will notify the user of the development plan and provide specific care methods.

[0953] In this way, the system helps users easily care for multiple plants and provides optimal methods to ensure their longevity.

[0954] Prompt Sentence Examples

[0955] "I would like to know how to care for the Monstera and roses I am growing at home. Please tell me the care plan for growing Monstera and roses at the same time."

[0956] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0957] Step 1: User submits image

[0958] 1. The user launches the app and the main screen displays the camera and gallery icons.

[0959] 2. The user taps the camera icon to launch the device's camera app and take a picture of the plant or bouquet.

[0960] 3. Input: An image of a plant taken with the device's camera.

[0961] 4. The user presses the shutter button to take a picture, which is then displayed on the preview screen.

[0962] 5. The user presses the "Post" button on the preview screen and the image is saved within the app.

[0963] 6. Output: Image data captured by the user.

[0964] Step 2: Submit your images and metadata

[0965] 1. The device obtains metadata, such as the date and time of the image taken and its location, for the image selected or taken by the user.

[0966] 2. Input: Images taken by the user, along with metadata such as the date and time of the image and its location.

[0967] 3. The device sends the image and metadata to the server over a secure protocol such as HTTPS.

[0968] 4. Output: Image data and metadata sent to the server.

[0969] Step 3: Receiving and preprocessing images

[0970] 1. The server receives the image and metadata sent from the device.

[0971] 2. Input: Image data and metadata sent from the device.

[0972] 3. The server standardizes the image format and resizes it as needed, for example converting the image width and height to a consistent number of pixels.

[0973] 4. Output: Resized and standardized image data.

[0974] Step 4: Image Recognition Processing

[0975] 1. The server inputs the preprocessed image into an image recognition model that has been trained on many plant images in advance.

[0976] 2. Input: Preprocessed image data.

[0977] 3. The server uses an image recognition model to identify the plant type.

[0978] 4. Output: Identified plant species data (e.g. Monstera, Rose).

[0979] Step 5: Obtaining breeding conditions from the database

[0980] 1. The server searches the database for growing conditions based on the identified plant type.

[0981] 2. Input: Identified plant species data.

[0982] 3. The server obtains growing conditions, including watering frequency, sunlight requirements, and appropriate type of fertilizer.

[0983] 4. Output: Growing condition data for each plant.

[0984] Step 6: Integrating multiple plant conditions

[0985] 1. When multiple plants are identified, the server integrates their respective growing conditions.

[0986] 2. Input: Individual growing condition data for multiple plants.

[0987] 3. The server analyzes the similarities and differences between each breeding condition and generates a consistent breeding plan.

[0988] 4. Output: Integrated development plan.

[0989] Step 7: Generate and submit your development plan

[0990] 1. The server generates specific advice based on an integrated growing plan, including watering frequency, amount of sunlight, and timing of fertilizer application.

[0991] 2. Input: Integrated Development Plan.

[0992] 3. The server sends this development plan to the terminal.

[0993] 4. Output: Development plan data sent to the terminal.

[0994] Step 8: Notify users and take action

[0995] 1. The terminal notifies the user of the development plan received from the server.

[0996] 2. Input: Development plan data from the server.

[0997] 3. The user views the notification and checks detailed training instructions within the app.

[0998] 4. The user follows the plant care instructions, such as watering a Monstera twice a week or placing a rose in a sunny spot.

[0999] 5. The device will provide regular reminders to help users continue to take proper care.

[1000] (Application example 1)

[1001] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1002] Conventional plant growth management systems are primarily used to manage plants in indoor environments, making it difficult to manage plants that adapt to the unique environmental conditions inside autonomous vehicles. Furthermore, there is a lack of technology that provides growth plans that take into account environmental factors such as temperature, humidity, and CO2 concentration inside the vehicle. This makes it difficult for users who want to grow plants inside their vehicles to provide an optimal growing environment, making it difficult to maintain the health of their plants.

[1003] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1004] In this invention, the server includes means for users to post plant images, means for a terminal to transmit the posted plant images and metadata to the server, means for the server to store the received images and metadata, means for the server to identify the type of plant using an image recognition model for the received images, means for the server to extract cultivation conditions from a database based on the identified plant type, means for the server to integrate the extracted cultivation conditions and generate an optimal cultivation plan for the user, means for transmitting the generated cultivation plan to the terminal, means for the terminal to present the received cultivation plan to the user, means for acquiring vehicle sensor data, and means for combining the acquired sensor data with plant cultivation conditions to generate an optimal cultivation plan, thereby enabling optimal plant cultivation management tailored to the in-vehicle environment of an autonomous vehicle.

[1005] "User" refers to an entity that uses this system to manage plants.

[1006] A "terminal" is a device used by a user that is equipped with means for taking images of plants and transmitting data.

[1007] The "server" is a device that receives, stores, and processes images and data sent from the terminal, and runs an image recognition model to identify plants and generate cultivation plans.

[1008] "Metadata" is additional information related to the main data, such as the date and time an image was taken and location information.

[1009] An "image recognition model" is a software model that runs on a server and is a technology used to identify plant species from submitted images.

[1010] A "database" is a collection of data that stores information about plant growing conditions.

[1011] A "growth plan" is a systematic compilation of the various conditions necessary for healthy plant growth (e.g., frequency of watering, sunlight conditions, timing of fertilizer application, etc.).

[1012] "Sensor data" refers to data obtained from sensors used to detect environmental information inside a vehicle, such as temperature, humidity, and CO2 concentration.

[1013] The "vehicle" is a means of transportation that utilizes autonomous driving technology and is equipped with environmental sensors to support plant growth.

[1014] "Standardization" is the process of converting incoming image data into a consistent format.

[1015] "Resizing" is the process of changing the size of image data to an appropriate size.

[1016] In the embodiment of the present invention, a system for managing plant growth inside an autonomous vehicle will be specifically described.

[1017] Users use a smartphone application called "CAR BOTANICA" to manage the plants installed in their vehicle. The user launches the application and takes a picture of the plant using the vehicle's camera. The image is then sent to a server via the device.

[1018] The server stores the received images and metadata (date and time of capture, location, sensor data), then uses an image recognition model to identify the plant species from the submitted image, using a machine learning framework such as TensorFlow.

[1019] Based on the identified plant type, the server extracts the growing conditions from a database. The database stores the growing conditions for various plants (watering frequency, sunlight conditions, fertilizer application timing, etc.). Furthermore, by combining this with sensor data (temperature, humidity, CO2 concentration), the server generates an optimal growing plan. This provides a growing plan that adapts to the unique environmental conditions inside the vehicle.

[1020] The development plan generated by the server is sent to the device (smartphone application). The user can check the development plan through the app and take appropriate care. The application also uses notification services such as Firebase to send reminders to the user and support them in taking appropriate care.

[1021] For example, if a user takes a picture of a Monstera plant in their car, the image is sent to a server where it is normalized and resized. An image recognition model then identifies the Monstera plant and extracts its optimal growing conditions from a database. This, combined with sensor data, generates a growing plan that, for example, notifies the user to open a window to ensure optimal temperature and humidity control.

[1022] Example prompt sentence:

[1023] Input plant images taken inside the car into a TensorFlow model to identify the plant type. Retrieve the identified plant's growing conditions from a database and combine them with sensor data to generate an optimal growing plan. Notify the driver of the generated growing plan using the Firebase notification service.

[1024] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1025] Step 1:

[1026] The user launches the smartphone application and takes a picture of a plant using the in-car camera. The application displays the image the user has taken and provides a posting button. The input is the image data captured by the camera, and the output is the image that the user can review and post.

[1027] Step 2:

[1028] The device sends images posted by users to the server, along with metadata such as the date and time of the photo, location information, and sensor data from inside the vehicle (temperature, humidity, CO2 concentration). The inputs are image data and metadata, and the output is this data sent to the server.

[1029] Step 3:

[1030] The server stores the received images and metadata. The server standardizes the image data and resizes it as necessary, improving the processing efficiency of the image recognition model. The inputs are the received images and metadata, and the standardized and resized image data is stored as the output.

[1031] Step 4:

[1032] The server uses an image recognition model to identify the plant species from the received image. It uses a machine learning framework such as TensorFlow to identify the plant species from the image. The input is the normalized and resized image data, and the output is the identified plant species.

[1033] Step 5:

[1034] The server extracts the growing conditions from a database based on the identified plant type. The database stores the growing conditions for various plants (watering frequency, sunlight conditions, fertilizer application timing, etc.). The input is the identified plant type, and the output is the extracted growing conditions.

[1035] Step 6:

[1036] The server combines the sensor data acquired with the plant's growing conditions to generate an optimal growing plan. For example, if the temperature and humidity inside the car do not meet certain conditions, it creates a growing plan that includes suggestions for providing an environment suitable for the plant (opening windows, adjusting temperature settings, etc.). The inputs are sensor data and growing conditions, and the output is an optimized growing plan.

[1037] Step 7:

[1038] The server sends the generated training plan to the device. The device receives this training plan and notifies the user. Services such as Firebase are used for notification. The input is the generated training plan, and the output is the training plan sent to the device.

[1039] Step 8:

[1040] The device receives the training plan and presents it to the user, informing them of specific care methods. The user can then check the detailed instructions through the app and provide appropriate care. The input is the received training plan, and the output is the training plan and care methods presented to the user.

[1041] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1042] This invention is a system that recognizes the user's emotional state by combining an emotion engine with the "MY Flower Bed" application, and provides care methods and encouraging messages based on that. This system consists of a server, a terminal, an image recognition model, an emotion engine, and a database.

[1043] Program processing flow and explanation

[1044] 1. User-submitted images

[1045] The user opens the application, takes a picture of a plant or bouquet using the device's camera, or selects an existing image from the photo gallery. The user then checks the image they have taken or selected within the application and presses the submit button to send the image to the server.

[1046] 2. Submitting images and metadata

[1047] The device acquires the image selected by the user along with the date and time of the photo, location information, and the user's emotional data, and sends them to the server.

[1048] 3. Image Reception and Preprocessing

[1049] The server receives the image, metadata, and emotion data sent from the device. First, it normalizes the image and resizes it if necessary.

[1050] 4. Image Recognition Processing

[1051] The server then uses an image recognition model to identify the plant species from the received image. This model has been trained on many plant images in advance, allowing it to identify plant species with high accuracy.

[1052] 5. Obtaining breeding conditions from a database

[1053] Based on the identified plant type, the server extracts from a database the optimal growing conditions for each plant, including watering frequency, light requirements, and the appropriate type of fertilizer.

[1054] 6. Integration of multiple plant conditions

[1055] If an image contains multiple plants, the server will integrate their respective growing conditions and generate a consistent growing plan. For example, if a Monstera and a rose are identified together, a growing plan will be created based on their respective characteristics.

[1056] 7. Emotion Data Analysis

[1057] The server uses an emotion engine to analyze the user's emotional data, identify the user's emotional state (e.g., joy, sadness, stress, etc.), and generate appropriate advice or encouraging messages based on this.

[1058] 8. Creation and integration of development plans

[1059] The server further customizes the planting plan by taking into account the user's emotional state, as analyzed by the emotion engine. For example, if the user is feeling stressed, the plan will include advice on plant care that has a relaxing effect.

[1060] 9. Development plan and message sending

[1061] The server generates a training plan and sends messages based on the emotions to the device, including specific care methods and encouraging messages.

[1062] 10. User Notice and Execution

[1063] The device displays the plant care plan and messages received on the app's user interface. Push notifications and reminder functions are used to prompt the user to care for their plants in a timely manner. The user then follows the plant care plan and emotion-based advice provided. Additional feedback can also be input into the app, allowing the emotion engine to continuously track changes in the user's emotional state.

[1064] Specific examples

[1065] For example, if a user posts a photo of a Monstera plant in their living room and a vase of roses to the app, the following process occurs:

[1066] 1. The user takes a picture using the application and presses the post button.

[1067] 2. The device sends the image, metadata, and user emotion data to the server.

[1068] 3. The server receives the image, normalizes it, and resizes it.

[1069] 4. The server uses an image recognition model to distinguish between a Monstera and a rose.

[1070] 5. The server retrieves the growing conditions for each plant from the database.

[1071] 6. The server integrates the growing conditions for Monstera and roses.

[1072] 7. The server analyzes the user's emotional data using an emotion engine and determines that the user is feeling stressed.

[1073] 8. The server creates a development plan by adding development advice that helps reduce stress.

[1074] 9. The server sends the training plan and encouraging messages to the device.

[1075] 10. The device notifies the user of the received development plan and message and displays it.

[1076] 11. The user will care for the plant in accordance with the growing plan and advice.

[1077] In this way, the system can provide optimal plant cultivation methods taking into account the user's emotional state, while also supporting the user's physical and mental care.

[1078] The processing flow will be explained below.

[1079] Program processing flow

[1080] Step 1:

[1081] The user opens the "MY Flower Bed" app and uses the device's camera to take a picture of a plant or bouquet, or select an existing image from the photo gallery.

[1082] Step 2:

[1083] The user presses the submit button to send the selected image to the server, optionally including emotional data (e.g., emotion estimation via an in-app survey or facial recognition).

[1084] Step 3:

[1085] The device acquires the image selected by the user along with metadata such as the date and time of shooting, location information, and emotional data, and makes a request to the server to send the data.

[1086] Step 4:

[1087] The server receives the images, metadata, and emotion data, normalizes the received images, and resizes them if necessary.

[1088] Step 5:

[1089] The server passes the standardized images to an image recognition algorithm, which uses a pre-trained convolutional neural network (CNN) to identify the type of plant in the image.

[1090] Step 6:

[1091] The server analyzes multiple segments within an image and individually identifies the plants contained within each segment, allowing for accurate species identification even when multiple plants are present in a photo.

[1092] Step 7:

[1093] The server extracts the optimal growing conditions (such as watering frequency, amount of sunlight, and type of fertilizer) from a database based on the identified plant type.

[1094] Step 8:

[1095] The server comprehensively assesses the growing conditions of multiple plants and creates a consistent growing plan for the user, which is adjusted to take into account the characteristics of each plant.

[1096] Step 9:

[1097] The server analyzes the user-provided emotion data using an emotion engine, which identifies the user's emotional state (e.g., joy, sadness, stress, etc.).

[1098] Step 10:

[1099] The server then customizes the plant care plan based on information gleaned from the emotional data, adding relaxing plant care advice if the user is feeling stressed, for example.

[1100] Step 11:

[1101] The server generates emotion-based messages (encouragement, positive feedback, etc.) and includes them in the development plan.

[1102] Step 12:

[1103] The server sends a customized development plan and a message based on the emotion to the terminal.

[1104] Step 13:

[1105] The device displays the received training plan and message on the user interface, and uses push notifications and reminder functions to encourage users to take timely care.

[1106] Step 14:

[1107] Users follow the advice based on the presented plant care plan and their emotions. If necessary, they can resubmit images and receive updated care plans. This allows users to not only manage the health of their plants, but also their own emotions at the same time.

[1108] Example 2

[1109] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1110] Conventional plant cultivation support systems simply provide plant cultivation conditions without considering the user's emotional state, and do not contribute to the user's mental care or stress reduction. Therefore, there is a need for a system that takes the user's emotional state into consideration and provides individually customized cultivation plans and encouraging messages to support the user's mental and physical health.

[1111] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1112] In this invention, the server includes means for users to post plant images, means for a terminal to transmit the posted plant images and additional information to the server, means for the server to store the received images and additional information, means for the server to identify the type of plant using an image recognition model for the received images, means for the server to extract cultivation conditions from a database based on the identified plant type, means for the server to integrate the extracted cultivation conditions and generate an optimal cultivation plan for the user, means for the server to transmit the generated cultivation plan to the terminal, means for the terminal to present the cultivation plan received by the terminal to the user, means for the server to analyze the user's emotional state using an emotion analysis device, and means for the server to customize the cultivation plan based on the analyzed emotional state and generate advice and encouraging messages associated with the emotions. This enables the provision of a plant cultivation plan that takes the user's emotional state into consideration and emotional care.

[1113] A "user" is someone who posts images of plants and receives advice from the system, including growing plans and emotionally relevant advice.

[1114] "Terminal" refers to the device used by the user to take images of plants, send the data to the server, and receive and display cultivation plans and advice.

[1115] "Server" refers to a central device that receives, stores, processes images and data sent from a terminal, and generates and sends development plans and advice to the terminal.

[1116] "Additional information" refers to additional metadata sent with an image, such as the date and time of the image capture, location information, or user emotional data.

[1117] "Image Recognition Model" refers to a model that uses machine learning algorithms to identify the type of plant contained in a received image.

[1118] "Database" refers to a system for storing and managing plant growing conditions and other related information.

[1119] "Growth conditions" refers to information including watering frequency, sunlight conditions, type of fertilizer, etc., as conditions for optimal growth of a particular plant.

[1120] "Growth plan" refers to a plan that integrates growing conditions extracted from the database and provides specific plant care methods to the user.

[1121] "Emotion analysis device" refers to hardware or software used to analyze a user's emotional state.

[1122] "Messages of advice and encouragement" refer to advice and support messages that are generated based on the user's emotional state and provided to the user.

[1123] This invention is a plant-growing support system that takes into account the user's emotional state. This system is composed of a server, a terminal, an image recognition model, an emotion analysis device, and a database, and provides users with individually customized plant-growing plans and advice.

[1124] Hardware and Software Configuration

[1125] Terminal

[1126] The terminal is a smartphone or tablet that allows users to take pictures of plants, send the data to the server, and receive and display cultivation plans and advice. The terminal is equipped with a camera, GPS module, and internet connection.

[1127] server

[1128] The server is a central device for storing and processing the received images and additional information. The server is equipped with an image recognition model, an emotion analysis device, and a database.

[1129] Image recognition model: Uses machine learning algorithms such as TensorFlow and PyTorch to identify the type of plant from the received image.

[1130] Emotion analysis device: A device that uses natural language processing (NLP) and facial expression recognition algorithms to analyze a user's emotional state.

[1131] Database: A system that stores and manages plant growth conditions and related information.

[1132] Data processing and calculation flow

[1133] 1. Image Reception and Preprocessing

[1134] The device sends the plant image and additional information (date and time of the photo, location information, and emotion data) to the server, which then receives the image and standardizes it by converting its format and resizing it.

[1135] 2. Image Recognition Processing

[1136] The server uses image recognition models to identify plant species, which identifies the names and genus species of plants contained in the image.

[1137] 3. Extraction of growth conditions

[1138] Based on the identified plant, the server extracts growing conditions (such as watering frequency, sunlight conditions, and type of fertilizer) from the database.

[1139] 5. Emotion Data Analysis

[1140] The server uses an emotion analyzer to identify the user's emotional state from their input data and facial expressions. For example, it may determine that the user is "stressed" based on their text input.

[1141] 6. Creating and customizing training plans

[1142] The server then combines the extracted training conditions with the emotional data to generate an optimal training plan for the user, adding customized advice and encouraging messages according to the user's emotional state.

[1143] 7. Development plan and message sending

[1144] The server sends the generated training plan and message to the terminal, which receives it and notifies and displays it to the user.

[1145] Specific examples

[1146] For example, if a user posts a photo of a Monstera plant in their living room and a vase of roses, the following happens:

[1147] 1. The user takes a photo of a plant using the in-app camera, then presses the "Post" button to send the image.

[1148] 2. The device sends the captured image and additional information (date and time of capture, location information, and emotion data) to the server.

[1149] 3. The server receives the image, normalizes it, and resizes it.

[1150] 4. The server uses image recognition models to identify the Monstera and the rose, and their plant names are identified.

[1151] 5. The server retrieves the growing conditions for the identified plants from the database. For example, a Monstera should be watered every two weeks and indirect sunlight, while a rose should be watered once a week and in direct sunlight.

[1152] 6. The server uses an emotion analyzer to analyze whether the user is feeling "stressed."

[1153] 7. The server generates a care plan by adding advice to help reduce stress. For example, it adds an encouraging message such as, "Watering this Monstera every morning will help you start your day relaxing."

[1154] 8. The server sends the final training plan and encouraging messages to the device.

[1155] 9. The device displays the training plan and message received within the app and notifies the user.

[1156] Prompt Sentence Examples

[1157] Use a prompt like this for the emotion engine:

[1158] "The user's current emotional state is stressed. Generate advice suggesting how to care for plants to reduce stress."

[1159] As described above, the present invention is a system that contributes to the care of the user's mind and body by providing a plant cultivation plan that takes into account the user's emotional state.

[1160] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1161] Step 1:

[1162] The user opens the application and takes a picture of a plant or bouquet using the device's camera, or selects an existing image from the photo gallery. When the user presses the "Post" button, the selected or taken image is temporarily saved on the device. The input is the taken or selected image, and the output is the temporarily saved image.

[1163] Step 2:

[1164] Based on the user's operation, the device sends the temporarily saved image and additional information (date and time of shooting, location information, emotional data) to the server. Emotional data is extracted from direct input from the user or facial expressions captured by the device's camera. The device then sends the acquired data to the server. The input is the image and additional information, and the output is the data sent to the server.

[1165] Step 3:

[1166] The server receives the image and additional information sent from the device. It then standardizes the image format and resizes it as necessary. Image standardization includes unifying the image format and adjusting the resolution. The input is the data received from the device, and the output is the standardized image and metadata.

[1167] Step 4:

[1168] The server uses an image recognition model on the standardized images to identify the plant type. This process is performed using machine learning models such as TensorFlow and PyTorch. The model identifies plant types with high accuracy based on the training dataset. The input is the standardized image, and the output is the plant type identification result.

[1169] Step 5:

[1170] Based on the identified plant type, the server extracts the growing conditions from the database. Growing conditions include watering frequency, sunlight requirements, type of fertilizer, etc. This data extraction is performed using SQL queries. The input is the plant identification results, and the output is the growing conditions for each plant.

[1171] Step 6:

[1172] When the server identifies multiple plants in an image, it integrates their respective growing conditions to generate a consistent growing plan. It compares the multiple growing conditions and adjusts and integrates them to find common and conflicting parts. The input is multiple growing conditions, and the output is an integrated growing plan.

[1173] Step 7:

[1174] The server uses an emotion analyzer to analyze the transmitted emotional data. It uses natural language processing and facial expression recognition algorithms to identify the user's emotional state (e.g., stress, joy, sadness). The input is the emotional data, and the output is the analyzed emotional state.

[1175] Step 8:

[1176] The server further customizes the integrated training plan based on the analyzed emotional state. For example, if the user is feeling stressed, it adds relaxing training advice or encouraging messages. The input is the training plan and the user's emotional state, and the output is a customized training plan.

[1177] Step 9:

[1178] The server transmits the generated customized development plan and encouraging message to the terminal. The input is the customized development plan and message, and the output is the data transmitted to the terminal.

[1179] Step 10:

[1180] The device notifies the user of the plant care plan and messages received and displays them on the app's user interface. Push notifications and reminder functions are used to encourage timely care. Users care for their plants according to the provided plant care plan and advice, and provide additional feedback to the app, allowing the emotion engine to track changes in the user's emotional state. The input is data received from the server, and the output is notification and display to the user, and feedback collection.

[1181] Through these steps, the system takes into account the user's emotional state, provides an optimal plant cultivation plan, and supports the user's physical and mental health.

[1182] (Application example 2)

[1183] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1184] Conventional plant-growing applications were able to identify plant species from images posted by users and provide information on growing conditions. However, these applications were unable to customize growing plans that took into account the user's emotional state, making it difficult to improve user satisfaction and care for their mind and body. Furthermore, when selecting plants in physical stores, there was a lack of support for users to choose plants that match the plant's characteristics and their own mental and physical state. This has created a need for applications that can provide both appropriate growing plans and care for the user's mind and body.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1186] In this invention, the server includes a means for generating a customized plant growing plan based on the user's emotional state, a means for transmitting the generated plant growing plan and a message based on the user's emotion to the terminal, and a means for presenting the plant growing plan and message received by the terminal to the user. This makes it possible to provide a plant growing plan that takes the user's emotional state into consideration, and realizes a system that supports users in selecting appropriate plants in a physical store while also providing physical and mental care.

[1187] "User" refers to an individual who posts an image of a plant and receives a growing plan.

[1188] "Terminal" refers to an electronic device used by a User, including a smartphone, tablet, or computer.

[1189] "Image" means a visual representation of a plant photographed or selected by a User using a Device.

[1190] "Metadata" refers to additional information that accompanies an image, including the date and time the image was taken, location information, and the user's emotional data.

[1191] "Server" refers to a remote computer that receives, stores, and analyzes data sent from the terminal and generates training plans and messages.

[1192] "Image recognition model" refers to a program that implements a machine learning algorithm used to analyze images and identify plant species.

[1193] "Plant type" refers to the classification of a particular plant as identified by the image recognition model.

[1194] "Growth conditions" refers to information that indicates the optimal cultivation method based on the type of plant, and refers to data including watering frequency, sunlight conditions, and appropriate type of fertilizer.

[1195] A "growing plan" refers to a plan that integrates the growing conditions of the identified plants and indicates specific growing methods customized based on the user's emotional state.

[1196] "Emotional State" refers to the user's current mental state as analyzed using the user's emotion engine.

[1197] "Emotion-based messages" refer to words of encouragement or advice that are generated based on the user's emotional state.

[1198] A system for implementing the present invention includes an application for supporting plant growth and a configuration for providing customized advice based on emotional states. Specific embodiments of this system will be described below.

[1199] Hardware and software used

[1200] Device: A smartphone, tablet, or computer device through which users can post plant images.

[1201] Server: A remote computer that receives, stores, and analyzes data sent from the device.

[1202] Image recognition models: Use algorithms built using machine learning frameworks such as TensorFlow or PyTorch.

[1203] Sentiment engine: Uses natural language processing services such as IBM Watson and Google Cloud Natural Language API.

[1204] Database: Use a relational database such as PostgreSQL or MySQL.

[1205] Front-end: Building the application's user interface using React Native or Flutter.

[1206] Specific details of data processing and calculation

[1207] 1. User-submitted images

[1208] The device receives input from the user and sends the plant image and metadata (date and time of capture, location, and emotional data) to the server, where the user is prompted to enter their real-time emotional state.

[1209] 2. Data processing by the server

[1210] The server standardizes and resizes the received image data, identifies the plant type using an image recognition model, extracts growing conditions from a database based on the identified plant type, and then integrates the extracted growing conditions to generate a customized growing plan and message based on the user's emotional state.

[1211] 3. Notice to Users

[1212] The server sends the generated plant care plan and emotion-based messages to the device, which then presents them to the user, allowing the user to learn the optimal plant care method based on their emotional state.

[1213] Specific examples

[1214] For example, consider a case where a user takes a photo of a plant and then inputs their emotional state according to the following prompt:

[1215] Example prompt sentence:

[1216] "Please tell us about your recent stressful situations. Please talk about your feelings and worries."

[1217] If the user types "I'm tired," the server processing will generate a message like this:

[1218] Sample message:

[1219] "The plant chosen this time is the Monstera, which helps relieve stress. Make sure to water it regularly and provide it with adequate sunlight. When you're busy, just looking at the greenery of this plant can have a relaxing effect."

[1220] This system allows users to take their emotional state into consideration and implement optimal plant care methods, as well as receive advice that can help relieve daily stress.

[1221] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1222] Step 1:

[1223] User-submitted images

[1224] The user takes a photo of the plant using their device or selects an existing image, then reviews the image within the application and presses the submit button to send the image to the server.

[1225] Input: A user-taken or selected image of a plant.

[1226] Output: Image data is stored on the device and a request is generated to be sent to the server.

[1227] Step 2:

[1228] Sending images and metadata

[1229] The device sends the image selected by the user, along with the date and time of the photo, location information, and emotion data collected based on the prompt text, to the server.

[1230] Input: Plant image, photo date and time, location information, and user emotion data.

[1231] Output: The request is processed and the submitted data is saved on the server.

[1232] Step 3:

[1233] Image reception and preprocessing

[1234] The server receives the image and metadata sent from the device, normalizes the image data, and resizes it if necessary, so that the image is in a format suitable for the image recognition model.

[1235] Input: Received plant images and metadata.

[1236] Output: Normalized and resized image data.

[1237] Step 4:

[1238] Image Recognition Processing

[1239] The server uses an image recognition model to identify the type of plant from the received image. This model has been trained on many plant images in advance, allowing it to identify plant species with high accuracy.

[1240] Input: Standardized plant image data.

[1241] Output: The plant species identified.

[1242] Step 5:

[1243] Obtaining breeding conditions from a database

[1244] Based on the identified plant type, the server extracts from a database the optimal growing conditions for each plant, including watering frequency, light requirements, and the appropriate type of fertilizer.

[1245] Input: Identified plant species.

[1246] Output: Extracted growing condition data.

[1247] Step 6:

[1248] Integration of multiple plant conditions

[1249] If the image contains multiple plants, the server integrates the growing conditions of each plant and generates a consistent growing plan.

[1250] Input: Growing conditions for multiple plants.

[1251] Output: Integrated development plan.

[1252] Step 7:

[1253] Emotional Data Analysis

[1254] The server uses an emotion engine to analyze the user's emotional data, identify the user's emotional state (e.g., joy, sadness, stress, etc.), and generate appropriate advice or encouraging messages based on the data.

[1255] Input: User emotion data.

[1256] Output: User's emotional state data and an encouraging message.

[1257] Step 8:

[1258] Creation and integration of development plans

[1259] The server further customizes the planting plan by taking into account the user's emotional state, as analyzed by the emotion engine. For example, if the user is feeling stressed, the plan will include advice on plant care that has a relaxing effect.

[1260] Input: User emotional state data.

[1261] Output: A customized development plan.

[1262] Step 9:

[1263] Development plan and message sending

[1264] The server sends a message based on the generated development plan and emotion to the terminal.

[1265] Input: A customized development plan and encouraging message.

[1266] Output: The development plan and message sent.

[1267] Step 10:

[1268] User notification and action

[1269] The device displays the plant care plan and messages received on the app's user interface. Push notifications and reminder functions are used to prompt the user to provide timely care. The user can then care for their plants according to the provided care plan and emotional advice.

[1270] Input: Received development plans and messages.

[1271] Output: The development plan and messages displayed in the user interface.

[1272] In this way, each processing step is designed to perform appropriate calculations on data inputs and obtain customized outputs based on the user's emotional state.

[1273] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1274] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1275] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1276] [Fourth embodiment]

[1277] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1278] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1279] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1280] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1281] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1282] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1283] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1284] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1285] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1286] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1287] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1288] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1289] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1290] Users use the "My Flowerbed" application to manage the plants they grow at home or in the office. The system utilizes a server, user devices, and image recognition models as part of the platform.

[1291] Program processing flow and explanation

[1292] 1. User-submitted images

[1293] Users open the app and use their device's camera to take a picture of a plant or bouquet. In some cases, they can also select an existing image from their device's gallery. The image is displayed in the app, and users can then click the "Submit" button to send it to the server.

[1294] 2. Submitting images and metadata

[1295] The device collects the images selected by the user along with metadata such as the date and time of the photo and location, and transmits this data to a server over a secure protocol.

[1296] 3. Image Reception and Preprocessing

[1297] The server receives the images and metadata sent from the device, standardizes the image format, and resizes them if necessary, improving the processing efficiency of the image recognition model.

[1298] 4. Image Recognition Processing

[1299] The server then uses an image recognition model to identify the plant species from the received image. This model has been trained on many plant images in advance, allowing it to identify plant species with high accuracy.

[1300] 5. Obtaining breeding conditions from a database

[1301] Based on the identified plant type, the server extracts from a database the optimal growing conditions for each plant, including watering frequency, light requirements, and the appropriate type of fertilizer.

[1302] 6. Integration of multiple plant conditions

[1303] If an image contains multiple plants, the server integrates their respective growing conditions and generates a consistent growing plan. For example, if a Monstera and a rose are identified together, the server creates a plan based on their respective characteristics.

[1304] 7. Generate and submit a training plan

[1305] The generated cultivation plan is sent from the server to the device, and includes specific advice such as how often to water the plant, the amount of sunlight, and when to use fertilizer.

[1306] 8. User Notification and Execution

[1307] The device will notify the user of the received care plan, and the user can check detailed care instructions in the app and follow the instructions to care for their plants. The app also provides regular reminders to help users continue to care for their plants properly.

[1308] Specific examples

[1309] For example, if a user posts a photo of a Monstera plant in their living room and a vase of roses to the app, the following process occurs:

[1310] 1. The user takes a photo using the app and presses the post button.

[1311] 2. The device sends the image along with its metadata to the server.

[1312] 3. The server receives the image, normalizes it, and resizes it.

[1313] 4. The server uses an image recognition model to distinguish between a Monstera and a rose.

[1314] 5. The server retrieves the optimal growing conditions for each plant from the database.

[1315] 6. The server integrates the conditions for the Monstera and roses and generates a consistent cultivation plan.

[1316] 7. The server sends the development plan to the device.

[1317] 8. The device will notify the user of the development plan and provide specific care methods.

[1318] In this way, the system helps users easily care for multiple plants and provides optimal methods to ensure their longevity.

[1319] The processing flow will be explained below.

[1320] Program processing flow

[1321] Step 1:

[1322] Users open the "MY Flower Bed" app and use their device's camera to take a picture of the plants or bouquet they are growing, or select an existing image from their photo gallery.

[1323] Step 2:

[1324] The user presses the submit button to send the selected image to the server, optionally adding comments and metadata about the plant's condition.

[1325] Step 3:

[1326] The device acquires the image selected by the user and the associated metadata (e.g., shooting date and time, location information), and makes a request to the server to send the data.

[1327] Step 4:

[1328] The server receives the images and metadata sent, and first converts the file format and adjusts the image size to standardize the received images.

[1329] Step 5:

[1330] The server passes the standardized images to an image recognition algorithm, which uses a pre-trained convolutional neural network (CNN) or similar to identify the type of plant in the image.

[1331] Step 6:

[1332] The server analyzes multiple segments within an image and individually identifies the plants contained within each segment, allowing for accurate species identification even when multiple plants are present in a photo.

[1333] Step 7:

[1334] The server extracts the optimal growing conditions (such as watering frequency, amount of sunlight, and type of fertilizer) from a database based on the identified plant type.

[1335] Step 8:

[1336] The server comprehensively assesses the growing conditions of multiple plants and creates a consistent growing plan for the user, which is adjusted to take into account the characteristics of each plant.

[1337] Step 9:

[1338] The server formats the plan into specific instructions and sends them to the device, including instructions on when to water the plant, how much sunlight it needs, and how to properly fertilize it.

[1339] Step 10:

[1340] The device receives the training plan from the server and displays it on the app's user interface. Push notifications and alarms can also be used to set reminders to encourage timely care.

[1341] Step 11:

[1342] Users can care for their plants according to the provided care plan, and can also post images again to receive updated care plans as the situation evolves.

[1343] Example 1

[1344] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1345] Conventional plant growth management systems make it difficult for users to integrate individual conditions and generate consistent growth plans when managing multiple plants. They also have limited functionality for collecting detailed metadata along with plant images and providing regular reminders to users. This makes it difficult for users to effectively care for their plants, and it takes a lot of time and effort.

[1346] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1347] In this invention, the server includes means for standardizing and resizing the image format, means for integrating the growing conditions of the identified multiple plants to generate a consistent growing plan, and means for providing regular reminders to support the user in continuing appropriate care. This makes it easier for the user to manage multiple plants, enables the generation of a growing plan that integrates the optimal growing conditions for each plant, and enables the user to care for the plants efficiently and effectively.

[1348] "User" refers to a person who uses the system or application to manage and cultivate plants.

[1349] "Device" refers to the device a user uses to capture images of plants and collect and transmit metadata, including smartphones and tablets.

[1350] "Server" refers to a computer system that receives, stores, and processes images and metadata submitted by users.

[1351] "Image recognition model" refers to the machine learning or deep learning algorithm used to identify plant species from received images.

[1352] "Metadata" is data that includes information related to an image, specifically including the date and time of the image capture, location information, camera settings, and the like.

[1353] "Growing conditions" refers to the environmental factors that a particular plant needs to grow optimally, including watering frequency, light requirements, and type of fertilizer.

[1354] A "growth plan" refers to specific care methods created based on the optimal growing conditions for each type of plant.

[1355] "Periodic reminders" refers to a feature that sends regular notifications to users to encourage them to continue proper development.

[1356] "Normalization" refers to the process of converting received images into a consistent format so that subsequent processing can be done efficiently.

[1357] "Resizing" refers to the process of changing the size of an image, which improves the efficiency of image recognition models.

[1358] The present invention provides a system that allows users to easily and effectively manage and cultivate plants, and is implemented using a server, a terminal, and an image recognition model, allowing users to efficiently obtain information necessary for plant care.

[1359] First, a user installs an application for managing plants. When a user wants to post an image of a plant, they take or select an image using the device's camera or gallery. The device then collects the image along with metadata such as the date and time of the photo and its location, and sends it to a server via a secure protocol (e.g., HTTPS).

[1360] The server first stores the received images and metadata. Next, it standardizes the image format and resizes it as necessary. This allows for efficient processing by an image recognition model (e.g., a deep learning model using TensorFlow or PyTorch). The server then uses the image recognition model to identify the plant species from the received images. This model has been trained in advance on many plant images and can identify plant species with high accuracy.

[1361] After identifying the type of plant, the server extracts the optimal growing conditions for each plant from a database (e.g., a relational database such as MySQL). This includes watering frequency, sunlight requirements, and the appropriate type of fertilizer. If multiple plants are identified, their respective growing conditions are integrated to generate a consistent growing plan. For example, if a Monstera and a rose are identified together, a growing plan tailored to the characteristics of each will be created. The generated growing plan is sent from the server to the device, and the device notifies the user. The user can check detailed growing instructions through the app and practice appropriate care. The device also provides regular reminders to help the user continue to provide appropriate care.

[1362] Specific examples

[1363] For example, if a user posts a photo of a monstera plant in their living room and a vase of roses, the following process occurs:

[1364] 1. The user takes a photo using the app and presses the post button.

[1365] 2. The device sends the image along with its metadata to the server.

[1366] 3. The server receives the image, normalizes it, and resizes it.

[1367] 4. The server uses an image recognition model to distinguish between a Monstera and a rose.

[1368] 5. The server retrieves the optimal growing conditions for each plant from the database.

[1369] 6. The server integrates the conditions for the Monstera and roses and generates a consistent cultivation plan.

[1370] 7. The server sends the development plan to the device.

[1371] 8. The device will notify the user of the development plan and provide specific care methods.

[1372] In this way, the system helps users easily care for multiple plants and provides optimal methods to ensure their longevity.

[1373] Prompt Sentence Examples

[1374] "I would like to know how to care for the Monstera and roses I am growing at home. Please tell me the care plan for growing Monstera and roses at the same time."

[1375] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1376] Step 1: User submits image

[1377] 1. The user launches the app and the main screen displays the camera and gallery icons.

[1378] 2. The user taps the camera icon to launch the device's camera app and take a picture of the plant or bouquet.

[1379] 3. Input: An image of a plant taken with the device's camera.

[1380] 4. The user presses the shutter button to take a picture, which is then displayed on the preview screen.

[1381] 5. The user presses the "Post" button on the preview screen and the image is saved within the app.

[1382] 6. Output: Image data captured by the user.

[1383] Step 2: Submit your images and metadata

[1384] 1. The device obtains metadata, such as the date and time of the image taken and its location, for the image selected or taken by the user.

[1385] 2. Input: Images taken by the user, along with metadata such as the date and time of the image and its location.

[1386] 3. The device sends the image and metadata to the server over a secure protocol such as HTTPS.

[1387] 4. Output: Image data and metadata sent to the server.

[1388] Step 3: Receiving and preprocessing images

[1389] 1. The server receives the image and metadata sent from the device.

[1390] 2. Input: Image data and metadata sent from the device.

[1391] 3. The server standardizes the image format and resizes it as needed, for example converting the image width and height to a consistent number of pixels.

[1392] 4. Output: Resized and standardized image data.

[1393] Step 4: Image Recognition Processing

[1394] 1. The server inputs the preprocessed image into an image recognition model that has been trained on many plant images in advance.

[1395] 2. Input: Preprocessed image data.

[1396] 3. The server uses an image recognition model to identify the plant type.

[1397] 4. Output: Identified plant species data (e.g. Monstera, Rose).

[1398] Step 5: Obtaining breeding conditions from the database

[1399] 1. The server searches the database for growing conditions based on the identified plant type.

[1400] 2. Input: Identified plant species data.

[1401] 3. The server obtains growing conditions, including watering frequency, sunlight requirements, and appropriate type of fertilizer.

[1402] 4. Output: Growing condition data for each plant.

[1403] Step 6: Integrating multiple plant conditions

[1404] 1. When multiple plants are identified, the server integrates their respective growing conditions.

[1405] 2. Input: Individual growing condition data for multiple plants.

[1406] 3. The server analyzes the similarities and differences between each breeding condition and generates a consistent breeding plan.

[1407] 4. Output: Integrated development plan.

[1408] Step 7: Generate and submit your development plan

[1409] 1. The server generates specific advice based on an integrated growing plan, including watering frequency, amount of sunlight, and timing of fertilizer application.

[1410] 2. Input: Integrated Development Plan.

[1411] 3. The server sends this development plan to the terminal.

[1412] 4. Output: Development plan data sent to the terminal.

[1413] Step 8: Notify users and take action

[1414] 1. The terminal notifies the user of the development plan received from the server.

[1415] 2. Input: Development plan data from the server.

[1416] 3. The user views the notification and checks detailed training instructions within the app.

[1417] 4. The user follows the plant care instructions, such as watering a Monstera twice a week or placing a rose in a sunny spot.

[1418] 5. The device will provide regular reminders to help users continue to take proper care.

[1419] (Application example 1)

[1420] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1421] Conventional plant growth management systems are primarily used to manage plants in indoor environments, making it difficult to manage plants that adapt to the unique environmental conditions inside autonomous vehicles. Furthermore, there is a lack of technology that provides growth plans that take into account environmental factors such as temperature, humidity, and CO2 concentration inside the vehicle. This makes it difficult for users who want to grow plants inside their vehicles to provide an optimal growing environment, making it difficult to maintain the health of their plants.

[1422] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1423] In this invention, the server includes means for users to post plant images, means for a terminal to transmit the posted plant images and metadata to the server, means for the server to store the received images and metadata, means for the server to identify the type of plant using an image recognition model for the received images, means for the server to extract cultivation conditions from a database based on the identified plant type, means for the server to integrate the extracted cultivation conditions and generate an optimal cultivation plan for the user, means for transmitting the generated cultivation plan to the terminal, means for the terminal to present the received cultivation plan to the user, means for acquiring vehicle sensor data, and means for combining the acquired sensor data with plant cultivation conditions to generate an optimal cultivation plan, thereby enabling optimal plant cultivation management tailored to the in-vehicle environment of an autonomous vehicle.

[1424] "User" refers to an entity that uses this system to manage plants.

[1425] A "terminal" is a device used by a user that is equipped with means for taking images of plants and transmitting data.

[1426] The "server" is a device that receives, stores, and processes images and data sent from the terminal, and runs an image recognition model to identify plants and generate cultivation plans.

[1427] "Metadata" is additional information related to the main data, such as the date and time an image was taken and location information.

[1428] An "image recognition model" is a software model that runs on a server and is a technology used to identify plant species from submitted images.

[1429] A "database" is a collection of data that stores information about plant growing conditions.

[1430] A "growth plan" is a systematic compilation of the various conditions necessary for healthy plant growth (e.g., frequency of watering, sunlight conditions, timing of fertilizer application, etc.).

[1431] "Sensor data" refers to data obtained from sensors used to detect environmental information inside a vehicle, such as temperature, humidity, and CO2 concentration.

[1432] The "vehicle" is a means of transportation that utilizes autonomous driving technology and is equipped with environmental sensors to support plant growth.

[1433] "Standardization" is the process of converting incoming image data into a consistent format.

[1434] "Resizing" is the process of changing the size of image data to an appropriate size.

[1435] In the embodiment of the present invention, a system for managing plant growth inside an autonomous vehicle will be specifically described.

[1436] Users use a smartphone application called "CAR BOTANICA" to manage the plants installed in their vehicle. The user launches the application and takes a picture of the plant using the vehicle's camera. The image is then sent to a server via the device.

[1437] The server stores the received images and metadata (date and time of capture, location, sensor data), then uses an image recognition model to identify the plant species from the submitted image, using a machine learning framework such as TensorFlow.

[1438] Based on the identified plant type, the server extracts the growing conditions from a database. The database stores the growing conditions for various plants (watering frequency, sunlight conditions, fertilizer application timing, etc.). Furthermore, by combining this with sensor data (temperature, humidity, CO2 concentration), the server generates an optimal growing plan. This provides a growing plan that adapts to the unique environmental conditions inside the vehicle.

[1439] The development plan generated by the server is sent to the device (smartphone application). The user can check the development plan through the app and take appropriate care. The application also uses notification services such as Firebase to send reminders to the user and support them in taking appropriate care.

[1440] For example, if a user takes a picture of a Monstera plant in their car, the image is sent to a server where it is normalized and resized. An image recognition model then identifies the Monstera plant and extracts its optimal growing conditions from a database. This, combined with sensor data, generates a growing plan that, for example, notifies the user to open a window to ensure optimal temperature and humidity control.

[1441] Example prompt sentence:

[1442] Input plant images taken inside the car into a TensorFlow model to identify the plant type. Retrieve the identified plant's growing conditions from a database and combine them with sensor data to generate an optimal growing plan. Notify the driver of the generated growing plan using the Firebase notification service.

[1443] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1444] Step 1:

[1445] The user launches the smartphone application and takes a picture of a plant using the in-car camera. The application displays the image the user has taken and provides a posting button. The input is the image data captured by the camera, and the output is the image that the user can review and post.

[1446] Step 2:

[1447] The device sends images posted by users to the server, along with metadata such as the date and time of the photo, location information, and sensor data from inside the vehicle (temperature, humidity, CO2 concentration). The inputs are image data and metadata, and the output is this data sent to the server.

[1448] Step 3:

[1449] The server stores the received images and metadata. The server standardizes the image data and resizes it as necessary, improving the processing efficiency of the image recognition model. The inputs are the received images and metadata, and the standardized and resized image data is stored as the output.

[1450] Step 4:

[1451] The server uses an image recognition model to identify the plant species from the received image. It uses a machine learning framework such as TensorFlow to identify the plant species from the image. The input is the normalized and resized image data, and the output is the identified plant species.

[1452] Step 5:

[1453] The server extracts the growing conditions from a database based on the identified plant type. The database stores the growing conditions for various plants (watering frequency, sunlight conditions, fertilizer application timing, etc.). The input is the identified plant type, and the output is the extracted growing conditions.

[1454] Step 6:

[1455] The server combines the sensor data acquired with the plant's growing conditions to generate an optimal growing plan. For example, if the temperature and humidity inside the car do not meet certain conditions, it creates a growing plan that includes suggestions for providing an environment suitable for the plant (opening windows, adjusting temperature settings, etc.). The inputs are sensor data and growing conditions, and the output is an optimized growing plan.

[1456] Step 7:

[1457] The server sends the generated training plan to the device. The device receives this training plan and notifies the user. Services such as Firebase are used for notification. The input is the generated training plan, and the output is the training plan sent to the device.

[1458] Step 8:

[1459] The device receives the training plan and presents it to the user, informing them of specific care methods. The user can then check the detailed instructions through the app and provide appropriate care. The input is the received training plan, and the output is the training plan and care methods presented to the user.

[1460] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1461] This invention is a system that recognizes the user's emotional state by combining an emotion engine with the "MY Flower Bed" application, and provides care methods and encouraging messages based on that. This system consists of a server, a terminal, an image recognition model, an emotion engine, and a database.

[1462] Program processing flow and explanation

[1463] 1. User-submitted images

[1464] The user opens the application, takes a picture of a plant or bouquet using the device's camera, or selects an existing image from the photo gallery. The user then checks the image they have taken or selected within the application and presses the submit button to send the image to the server.

[1465] 2. Submitting images and metadata

[1466] The device acquires the image selected by the user along with the date and time of the photo, location information, and the user's emotional data, and sends them to the server.

[1467] 3. Image Reception and Preprocessing

[1468] The server receives the image, metadata, and emotion data sent from the device. First, it normalizes the image and resizes it if necessary.

[1469] 4. Image Recognition Processing

[1470] The server then uses an image recognition model to identify the plant species from the received image. This model has been trained on many plant images in advance, allowing it to identify plant species with high accuracy.

[1471] 5. Obtaining breeding conditions from a database

[1472] Based on the identified plant type, the server extracts from a database the optimal growing conditions for each plant, including watering frequency, light requirements, and the appropriate type of fertilizer.

[1473] 6. Integration of multiple plant conditions

[1474] If an image contains multiple plants, the server will integrate their respective growing conditions and generate a consistent growing plan. For example, if a Monstera and a rose are identified together, a growing plan will be created based on their respective characteristics.

[1475] 7. Emotion Data Analysis

[1476] The server uses an emotion engine to analyze the user's emotional data, identify the user's emotional state (e.g., joy, sadness, stress, etc.), and generate appropriate advice or encouraging messages based on this.

[1477] 8. Creation and integration of development plans

[1478] The server further customizes the planting plan by taking into account the user's emotional state, as analyzed by the emotion engine. For example, if the user is feeling stressed, the plan will include advice on plant care that has a relaxing effect.

[1479] 9. Development plan and message sending

[1480] The server generates a training plan and sends messages based on the emotions to the device, including specific care methods and encouraging messages.

[1481] 10. User Notice and Execution

[1482] The device displays the plant care plan and messages received on the app's user interface. Push notifications and reminder functions are used to prompt the user to care for their plants in a timely manner. The user then follows the plant care plan and emotion-based advice provided. Additional feedback can also be input into the app, allowing the emotion engine to continuously track changes in the user's emotional state.

[1483] Specific examples

[1484] For example, if a user posts a photo of a Monstera plant in their living room and a vase of roses to the app, the following process occurs:

[1485] 1. The user takes a picture using the application and presses the post button.

[1486] 2. The device sends the image, metadata, and user emotion data to the server.

[1487] 3. The server receives the image, normalizes it, and resizes it.

[1488] 4. The server uses an image recognition model to distinguish between a Monstera and a rose.

[1489] 5. The server retrieves the growing conditions for each plant from the database.

[1490] 6. The server integrates the growing conditions for Monstera and roses.

[1491] 7. The server analyzes the user's emotional data using an emotion engine and determines that the user is feeling stressed.

[1492] 8. The server creates a development plan by adding development advice that helps reduce stress.

[1493] 9. The server sends the training plan and encouraging messages to the device.

[1494] 10. The device notifies the user of the received development plan and message and displays it.

[1495] 11. The user will care for the plant in accordance with the growing plan and advice.

[1496] In this way, the system can provide optimal plant cultivation methods taking into account the user's emotional state, while also supporting the user's physical and mental care.

[1497] The processing flow will be explained below.

[1498] Program processing flow

[1499] Step 1:

[1500] The user opens the "MY Flower Bed" app and uses the device's camera to take a picture of a plant or bouquet, or select an existing image from the photo gallery.

[1501] Step 2:

[1502] The user presses the submit button to send the selected image to the server, optionally including emotional data (e.g., emotion estimation via an in-app survey or facial recognition).

[1503] Step 3:

[1504] The device acquires the image selected by the user along with metadata such as the date and time of shooting, location information, and emotional data, and makes a request to the server to send the data.

[1505] Step 4:

[1506] The server receives the images, metadata, and emotion data, normalizes the received images, and resizes them if necessary.

[1507] Step 5:

[1508] The server passes the standardized images to an image recognition algorithm, which uses a pre-trained convolutional neural network (CNN) to identify the type of plant in the image.

[1509] Step 6:

[1510] The server analyzes multiple segments within an image and individually identifies the plants contained within each segment, allowing for accurate species identification even when multiple plants are present in a photo.

[1511] Step 7:

[1512] The server extracts the optimal growing conditions (such as watering frequency, amount of sunlight, and type of fertilizer) from a database based on the identified plant type.

[1513] Step 8:

[1514] The server comprehensively assesses the growing conditions of multiple plants and creates a consistent growing plan for the user, which is adjusted to take into account the characteristics of each plant.

[1515] Step 9:

[1516] The server analyzes the user-provided emotion data using an emotion engine, which identifies the user's emotional state (e.g., joy, sadness, stress, etc.).

[1517] Step 10:

[1518] The server then customizes the plant care plan based on information gleaned from the emotional data, adding relaxing plant care advice if the user is feeling stressed, for example.

[1519] Step 11:

[1520] The server generates emotion-based messages (encouragement, positive feedback, etc.) and includes them in the development plan.

[1521] Step 12:

[1522] The server sends a customized development plan and a message based on the emotion to the terminal.

[1523] Step 13:

[1524] The device displays the received training plan and message on the user interface, and uses push notifications and reminder functions to encourage users to take timely care.

[1525] Step 14:

[1526] Users follow the advice based on the presented plant care plan and their emotions. If necessary, they can resubmit images and receive updated care plans. This allows users to not only manage the health of their plants, but also their own emotions at the same time.

[1527] Example 2

[1528] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1529] Conventional plant cultivation support systems simply provide plant cultivation conditions without considering the user's emotional state, and do not contribute to the user's mental care or stress reduction. Therefore, there is a need for a system that takes the user's emotional state into consideration and provides individually customized cultivation plans and encouraging messages to support the user's mental and physical health.

[1530] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1531] In this invention, the server includes means for users to post plant images, means for a terminal to transmit the posted plant images and additional information to the server, means for the server to store the received images and additional information, means for the server to identify the type of plant using an image recognition model for the received images, means for the server to extract cultivation conditions from a database based on the identified plant type, means for the server to integrate the extracted cultivation conditions and generate an optimal cultivation plan for the user, means for the server to transmit the generated cultivation plan to the terminal, means for the terminal to present the cultivation plan received by the terminal to the user, means for the server to analyze the user's emotional state using an emotion analysis device, and means for the server to customize the cultivation plan based on the analyzed emotional state and generate advice and encouraging messages associated with the emotions. This enables the provision of a plant cultivation plan that takes the user's emotional state into consideration and emotional care.

[1532] A "user" is someone who posts images of plants and receives advice from the system, including growing plans and emotionally relevant advice.

[1533] "Terminal" refers to the device used by the user to take images of plants, send the data to the server, and receive and display cultivation plans and advice.

[1534] "Server" refers to a central device that receives, stores, processes images and data sent from a terminal, and generates and sends development plans and advice to the terminal.

[1535] "Additional information" refers to additional metadata sent with an image, such as the date and time of the image capture, location information, or user emotional data.

[1536] "Image Recognition Model" refers to a model that uses machine learning algorithms to identify the type of plant contained in a received image.

[1537] "Database" refers to a system for storing and managing plant growing conditions and other related information.

[1538] "Growth conditions" refers to information including watering frequency, sunlight conditions, type of fertilizer, etc., as conditions for optimal growth of a particular plant.

[1539] "Growth plan" refers to a plan that integrates growing conditions extracted from the database and provides specific plant care methods to the user.

[1540] "Emotion analysis device" refers to hardware or software used to analyze a user's emotional state.

[1541] "Messages of advice and encouragement" refer to advice and support messages that are generated based on the user's emotional state and provided to the user.

[1542] This invention is a plant-growing support system that takes into account the user's emotional state. This system is composed of a server, a terminal, an image recognition model, an emotion analysis device, and a database, and provides users with individually customized plant-growing plans and advice.

[1543] Hardware and Software Configuration

[1544] Terminal

[1545] The terminal is a smartphone or tablet that allows users to take pictures of plants, send the data to the server, and receive and display cultivation plans and advice. The terminal is equipped with a camera, GPS module, and internet connection.

[1546] server

[1547] The server is a central device for storing and processing the received images and additional information. The server is equipped with an image recognition model, an emotion analysis device, and a database.

[1548] Image recognition model: Uses machine learning algorithms such as TensorFlow and PyTorch to identify the type of plant from the received image.

[1549] Emotion analysis device: A device that uses natural language processing (NLP) and facial expression recognition algorithms to analyze a user's emotional state.

[1550] Database: A system that stores and manages plant growth conditions and related information.

[1551] Data processing and calculation flow

[1552] 1. Image Reception and Preprocessing

[1553] The device sends the plant image and additional information (date and time of the photo, location information, and emotion data) to the server, which then receives the image and standardizes it by converting its format and resizing it.

[1554] 2. Image Recognition Processing

[1555] The server uses image recognition models to identify plant species, which identifies the names and genus species of plants contained in the image.

[1556] 3. Extraction of growth conditions

[1557] Based on the identified plant, the server extracts growing conditions (such as watering frequency, sunlight conditions, and type of fertilizer) from the database.

[1558] 5. Emotion Data Analysis

[1559] The server uses an emotion analyzer to identify the user's emotional state from their input data and facial expressions. For example, it may determine that the user is "stressed" based on their text input.

[1560] 6. Creating and customizing training plans

[1561] The server then combines the extracted training conditions with the emotional data to generate an optimal training plan for the user, adding customized advice and encouraging messages according to the user's emotional state.

[1562] 7. Development plan and message sending

[1563] The server sends the generated training plan and message to the terminal, which receives it and notifies and displays it to the user.

[1564] Specific examples

[1565] For example, if a user posts a photo of a Monstera plant in their living room and a vase of roses, the following happens:

[1566] 1. The user takes a photo of a plant using the in-app camera, then presses the "Post" button to send the image.

[1567] 2. The device sends the captured image and additional information (date and time of capture, location information, and emotion data) to the server.

[1568] 3. The server receives the image, normalizes it, and resizes it.

[1569] 4. The server uses image recognition models to identify the Monstera and the rose, and their plant names are identified.

[1570] 5. The server retrieves the growing conditions for the identified plants from the database. For example, a Monstera should be watered every two weeks and indirect sunlight, while a rose should be watered once a week and in direct sunlight.

[1571] 6. The server uses an emotion analyzer to analyze whether the user is feeling "stressed."

[1572] 7. The server generates a care plan by adding advice to help reduce stress. For example, it adds an encouraging message such as, "Watering this Monstera every morning will help you start your day relaxing."

[1573] 8. The server sends the final training plan and encouraging messages to the device.

[1574] 9. The device displays the training plan and message received within the app and notifies the user.

[1575] Prompt Sentence Examples

[1576] Use a prompt like this for the emotion engine:

[1577] "The user's current emotional state is stressed. Generate advice suggesting how to care for plants to reduce stress."

[1578] As described above, the present invention is a system that contributes to the care of the user's mind and body by providing a plant cultivation plan that takes into account the user's emotional state.

[1579] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1580] Step 1:

[1581] The user opens the application and takes a picture of a plant or bouquet using the device's camera, or selects an existing image from the photo gallery. When the user presses the "Post" button, the selected or taken image is temporarily saved on the device. The input is the taken or selected image, and the output is the temporarily saved image.

[1582] Step 2:

[1583] Based on the user's operation, the device sends the temporarily saved image and additional information (date and time of shooting, location information, emotional data) to the server. Emotional data is extracted from direct input from the user or facial expressions captured by the device's camera. The device then sends the acquired data to the server. The input is the image and additional information, and the output is the data sent to the server.

[1584] Step 3:

[1585] The server receives the image and additional information sent from the device. It then standardizes the image format and resizes it as necessary. Image standardization includes unifying the image format and adjusting the resolution. The input is the data received from the device, and the output is the standardized image and metadata.

[1586] Step 4:

[1587] The server uses an image recognition model on the standardized images to identify the plant type. This process is performed using machine learning models such as TensorFlow and PyTorch. The model identifies plant types with high accuracy based on the training dataset. The input is the standardized image, and the output is the plant type identification result.

[1588] Step 5:

[1589] Based on the identified plant type, the server extracts the growing conditions from the database. Growing conditions include watering frequency, sunlight requirements, type of fertilizer, etc. This data extraction is performed using SQL queries. The input is the plant identification results, and the output is the growing conditions for each plant.

[1590] Step 6:

[1591] When the server identifies multiple plants in an image, it integrates their respective growing conditions to generate a consistent growing plan. It compares the multiple growing conditions and adjusts and integrates them to find common and conflicting parts. The input is multiple growing conditions, and the output is an integrated growing plan.

[1592] Step 7:

[1593] The server uses an emotion analyzer to analyze the transmitted emotional data. It uses natural language processing and facial expression recognition algorithms to identify the user's emotional state (e.g., stress, joy, sadness). The input is the emotional data, and the output is the analyzed emotional state.

[1594] Step 8:

[1595] The server further customizes the integrated training plan based on the analyzed emotional state. For example, if the user is feeling stressed, it adds relaxing training advice or encouraging messages. The input is the training plan and the user's emotional state, and the output is a customized training plan.

[1596] Step 9:

[1597] The server transmits the generated customized development plan and encouraging message to the terminal. The input is the customized development plan and message, and the output is the data transmitted to the terminal.

[1598] Step 10:

[1599] The device notifies the user of the plant care plan and messages received and displays them on the app's user interface. Push notifications and reminder functions are used to encourage timely care. Users care for their plants according to the provided plant care plan and advice, and provide additional feedback to the app, allowing the emotion engine to track changes in the user's emotional state. The input is data received from the server, and the output is notification and display to the user, and feedback collection.

[1600] Through these steps, the system takes into account the user's emotional state, provides an optimal plant cultivation plan, and supports the user's physical and mental health.

[1601] (Application example 2)

[1602] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1603] Conventional plant-growing applications were able to identify plant species from images posted by users and provide information on growing conditions. However, these applications were unable to customize growing plans that took into account the user's emotional state, making it difficult to improve user satisfaction and care for their mind and body. Furthermore, when selecting plants in physical stores, there was a lack of support for users to choose plants that match the plant's characteristics and their own mental and physical state. This has created a need for applications that can provide both appropriate growing plans and care for the user's mind and body.

[1604] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1605] In this invention, the server includes a means for generating a customized plant growing plan based on the user's emotional state, a means for transmitting the generated plant growing plan and a message based on the user's emotion to the terminal, and a means for presenting the plant growing plan and message received by the terminal to the user. This makes it possible to provide a plant growing plan that takes the user's emotional state into consideration, and realizes a system that supports users in selecting appropriate plants in a physical store while also providing physical and mental care.

[1606] "User" refers to an individual who posts an image of a plant and receives a growing plan.

[1607] "Terminal" refers to an electronic device used by a User, including a smartphone, tablet, or computer.

[1608] "Image" means a visual representation of a plant photographed or selected by a User using a Device.

[1609] "Metadata" refers to additional information that accompanies an image, including the date and time the image was taken, location information, and the user's emotional data.

[1610] "Server" refers to a remote computer that receives, stores, and analyzes data sent from the terminal and generates training plans and messages.

[1611] "Image recognition model" refers to a program that implements a machine learning algorithm used to analyze images and identify plant species.

[1612] "Plant type" refers to the classification of a particular plant as identified by the image recognition model.

[1613] "Growth conditions" refers to information that indicates the optimal cultivation method based on the type of plant, and refers to data including watering frequency, sunlight conditions, and appropriate type of fertilizer.

[1614] A "growing plan" refers to a plan that integrates the growing conditions of the identified plants and indicates specific growing methods customized based on the user's emotional state.

[1615] "Emotional State" refers to the user's current mental state as analyzed using the user's emotion engine.

[1616] "Emotion-based messages" refer to words of encouragement or advice that are generated based on the user's emotional state.

[1617] A system for implementing the present invention includes an application for supporting plant growth and a configuration for providing customized advice based on emotional states. Specific embodiments of this system will be described below.

[1618] Hardware and software used

[1619] Device: A smartphone, tablet, or computer device through which users can post plant images.

[1620] Server: A remote computer that receives, stores, and analyzes data sent from the device.

[1621] Image recognition models: Use algorithms built using machine learning frameworks such as TensorFlow or PyTorch.

[1622] Sentiment engine: Uses natural language processing services such as IBM Watson and Google Cloud Natural Language API.

[1623] Database: Use a relational database such as PostgreSQL or MySQL.

[1624] Front-end: Building the application's user interface using React Native or Flutter.

[1625] Specific details of data processing and calculation

[1626] 1. User-submitted images

[1627] The device receives input from the user and sends the plant image and metadata (date and time of capture, location, and emotional data) to the server, where the user is prompted to enter their real-time emotional state.

[1628] 2. Data processing by the server

[1629] The server standardizes and resizes the received image data, identifies the plant type using an image recognition model, extracts growing conditions from a database based on the identified plant type, and then integrates the extracted growing conditions to generate a customized growing plan and message based on the user's emotional state.

[1630] 3. Notice to Users

[1631] The server sends the generated plant care plan and emotion-based messages to the device, which then presents them to the user, allowing the user to learn the optimal plant care method based on their emotional state.

[1632] Specific examples

[1633] For example, consider a case where a user takes a photo of a plant and then inputs their emotional state according to the following prompt:

[1634] Example prompt sentence:

[1635] "Please tell us about your recent stressful situations. Please talk about your feelings and worries."

[1636] If the user types "I'm tired," the server processing will generate a message like this:

[1637] Sample message:

[1638] "The plant chosen this time is the Monstera, which helps relieve stress. Make sure to water it regularly and provide it with adequate sunlight. When you're busy, just looking at the greenery of this plant can have a relaxing effect."

[1639] This system allows users to take their emotional state into consideration and implement optimal plant care methods, as well as receive advice that can help relieve daily stress.

[1640] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1641] Step 1:

[1642] User-submitted images

[1643] The user takes a photo of the plant using their device or selects an existing image, then reviews the image within the application and presses the submit button to send the image to the server.

[1644] Input: A user-taken or selected image of a plant.

[1645] Output: Image data is stored on the device and a request is generated to be sent to the server.

[1646] Step 2:

[1647] Sending images and metadata

[1648] The device sends the image selected by the user, along with the date and time of the photo, location information, and emotion data collected based on the prompt text, to the server.

[1649] Input: Plant image, photo date and time, location information, and user emotion data.

[1650] Output: The request is processed and the submitted data is saved on the server.

[1651] Step 3:

[1652] Image reception and preprocessing

[1653] The server receives the image and metadata sent from the device, normalizes the image data, and resizes it if necessary, so that the image is in a format suitable for the image recognition model.

[1654] Input: Received plant images and metadata.

[1655] Output: Normalized and resized image data.

[1656] Step 4:

[1657] Image Recognition Processing

[1658] The server uses an image recognition model to identify the type of plant from the received image. This model has been trained on many plant images in advance, allowing it to identify plant species with high accuracy.

[1659] Input: Standardized plant image data.

[1660] Output: The plant species identified.

[1661] Step 5:

[1662] Obtaining breeding conditions from a database

[1663] Based on the identified plant type, the server extracts from a database the optimal growing conditions for each plant, including watering frequency, light requirements, and the appropriate type of fertilizer.

[1664] Input: Identified plant species.

[1665] Output: Extracted growing condition data.

[1666] Step 6:

[1667] Integration of multiple plant conditions

[1668] If the image contains multiple plants, the server integrates the growing conditions of each plant and generates a consistent growing plan.

[1669] Input: Growing conditions for multiple plants.

[1670] Output: Integrated development plan.

[1671] Step 7:

[1672] Emotional Data Analysis

[1673] The server uses an emotion engine to analyze the user's emotional data, identify the user's emotional state (e.g., joy, sadness, stress, etc.), and generate appropriate advice or encouraging messages based on the data.

[1674] Input: User emotion data.

[1675] Output: User's emotional state data and an encouraging message.

[1676] Step 8:

[1677] Creation and integration of development plans

[1678] The server further customizes the planting plan by taking into account the user's emotional state, as analyzed by the emotion engine. For example, if the user is feeling stressed, the plan will include advice on plant care that has a relaxing effect.

[1679] Input: User emotional state data.

[1680] Output: A customized development plan.

[1681] Step 9:

[1682] Development plan and message sending

[1683] The server sends a message based on the generated development plan and emotion to the terminal.

[1684] Input: A customized development plan and encouraging message.

[1685] Output: The development plan and message sent.

[1686] Step 10:

[1687] User notification and action

[1688] The device displays the plant care plan and messages received on the app's user interface. Push notifications and reminder functions are used to prompt the user to provide timely care. The user can then care for their plants according to the provided care plan and emotional advice.

[1689] Input: Received development plans and messages.

[1690] Output: The development plan and messages displayed in the user interface.

[1691] In this way, each processing step is designed to perform appropriate calculations on data inputs and obtain customized outputs based on the user's emotional state.

[1692] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1693] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1694] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1695] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1696] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1697] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1698] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1699] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1700] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1701] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1702] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1703] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1704] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1705] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1706] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1707] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1708] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1709] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1710] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1711] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1712] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1713] The following is further disclosed regarding the above embodiment.

[1714] (Claim 1)

[1715] a means for users to post images of plants;

[1716] means for transmitting the posted plant image and metadata to a server by the terminal;

[1717] means for storing the images and metadata received by the server;

[1718] means for identifying the type of plant using an image recognition model on the image received by the server;

[1719] means for the server to extract growing conditions from a database based on the identified plant type;

[1720] A means for the server to integrate the extracted training conditions and generate an optimal training plan for the user;

[1721] A means for transmitting the development plan generated by the server to the terminal;

[1722] A means for presenting the training plan received by the terminal to the user;

[1723] A system including:

[1724] (Claim 2)

[1725] 10. The system of claim 1, wherein the image format is standardized and resized as needed.

[1726] (Claim 3)

[1727] 10. The system of claim 1, wherein the system identifies multiple segments of the image and calculates independent growing conditions for each plant when multiple plants are present.

[1728] "Example 1"

[1729] (Claim 1)

[1730] a means for users to post images of plants;

[1731] means for transmitting the posted plant image and metadata to a server by the terminal;

[1732] means for storing the images and metadata received by the server;

[1733] means for identifying the type of plant using an image recognition model on the image received by the server;

[1734] means for the server to extract growing conditions from a database based on the identified plant type;

[1735] A means for the server to integrate the extracted training conditions and generate an optimal training plan for the user;

[1736] A means for transmitting the development plan generated by the server to the terminal;

[1737] A means for presenting the training plan received by the terminal to the user;

[1738] A means for the device to collect metadata such as location information, shooting date and time, etc.;

[1739] A means for the server to standardize the format of images and resize them as preprocessing;

[1740] a means for the server to integrate the identified growing conditions of the plurality of plants and generate a consistent growing plan;

[1741] A system that includes a means for the device to provide periodic reminders to support the user in continuing to provide appropriate care.

[1742] (Claim 2)

[1743] 10. The system of claim 1, wherein the server standardizes the format of the image and resizes it as needed.

[1744] (Claim 3)

[1745] 10. The system of claim 1, wherein the server identifies multiple segments of the image and calculates independent growing conditions for each plant if multiple plants are present.

[1746] "Application Example 1"

[1747] (Claim 1)

[1748] a means for users to post images of plants;

[1749] means for transmitting the posted plant image and metadata to a server by the terminal;

[1750] means for storing the images and metadata received by the server;

[1751] means for identifying the type of plant using an image recognition model on the image received by the server;

[1752] means for the server to extract growing conditions from a database based on the identified plant type;

[1753] A means for the server to integrate the extracted training conditions and generate an optimal training plan for the user;

[1754] A means for transmitting the development plan generated by the server to the terminal;

[1755] A means for presenting the training plan received by the terminal to the user;

[1756] means for acquiring vehicle sensor data;

[1757] A means for generating an optimal plant growth plan by combining the acquired sensor data with plant growth conditions;

[1758] A system including:

[1759] (Claim 2)

[1760] 10. The system of claim 1, wherein the image format is standardized and resized as needed.

[1761] (Claim 3)

[1762] 10. The system of claim 1, wherein the system identifies multiple segments of the image and calculates independent growing conditions for each plant when multiple plants are present.

[1763] "Example 2: Combining Emotion Engines"

[1764] (Claim 1)

[1765] a means for users to post images of plants;

[1766] means for transmitting the posted plant image and additional information to a server from the terminal;

[1767] means for storing the image and additional information received by the server;

[1768] means for identifying the type of plant using an image recognition model on the image received by the server;

[1769] means for the server to extract growing conditions from a database based on the identified plant type;

[1770] A means for the server to integrate the extracted training conditions and generate an optimal training plan for the user;

[1771] A means for transmitting the development plan generated by the server to the terminal;

[1772] A means for presenting the training plan received by the terminal to the user;

[1773] means for the server to analyze the emotional state of the user using an emotion analyzer;

[1774] means for the server to customize the development plan based on the analyzed emotional state and generate an advice or encouraging message associated with the emotion;

[1775] A system including:

[1776] (Claim 2)

[1777] 10. The system of claim 1, wherein the image format is standardized and resized as needed.

[1778] (Claim 3)

[1779] 10. The system of claim 1, wherein the system identifies multiple segments of the image and calculates and integrates independent growing conditions for each plant when multiple plants are present.

[1780] "Application example 2 when combining emotion engines"

[1781] (Claim 1)

[1782] a means for users to post images of plants;

[1783] means for transmitting the posted plant image and metadata to a server by the terminal;

[1784] means for storing the images and metadata received by the server;

[1785] means for identifying the type of plant using an image recognition model on the image received by the server;

[1786] means for the server to extract growing conditions from a database based on the identified plant type;

[1787] a means for the server to integrate the extracted development conditions and generate a customized development plan based on the user's emotional state;

[1788] means for transmitting a message based on the development plan and emotion generated by the server to the terminal;

[1789] means for presenting the training plan and message received by the terminal to the user;

[1790] A system including:

[1791] (Claim 2)

[1792] 10. The system of claim 1, wherein the image format is standardized and resized as needed.

[1793] (Claim 3)

[1794] 10. The system of claim 1, wherein the system identifies multiple segments of the image and calculates independent growing conditions for each plant when multiple plants are present. [Explanation of symbols]

[1795] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for users to post images of plants; means for transmitting the posted plant image and metadata to a server by the terminal; means for storing the received images and metadata by the server; means for identifying the type of plant using an image recognition model on the image received by the server; means for the server to extract growing conditions from a database based on the identified plant type; A means for the server to integrate the extracted training conditions and generate an optimal training plan for the user; A means for transmitting the development plan generated by the server to the terminal; A means for presenting the training plan received by the terminal to the user; A system including:

2. 10. The system of claim 1, wherein the image format is standardized and resized as needed.

3. 10. The system of claim 1, wherein the system identifies multiple segments of the image and calculates independent growing conditions for each plant when multiple plants are present.

Images