System

Abstract

A system is provided.SOLUTION: A system comprising means for receiving image data and video data obtained from a user, means for analyzing the received image data and video data to identify the type, density, growth stage, and presence of pests of trees, means for generating an evaluation report of a forest based on the analyzed information and weather data obtained from a weather database, means for creating an optimal forest management strategy using AI generated based on the evaluation report, means for transmitting and displaying the created forest management strategy to a terminal of a user, and means for receiving again an execution result of the user and evaluating the effect of the forest management strategy.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern times, camping and outdoor activities have become increasingly popular, leading to an increase in the number of individuals owning their own mountains. However, many new forest owners lack knowledge and experience in forest management, making it difficult to properly manage their forests. As a result, forest health is declining, raising concerns about adverse effects on ecosystems and the degradation of timber resources. Furthermore, forest management requires consideration of many factors, such as tree species, growth rates, and climate change, and making appropriate judgments on these requires advanced expertise. Given this background, there is a need for support to enable even beginners to achieve sustainable forest management and to implement appropriate management that is environmentally friendly. [Means for solving the problem]

[0005] This invention provides a means for receiving image and video data acquired from a user and analyzing them to identify the tree species, density, growth stage, and presence of pests and diseases. Furthermore, a forest assessment report is generated based on the analyzed information and climate data acquired from a weather database. An optimal forest management strategy is created using a generation AI based on the assessment report, and the created strategy is sent to the user's device and displayed, providing specific instructions and schedules in an easy-to-understand format for the user. The system also receives the user's execution results again, evaluates the effectiveness of the forest management strategy, and makes further proposals as necessary. This provides a system that enables even beginners with little knowledge or experience to achieve environmentally friendly, sustainable forest management.

[0006] "User" refers to any individual or entity that owns and manages their own forest land.

[0007] "Image data and video data" refers to still image and video files that users take and upload to the system.

[0008] "Means for receiving" refers to components and protocols for incorporating image data and video data into the system.

[0009] "Means for analyzing" refers to the algorithms and computer vision techniques used to evaluate and analyze received data.

[0010] "Tree type, density, growth stage and presence of pests and diseases" refers to the condition of each element of the forest as determined through image and video analysis.

[0011] "Climate Data" refers to information about local weather conditions obtained from a weather database.

[0012] "Assessment Report" refers to a report on forest health and risks based on analytical results and climate data.

[0013] "Generative AI" refers to algorithms that use artificial intelligence techniques to create optimal forest management strategies.

[0014] "Forest management strategy" refers to specific policies and procedures for sustainable forest management, including optimal harvesting schedules, planting schedules and timing of care.

[0015] "Terminal" refers to a device used by a user, such as a computer, smartphone, or tablet.

[0016] "Implementation Results" refers to the data reported and recorded by users as a result of the work they have done based on the forest management strategy.

[0017] "Means to evaluate" refers to components and protocols for analyzing implementation results, assessing the effectiveness of the strategy, and adjusting it if necessary. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention relates to a system that receives image and video data acquired from users, analyzes them to evaluate the current state of a forest, and generates and presents an optimal forest management strategy. This system provides support, especially for novice forestry managers, to achieve sustainable forest management.

[0040] System Overview

[0041] The system receives image and video data taken by users of their own mountains and analyzes it to identify forest species, density, growth stage, and the presence of pests and diseases. It also collects meteorological data to generate a comprehensive assessment report and uses AI to develop an optimal forest management strategy. The strategy is then sent to the user's device, and the user can carry out specific tasks based on it, with the results then fed back into the system.

[0042] System configuration

[0043] 1. User's device

[0044] Photography function: Users can use their smartphones or cameras to capture image and video data of the forest.

[0045] Data upload function: Upload captured data to the system via a dedicated application.

[0046] 2. Server

[0047] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database.

[0048] Analysis features:

[0049] Image analysis algorithms are used to identify tree species, density, growth stage, and the presence of pests and diseases.

[0050] Use the API to obtain local climate information from a weather database.

[0051] Assessment report generation function: Integrates analyzed information with meteorological data to generate assessment reports on forest health and risks.

[0052] Strategic planning function using generative AI: Automatically creates optimal forest management strategies based on assessment reports.

[0053] Strategy notification function: The created management strategy is sent to the user's terminal and presented to the user in a visual interface.

[0054] Feedback analysis function: Receives execution results from users, reevaluates the effectiveness of the strategy, and adjusts it as necessary.

[0055] Program processing

[0056] Below, we will explain in detail an example of how the system's programs are used and the processing flow.

[0057] 1. Uploading image and video data

[0058] To understand the current state of their mountain, users take multiple photos and videos using their smartphone or camera and upload them to the system via the application.

[0059] 2. Receipt and storage of data

[0060] The server receives the data uploaded by the user and stores it securely in a database.

[0061] 3. Data Analysis

[0062] The server passes the received data through image analysis algorithms to identify tree species, density, growth stage and the presence of pests and diseases.

[0063] The server also retrieves climate information for the target area from a weather database via an API.

[0064] 4. Generate an evaluation report

[0065] The server integrates image analysis results with meteorological data to generate assessment reports on forest health and risks.

[0066] 5. Generating a management strategy

[0067] Based on the assessment report, the server uses generative AI to create an optimal forest management strategy, including a felling plan, a planting plan, and the timing of maintenance.

[0068] 6. Informing and implementing control strategies

[0069] The server transmits the created management strategy to the user's terminal and presents it to the user through a visual interface.

[0070] The user carries out specific tasks based on the presented strategies.

[0071] 7. Feedback on execution results

[0072] The user reports the results of their work to the system through the application, which includes photos of the work and a status report.

[0073] 8. Evaluate and recalibrate your strategy

[0074] The server receives the execution results from the user, evaluates the effectiveness of the strategy, adjusts the strategy if necessary, and makes next suggestions.

[0075] This system makes it easy for even new forest owners to achieve sustainable forest management. By continuing to implement appropriate forest management through specific instructions and feedback, environmentally friendly forest management becomes possible.

[0076] The processing flow will be explained below.

[0077] Step 1:

[0078] Users take pictures of their mountain using a smartphone or camera. In order to collect sufficient information, they take multiple images and videos from various angles.

[0079] Step 2:

[0080] The user launches a dedicated application and uploads the captured image and video data to the application, which then transmits the data to the system.

[0081] Step 3:

[0082] The server receives the image and video data sent by the user and stores the received data securely in a database.

[0083] Step 4:

[0084] The server then passes the received data to an image analysis algorithm that uses computer vision technology to identify tree species, density, growth stage, and the presence of pests and diseases.

[0085] Step 5:

[0086] The server connects to a weather database and retrieves local weather information through an API, including past weather conditions and future forecast data.

[0087] Step 6:

[0088] The server combines image analysis results with meteorological data to generate forest health and risk assessment reports, which contain detailed information about the condition of the forest.

[0089] Step 7:

[0090] The server uses generative AI to create an optimal forest management strategy based on the assessment report, including optimal harvesting plans, planting plans, and maintenance timing.

[0091] Step 8:

[0092] The server then sends the created forest management strategy to the user's device, where the strategy can be viewed through a visual interface.

[0093] Step 9:

[0094] Users are then presented with a management strategy on their device, which involves carrying out specific tasks, such as cutting trees, planting trees, and tending them at set times.

[0095] Step 10:

[0096] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[0097] Step 11:

[0098] The server again receives and analyzes the execution result data from the user, thereby evaluating the effectiveness of the strategy and readjusting it if necessary.

[0099] Step 12:

[0100] The server notifies the user's device of updated strategies and next proposals, allowing the user to continuously implement appropriate forest management.

[0101] In this way, the system provides continuous support to users in achieving sustainable forest management.

[0102] Example 1

[0103] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0104] Traditional forest management requires specialized knowledge and experience, making it difficult for novice forest managers. Accurately assessing forest health and risks and formulating appropriate management strategies requires advanced technology and a great deal of time and effort. Furthermore, it is difficult to effectively utilize meteorological data, making it difficult to develop optimal felling and planting plans. This makes it difficult to achieve sustainable forest management.

[0105] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0106] In this invention, the server includes means for receiving image data and video data of a forest captured by a user, means for analyzing the received image data and video data using an image analysis algorithm to identify the forest type, density, growth stage, and presence of pests and diseases, means for generating a forest assessment report based on the analyzed information and meteorological data obtained from an external database, means for creating an optimal forest management strategy using a generative AI model based on the assessment report, means for sending the created forest management strategy to the user's device and displaying it in a visual interface, and means for receiving the user's execution results again and evaluating the effectiveness of the forest management strategy. This enables sustainable forest management even for novice foresters without advanced expertise.

[0107] A "user" is a person who uses the system to provide image and video data related to forest management, and receives and implements analysis results and management strategies.

[0108] "Image and video data" refers to photographs and video information taken by users of the current state of the forest, and is used to identify the type and density of trees, their growth stage, and the presence of pests and diseases.

[0109] The "means for receiving" refers to the process and mechanism by which the server acquires and stores image data and video data uploaded by users.

[0110] "Image analysis algorithm" means a mathematical and technical method for analyzing received image and video data to identify tree species, density, growth stage, and the presence of pests and diseases.

[0111] "Weather data" refers to local climate information obtained from external databases and is data that is integrated when generating forest assessment reports.

[0112] The "assessment report" is a report that evaluates the health and risks of a forest based on image analysis results and meteorological data.

[0113] A "generative AI model" is an artificial intelligence model that automatically creates optimal forest management strategies based on assessment reports.

[0114] A "forest management strategy" is a specific plan for managing forests sustainably, including optimal felling plans, planting plans, and maintenance timing.

[0115] The "visual interface" refers to the screen display and operation means that allow the server to display the forest management strategy generated by the server in an easy-to-understand manner for the user.

[0116] "Execution results" are the results of work carried out by the user based on the forest management strategy presented, and are re-evaluated.

[0117] The "means for evaluating effectiveness" refers to the process and techniques by which the server receives the execution results from the user and re-evaluates the effectiveness of the existing forest management strategy.

[0118] The present invention relates to a system that receives image and video data captured by users, analyzes them to evaluate the current state of a forest, and generates and presents an optimal forest management strategy. This system provides support, especially for novice foresters, to achieve sustainable forest management.

[0119] System Overview

[0120] The system receives image and video data taken by users of their own mountains and analyzes it to identify forest species, density, growth stage, and the presence of pests and diseases. It also collects meteorological data to generate a comprehensive assessment report and uses a generative AI model to develop an optimal forest management strategy. The strategy is then sent to the user's device, and the user can carry out specific tasks based on it, with the results then fed back into the system.

[0121] Hardware and Software Configuration

[0122] 1. User's device

[0123] Photography function: Users can use their smartphones or digital cameras to capture image and video data of the forest.

[0124] Data upload function: Upload captured data to the system through a dedicated application (e.g., "ForestCare" or "TreeHealth").

[0125] 2. Server

[0126] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database (e.g., MySQL, MongoDB).

[0127] Image analysis algorithms: Image analysis algorithms (e.g., TensorFlow, PyTorch) are used to identify tree species, density, growth stage, and the presence of pests and diseases.

[0128] Weather data acquisition: Use an API (e.g., OpenWeatherMap API) to obtain local climate information from a weather database.

[0129] Assessment report generation: Use Jupyter Notebook and Pandas to integrate the analyzed information with meteorological data and generate assessment reports on forest health and risks.

[0130] Strategy planning using generative AI: Based on the evaluation report, a generative AI model (e.g., GPT-4) is used to create an optimal forest management strategy.

[0131] Strategy notification: The created management strategy is sent to the user's terminal and presented to the user in a visual interface (e.g., a web application).

[0132] Feedback analysis: Receive execution results from users, reassess the effectiveness of the strategy, and readjust the strategy if necessary.

[0133] Specific examples

[0134] For example, a novice forestry manager might follow these steps to understand the current state of his or her mountain:

[0135] 1. Capture and upload data:

[0136] Users use their smartphones to take photos and videos of the mountain from multiple angles.

[0137] Launch the dedicated app "ForestCare" and upload the captured data to the system.

[0138] 2. Data analysis and evaluation:

[0139] The server passes the received data to a TensorFlow image analysis algorithm, which identifies tree species, density, growth stage, and the presence of pests and diseases.

[0140] Obtain weather data for the target area from OpenWeatherMap via API.

[0141] 3. Generate an assessment report:

[0142] The server integrates the image analysis results and meteorological data using Pandas and creates an evaluation report.

[0143] Create a report in Jupyter Notebook and save it as a PDF.

[0144] 4. Generate a management strategy:

[0145] The server uses a generative AI model (GPT-4) to input the following prompt based on the evaluation report: "Please tell me the appropriate tree-cutting and planting plan for this area."

[0146] Format the strategy proposed by GPT-4 and notify the user.

[0147] 5. Notice and Execution:

[0148] The server pushes the created management strategy to the user's app.

[0149] Users can check the strategy through the app and carry out the tasks presented.

[0150] 6. Feedback and reassessment of results:

[0151] Users report their work results to the app and upload photos and status reports.

[0152] The server receives and analyzes the reports and readjusts the strategy as needed.

[0153] In this way, this system enables even novice foresters to achieve sustainable forest management without specialized knowledge.

[0154] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0155] Step 1:

[0156] To understand the current state of their mountain, users take image and video data of the forest using a smartphone or digital camera. The input is the captured image and video data, and the output is the saving of this data on the user's device. Specifically, users take photos of the overall view from multiple angles, the condition of nearby trees, and the state of damage caused by pests and diseases.

[0157] Step 2:

[0158] The user launches a dedicated mobile application (e.g., "ForestCare" or "TreeHealth") and uploads the captured data to the system. The input is image and video data stored on the user's device, and the output is the data being sent to the system's server. Specifically, the user taps the "Upload Data" button on the application, selects the captured data, and uploads it.

[0159] Step 3:

[0160] The server immediately receives image and video data uploaded by users and securely stores it in a database (e.g., MySQL, MongoDB). The input is the image and video data received from users, and the output is the data stored in the database. Specifically, the server receives the data uploaded via an HTTP request and records it in the database.

[0161] Step 4:

[0162] The server passes the received image and video data to an image analysis algorithm (e.g., TensorFlow, PyTorch) and performs the analysis. The input is the image and video data stored in the database, and the output is the analysis results regarding the tree species, density, growth stage, and the presence of pests and diseases. Specifically, the server inputs the data into the image analysis algorithm, extracts specific features, and evaluates them.

[0163] Step 5:

[0164] The server obtains the analysis results and uses an API (e.g., OpenWeatherMap API) to obtain weather data for the target area. The input is the analysis results and information about the target area, and the output is the obtained weather data. Specifically, the server sends a request to the API endpoint to obtain the weather data.

[0165] Step 6:

[0166] The server integrates the image analysis results with meteorological data to generate an assessment report on forest health and risks. The input is the image analysis results and meteorological data, and the output is an assessment report. Specifically, the server integrates the analysis results and meteorological data using Pandas, creates an assessment report in Jupyter Notebook, and saves it in PDF format.

[0167] Step 7:

[0168] The server uses a generative AI model (e.g., GPT-4) to create an optimal forest management strategy based on the evaluation report. The input is the evaluation report and a prompt (e.g., "Please tell me the appropriate logging and planting plan for this area."), and the output is the generated forest management strategy. Specifically, the server inputs the evaluation report as a prompt to GPT-4 and obtains the generated strategy.

[0169] Step 8:

[0170] The server sends the created forest management strategy to the user's device and presents it to the user through a visual interface. The input is the created forest management strategy, and the output is the strategy displayed on the user's device. Specifically, the server pushes the strategy to the user's app in JSON format, and the strategy is displayed visually on the application.

[0171] Step 9:

[0172] The user carries out specific tasks based on the presented forest management strategy. The input is the forest management strategy displayed on the user's terminal, and the output is the results of the executed tasks. As specific actions, the user performs tasks such as cutting trees, planting, and maintenance according to the strategy.

[0173] Step 10:

[0174] Users report the results of their work through the application and upload photos and status reports after the work is completed to the system. The input is data about the work performed and its results, and the output is feedback sent to the system. Specifically, users upload photos and reports using the app's "Work Completion Report" function.

[0175] Step 11:

[0176] The server receives feedback from the user, analyzes it again, and evaluates the effectiveness of the forest management strategy. It readjusts the strategy as needed and presents the next proposal to the user. The input is the received feedback data, and the output is the evaluated effectiveness of the strategy and the readjusted management strategy. Specifically, the server passes the feedback data to an analysis algorithm for evaluation, and if necessary, uses a generative AI model to generate a new strategy.

[0177] This process allows even novice foresters to achieve sustainable forest management. The process of specific instructions and feedback is repeated, allowing for more precise management.

[0178] (Application example 1)

[0179] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0180] Maintaining and managing factory equipment and production lines requires a great deal of time and effort, and conventional methods have the problem of making it difficult to detect abnormalities early and formulate appropriate maintenance plans.The present invention aims to improve the operational efficiency of equipment and reduce unexpected downtime by efficiently evaluating the health of factory equipment and accurately proposing necessary maintenance.

[0181] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0182] In this invention, the server includes means for receiving image data and video data acquired from a user, means for analyzing the received image data and video data to identify the type of equipment, its operating status, and the presence or absence of abnormalities, and means for generating an equipment evaluation report based on the analyzed information and environmental data acquired from the manufacturing environment database. This makes it possible to evaluate the health of the equipment in real time and present optimal maintenance plans, replacement plans, and maintenance timing.

[0183] The "means for receiving image data and video data acquired from the user" is a function for transmitting image data and video data captured by the user to a server via the Internet and receiving the data.

[0184] "Means for analyzing received image data and video data to identify the type of equipment, its operating status, and whether or not there are any abnormalities" refers to a function that analyzes received image data and video data using computer vision technology and machine learning algorithms, etc., to identify the type of specific equipment, its operating status, and signs of abnormalities.

[0185] "Means for generating an equipment evaluation report based on the analyzed information and environmental data obtained from the manufacturing environment database" refers to a function for automatically generating an evaluation report by combining the analysis results with various environmental data (e.g., temperature, humidity, etc.) collected in advance.

[0186] "A means for creating an optimal asset management strategy using generative AI based on an evaluation report" is a function that utilizes a generative AI model to automatically consider and create optimal maintenance plans and strategies from the generated evaluation report.

[0187] "Means for sending and displaying the created facility management strategy to the user's device" refers to a function for sending the generated optimal management strategy to the user's smartphone, computer, etc., and displaying it in an intuitive interface.

[0188] "Means of receiving the user's execution results again and evaluating the effectiveness of the asset management strategy" refers to a function that receives the results of the maintenance work and maintenance that has been carried out again as feedback to the system, evaluates the results, and reflects them in future strategies.

[0189] "Equipment management strategy including optimal maintenance plan, replacement plan and maintenance timing" is a management strategy including the most appropriate maintenance work schedule, part replacement timing and maintenance method for maintaining the health of equipment.

[0190] "Facility management" refers to the planned maintenance, repair, and replacement work required to operate the equipment and machinery used in factories and production lines efficiently and effectively.

[0191] The present invention relates to a system that efficiently evaluates the health of equipment and manufacturing lines in a factory and proposes optimal maintenance plans and strategies. This system provides significant support, especially to factory managers who find equipment management difficult. The system is realized by the following procedure.

[0192] System configuration

[0193] 1. User's device

[0194] Photography function: Users can use their smartphones or cameras to capture image and video data of equipment and production lines.

[0195] Data upload function: Upload captured data to a server via a dedicated application.

[0196] 2. Server

[0197] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database.

[0198] Analysis features:

[0199] Image analysis algorithms are used to identify the type of equipment, its operating status, and whether or not there are any abnormalities.

[0200] Use the API to obtain environmental data from the manufacturing environment database.

[0201] Assessment report generation function: Integrates analyzed information with environmental data to generate assessment reports on the health and risk of facilities.

[0202] Generative AI strategy planning function: Based on the assessment report, generates an asset management strategy including optimal maintenance plans, replacement plans, and maintenance timing.

[0203] Strategy notification function: The created facility management strategy is sent to the user's device and presented to the user via a visual interface.

[0204] Feedback analysis function: Receives execution results from users, reevaluates the effectiveness of the strategy, and adjusts it as necessary.

[0205] Program processing explanation

[0206] Hardware:

[0207] Smartphones: Used as photography devices by factory robots.

[0208] Server: Cloud server (e.g. AWS, Google Cloud) for data analysis and management strategy generation.

[0209] software:

[0210] OpenCV: Used for image and video capture.

[0211] TensorFlow: Image analysis algorithms for anomaly detection and analysis.

[0212] REST API: Used for data upload and notifications (e.g. FastAPI).

[0213] Examples:

[0214] As an example, consider the case where the motor of equipment A is overheating. A user uses a smartphone to take pictures of equipment A and uploads the images and videos to the system. The server receives these and analyzes them using TensorFlow. Based on the analysis results, the generation AI creates an optimal maintenance plan and notifies the user again on their device. In this case, the following can be used as an example of a prompt text:

[0215] Example prompt sentence:

[0216] "Equipment A has been operating for a long time recently, and the motor temperature is rising sharply. Please detect this situation and generate an optimal maintenance plan."

[0217] The system helps reduce unplanned downtime while ensuring facility safety and efficiency.

[0218] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0219] Step 1:

[0220] Users use smartphones or cameras to take images and videos of the equipment and production lines in the factory. These become the input data. Specifically, images are taken from multiple viewpoints to record the equipment's condition in detail.

[0221] Step 2:

[0222] The user's device uploads the captured image and video data to the server via a dedicated application. The input data is the image and video data captured in the previous step, and the output is the completion of uploading to the server. Specifically, the data is sent using a data transfer protocol (e.g., HTTP).

[0223] Step 3:

[0224] The server receives uploaded image and video data and stores it securely in a database. The input data is the data uploaded by the user, and the output is the data that has been saved. Specifically, the received data is converted into an appropriate format and stored in the database.

[0225] Step 4:

[0226] The server passes the received data to an image analysis algorithm (using, for example, OpenCV or TensorFlow) to identify the type of equipment, its operating status, and whether or not there are any abnormalities. The input data is the stored image and video data, and the output is the analysis results (type of equipment, operating status, and whether or not there are any abnormalities). Specifically, the algorithm scans the image and detects specific patterns and abnormalities.

[0227] Step 5:

[0228] The server obtains environmental data (e.g., temperature, humidity) from the manufacturing environment database through the API. The input data is a request to the environment database, and the output is the obtained environmental data. Specifically, it accesses the API endpoint and retrieves the required data.

[0229] Step 6:

[0230] The server integrates the image analysis results with the environmental data to generate an assessment report on the health and risk of the facility. The input data are the analysis results and environmental data, and the output is an assessment report. Specifically, it runs an assessment algorithm that integrates both sets of data and generates the assessment results in text and graph format.

[0231] Step 7:

[0232] The server uses a generative AI model based on the evaluation report to create an equipment management strategy that includes optimal maintenance plans, replacement plans, and maintenance timing. The input data is the evaluation report, and the output is the management strategy. Specifically, the server inputs prompts into the generative AI model to generate the optimal strategy.

[0233] Step 8:

[0234] The server sends the created facility management strategy to the user's device and presents it to the user in a visual interface. The input data is the management strategy, and the output is a notification to the user's device. Specifically, the server encodes the management strategy as a message and sends it to the user's application.

[0235] Step 9:

[0236] The user performs specific maintenance work based on the transmitted management strategy and reports the results back to the system. The input data is the implementation results, and the output is feedback data. Specific operations include recording the status after the maintenance work using images and text, and reporting it through the application.

[0237] Step 10:

[0238] The server receives execution results from the user, analyzes them, and reevaluates the effectiveness of the asset management strategy. If necessary, it adjusts the strategy and makes the next proposal. The input data is feedback data, and the output is a new strategy. Specifically, it analyzes the execution results, reflects any necessary modifications in the generative AI model, and creates a new management strategy.

[0239] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0240] This invention relates to a system that receives image and video data acquired from users, analyzes them to evaluate the current state of a forest, and generates and presents an optimal forest management strategy. Furthermore, the system aims to enhance the effectiveness of the management strategy by incorporating an emotion engine that recognizes the user's emotions. This system provides support, especially for novice foresters, to achieve sustainable forest management.

[0241] System Overview

[0242] The system receives image and video data taken by users of their own mountains and analyzes it to identify forest species, density, growth stage, and the presence of pests and diseases. It also collects weather data to generate a comprehensive assessment report and uses generative AI to develop optimal forest management strategies. Furthermore, by utilizing an emotion engine that recognizes the user's emotions, the system adjusts the way strategies are presented to the user based on their emotional state, enabling them to implement strategies in a way that is more receptive to the user.

[0243] System configuration

[0244] 1. User's device

[0245] Photography function: Users can use their smartphones or cameras to capture image and video data of the forest.

[0246] Data upload function: Upload captured data to the system via a dedicated application.

[0247] Emotion recognition: Equipped with sensors and cameras to recognize the user's emotional state.

[0248] 2. Server

[0249] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database.

[0250] Analysis features:

[0251] Image analysis algorithms are used to identify tree species, density, growth stage, and the presence of pests and diseases.

[0252] Use the API to obtain local climate information from a weather database.

[0253] Assessment report generation function: Integrates analyzed information with meteorological data to generate assessment reports on forest health and risks.

[0254] Strategic planning function using generative AI: Automatically creates optimal forest management strategies based on assessment reports.

[0255] Sentiment visualization and strategy adjustment features:

[0256] Recognize user emotions in real time and adjust the presentation and content of management strategies.

[0257] Strategy notification function: The created management strategy is sent to the user's terminal and presented to the user in a visual interface.

[0258] Feedback analysis function: Receiving execution results from users, evaluating the effectiveness of strategies and adjusting them as necessary.

[0259] Program processing

[0260] Below, we will explain in detail an example of how the system's programs are used and the processing flow.

[0261] 1. Uploading image and video data

[0262] To understand the current state of their mountain, users take multiple photos and videos using their smartphone or camera and upload them to the system via the application.

[0263] 2. Collecting Emotional Data

[0264] The user's device uses techniques such as facial recognition and voice analysis to detect the user's emotional state in real time and transmits that data to a server.

[0265] 3. Receipt and storage of data

[0266] The server receives the image data, video data, and emotion data sent by the user and safely stores them in a database.

[0267] 4. Data Analysis

[0268] The server passes the received data through image analysis algorithms to identify tree species, density, growth stage and the presence of pests and diseases.

[0269] The server also retrieves climate information for the target area from a weather database via an API.

[0270] 5. Integrating evaluation reports and sentiment analysis

[0271] The server integrates the image analysis results, meteorological data, and user emotional data to generate an assessment report on the health and risks of the forest.

[0272] 6. Generating a management strategy

[0273] Based on the assessment report, the server uses generative AI to create an optimal forest management strategy, which includes a felling plan, a planting plan, and the timing of maintenance.

[0274] 7. Adjusting strategies based on emotional state

[0275] The server takes into account the user's emotional state and adjusts the content and presentation of the management strategies it presents, for example, providing encouraging messages or simplified instructions if the user is feeling anxious.

[0276] 8. Informing and implementing control strategies

[0277] The server transmits the created management strategy to the user's terminal, allowing the user to check the management strategy through a visual interface.

[0278] Users are then presented with a management strategy on their device, which involves carrying out specific tasks, such as cutting trees, planting trees, and tending them at set times.

[0279] 9. Feedback on execution results

[0280] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[0281] 10. Evaluate and recalibrate your strategy

[0282] The server again receives and analyzes the execution result data from the user, thereby evaluating the effectiveness of the strategy and readjusting it if necessary.

[0283] 11. Notification of next proposal

[0284] The server notifies the user's device of updated strategies and next proposals, allowing the user to continuously implement appropriate forest management.

[0285] This system allows even novice foresters to easily achieve sustainable forest management while taking into account the emotional state of the foresters. The introduction of the emotion engine is expected to improve user acceptance and lead to more effective management.

[0286] The processing flow will be explained below.

[0287] Step 1:

[0288] Users take pictures of their mountain using a smartphone or camera, and capture multiple image and video data to capture the overall picture and detailed parts of the forest.

[0289] Step 2:

[0290] The user starts the dedicated application and uploads the captured image data and video data to the application.

[0291] Step 3:

[0292] The user's device analyzes the user's facial expressions and voice in real time to recognize their emotional state, using an emotion recognition engine to distinguish emotions such as joy, surprise, sadness, and anger.

[0293] Step 4:

[0294] The user's terminal transmits the emotional state data and the captured image and video data to the server.

[0295] Step 5:

[0296] The server receives the image and video data and emotional state data sent by the user, and stores the received data securely in a database.

[0297] Step 6:

[0298] The server analyzes the received image and video data using an image analysis algorithm, specifically extracting the following information:

[0299] Tree types

[0300] Tree density

[0301] Tree growth stages

[0302] Presence of pests and diseases

[0303] Step 7:

[0304] The server uses an API to retrieve local weather information from a weather database, including historical and forecast weather conditions.

[0305] Step 8:

[0306] The server generates an assessment report on forest health and risks based on the image analysis results and acquired climate data.

[0307] Step 9:

[0308] The server uses generative AI to create an optimal forest management strategy based on the assessment report, including harvesting plans, planting plans, and maintenance timing.

[0309] Step 10:

[0310] The server tailors the presentation and content of management strategies based on the user's emotional state: for example, if the user is feeling anxious, it offers encouraging messages or simplified instructions.

[0311] Step 11:

[0312] The server transmits the created management strategy to the user's terminal, allowing the user to check the management strategy on a visual interface.

[0313] Step 12:

[0314] The user is then presented with a management strategy and performs specific tasks, including cutting, planting, and tending at specified times.

[0315] Step 13:

[0316] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[0317] Step 14:

[0318] The server again receives and analyzes the execution result data from the user, thereby evaluating the effectiveness of the strategy and readjusting it if necessary.

[0319] Step 15:

[0320] The server notifies the user's device of updated strategies and next proposals, allowing the user to continuously implement appropriate forest management.

[0321] In this way, a system incorporating an emotion engine can provide optimal strategies according to the user's emotional state, enabling even beginners to achieve sustainable forest management.

[0322] Example 2

[0323] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0324] Conventional forest management systems could use user-provided image and video data to identify tree species, density, growth stage, and the presence of pests and diseases. However, they lacked the ability to present appropriate management strategies that take the user's emotional state into account. This made it difficult for novice forest managers to receive appropriate guidance and implement sustainable forest management. The present invention aims to solve this problem and provide a more comprehensive and effective forest management strategy.

[0325] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving image data and video data acquired from a user, a means for analyzing the received image data and video data to identify the tree species, density, growth stage, and presence of pests and diseases, a means for generating a forest assessment report based on the analyzed information and climate data acquired from a weather database, a means for creating an optimal forest management strategy based on the assessment report using artificial intelligence, a means for acquiring user emotional data and adjusting the presentation method of the forest management strategy based on the data, a means for sending the created forest management strategy to the user's terminal and displaying it, and a means for receiving the user's execution results again and evaluating the effectiveness of the forest management strategy. This allows the server to provide an optimal forest management strategy while taking the user's emotional state into consideration, enabling even novice forestry managers to achieve sustainable forest management.

[0326] "User" refers to any person or entity that uses the system, and includes, in particular, novice foresters.

[0327] "Image data and video data" is visual information showing the state of the forest captured by a user using a smartphone or camera.

[0328] "Receiving" refers to the act of the server taking in data sent by the user.

[0329] "Analyzing" refers to the act of digitally processing received data using specific algorithms to extract useful information.

[0330] "Tree type, density, growth stage, and presence of pests and diseases" refer to various forest conditions and risk factors identified from the analyzed image and video data.

[0331] A "weather database" is an information system that stores regional climate information and is accessed through an API.

[0332] A "Forest Assessment Report" is a document that integrates analyzed information and meteorological data to assess the health and risks of a forest.

[0333] "Generative artificial intelligence" is a machine learning model or generative AI model designed to derive optimal solutions based on input data.

[0334] The "Forest Management Strategy" is a specific action plan for achieving sustainable forest management, created based on the assessment report.

[0335] "Emotion data" is information that indicates the user's psychological state, obtained from the user's facial expressions and voice.

[0336] "Terminal" refers to a device used by a user, and includes general computing devices such as smartphones, tablets, or personal computers.

[0337] "Adjusting the presentation method" means appropriately changing the content of the suggestion and its expression depending on the user's emotional state.

[0338] "Execution results" refer to the activities and deliverables that users actually perform based on the proposed management strategy.

[0339] "Evaluating effectiveness" is the act of assessing the extent to which the implementation results achieved the goals of the proposed management strategy.

[0340] The present invention relates to a system that allows users to understand the current state of their forests and generate and present optimal forest management strategies. This system provides support, particularly for novice forestry managers, to achieve sustainable forest management.

[0341] The components of the system and their operation are described below.

[0342] 1. User's device

[0343] Photography function: The user uses a smartphone or camera to capture image data and video data of the forest. For example, the user can take multiple photos of a mountain slope and shoot videos from various directions.

[0344] Data upload function: The user's device uploads the captured data to the system via a dedicated application. For example, by pressing the upload button in the app, images and videos are sent to the server.

[0345] Emotion recognition function: The user's device is equipped with sensors and functions for facial recognition and voice analysis, which detect the user's emotional state in real time and send it to the server. Facial recognition technology uses common face detection algorithms (e.g., EmoReact) and voice analysis technology (e.g., AWS Transcribe).

[0346] 2. Server

[0347] Data reception and storage function: The server receives image data, video data, and emotion data sent by the user and safely stores them in a database (e.g., MySQL).

[0348] Analysis features:

[0349] The server passes the received data to an image analysis algorithm (e.g., OpenCV) to identify tree species, density, growth stage, and the presence of pests and diseases.

[0350] The server obtains climate information for the target area from a weather database via an API (e.g., OpenWeatherMap API).

[0351] Assessment report generation function: The server integrates image analysis results with meteorological data to generate assessment reports on forest health and risks, including tree health, growth status, and pest and disease risk assessments.

[0352] Strategy planning function using generative AI: The server uses generative AI (e.g., GPT-4) to create optimal forest management strategies based on the evaluation report. The generated strategies include, for example, felling plans, planting plans, and maintenance timing.

[0353] Emotion visualization and strategy adjustment: The server recognizes the user's emotional state in real time and adjusts the content and method of management strategy presentation. For example, if the user is feeling anxious, it provides encouraging messages and simplified instructions.

[0354] Strategy notification function: The server sends the created management strategy to the user's terminal and presents it to the user through a visual interface.

[0355] Feedback analysis function: The server receives the execution results from the user again, evaluates the effectiveness of the strategy, and adjusts the strategy if necessary.

[0356] Specific examples

[0357] Below are some examples of specific prompt sentences.

[0358] "Analyze images and videos of a forest taken by a user and suggest optimal management strategies. This user is currently experiencing some anxiety."

[0359] This allows even novice forest managers to easily carry out sustainable forest management while taking into account emotional states. The introduction of the emotion engine is expected to improve user acceptance and lead to more effective management.

[0360] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0361] Step 1: Uploading image and video data

[0362] Users use their smartphones or cameras to capture images and video data of forests, for example, taking multiple photos and videos to cover a specific mountain slope or tree density.

[0363] Input: Forest image data and video data

[0364] The device uploads the captured data to the system via a dedicated application. When the user presses the upload button in the application, the data is sent to the server.

[0365] Output: Data is sent to the server

[0366] Step 2: Collecting emotion data

[0367] The device uses its built-in camera and microphone to analyze the user's facial expressions and voice in real time. The camera uses facial recognition technology (e.g., EmoReact), and the microphone uses voice analysis technology (e.g., AWS Transcribe).

[0368] Input: User's facial expression data and voice data

[0369] The terminal transmits the acquired emotion data to the server.

[0370] Output: Emotion data is sent to the server

[0371] Step 3: Receiving and storing data

[0372] The server receives the image data, video data, and emotion data sent from the device using a security protocol (e.g., HTTPS).

[0373] Input: Image data, video data, and emotion data sent from the device

[0374] The server securely stores the received data in a database (e.g. MySQL).

[0375] Output: Data stored in the database

[0376] Step 4: Analyze the data

[0377] The server passes the stored image and video data to an image analysis algorithm (e.g., OpenCV) to identify tree species, density, growth stage, and the presence of pests and diseases.

[0378] Input: Stored image and video data

[0379] The server uses an API (e.g., OpenWeatherMap API) to obtain climate information for the target area from a weather database.

[0380] Output: Tree species, density, growth stage, and pest and disease presence information, and acquired climate information

[0381] Step 5: Integrating evaluation reports and sentiment analysis

[0382] The server combines the image analysis results with meteorological data to generate an assessment report on forest health and risk, including tree health, growth status, and pest and disease risk assessment.

[0383] Input: Image analysis results, weather data, emotion data

[0384] The server also analyzes the user's emotional data and integrates this data.

[0385] Output: Consolidated evaluation report

[0386] Step 6: Generate a control strategy

[0387] Based on the evaluation report, the server uses generative AI (e.g., GPT-4) to create an optimal forest management strategy, including, for example, a felling plan, a planting plan, and the timing of maintenance.

[0388] Input: Consolidated Assessment Report

[0389] The server runs the generative AI model and generates a management strategy.

[0390] Output: Optimal forest management strategy

[0391] Step 7: Adjust your strategy based on your emotional state

[0392] The server takes into account the user's emotional state and adjusts the content and presentation of the management strategies it presents to them—for example, if the user is feeling anxious, it presents strategies that include encouraging messages.

[0393] Input: optimal forest management strategy, user's emotional state

[0394] The server adjusts how the strategy is presented based on the emotion data.

[0395] Output: Coordinated management strategy

[0396] Step 8: Inform and implement your control strategy

[0397] The server transmits the created management strategy to the user's terminal, allowing the user to check the strategy through a visual interface.

[0398] Input: Coordinated management strategies

[0399] The server uses the notification function to send the strategy to the user's terminal.

[0400] Output: The control strategy displayed to the user

[0401] Step 9: Feedback on execution results

[0402] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[0403] Input: Results of forest management operations

[0404] The user sends the result data from the terminal to the system.

[0405] Output: Execution results sent to the server

[0406] Step 10: Evaluate and recalibrate your strategy

[0407] The server again receives the execution result data from the user, evaluates the effectiveness of the strategy, and readjusts the strategy as necessary based on the evaluation results.

[0408] Input: Execution result data

[0409] The server analyzes again and makes the next proposal.

[0410] Output: Retuned management strategy

[0411] Step 11: Notification of next proposal

[0412] The server notifies the user's terminal of the updated strategy and the next proposal, allowing the user to continuously carry out appropriate forest management.

[0413] Input: Recalibrated management strategies

[0414] The server will notify the user of the next offer.

[0415] Output: Next suggestion notified to the user

[0416] (Application example 2)

[0417] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0418] Operational management at a logistics center requires efficient placement and movement of goods and work procedures, which requires a great deal of effort and experience. Furthermore, worker emotions and fatigue levels have a significant impact on productivity and work efficiency, but current systems make it difficult to manage and adjust these factors. Therefore, there is a need for a system that can accurately grasp the location, type, and quantity of goods and automatically propose optimal work procedures.

[0419] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for receiving image data and video data acquired from a user, a means for analyzing the received image data and video data to identify the type, location, quantity, and presence of obstacles of items, and a means for generating an evaluation report based on the analyzed information and environmental data acquired from an external database. This makes it possible to accurately grasp the location, type, and quantity of items in a logistics center and automatically develop and present optimal operation and management strategies. Furthermore, by recognizing the user's emotional state in real time and adjusting the strategy presentation method, worker stress and fatigue can be reduced, improving work efficiency.

[0420] "Users" are the workers and managers who use the system to manage the operation of the logistics center.

[0421] "Image data and video data" refers to data recorded in a visually identifiable format that shows the items, their locations, and their status within a logistics center.

[0422] The "receiving means" is a function for transmitting image data and video data captured by the user to the server and capturing the data.

[0423] "Type of goods" is information indicating the category or classification of products or items present in the logistics center.

[0424] "Position" is coordinate information that indicates the location of an item or an obstacle within a logistics center.

[0425] "Quantity" is information indicating how many of a particular type of item are present in the logistics center.

[0426] An "obstacle" is any material or structure that may affect the movement of goods or the efficiency of work within a logistics center.

[0427] "Means for analyzing" refers to algorithms or software that processes received image and video data to identify the type, location, and quantity of items, and the presence of obstacles.

[0428] An "external database" is a database that exists outside the distribution center and can be accessed and used by the server.

[0429] "Environmental data" refers to environmental information that may affect the operation of a logistics center, such as temperature, humidity, and lighting conditions inside and outside the logistics center.

[0430] An "assessment report" is a report that shows the current status and risks of a logistics center, generated based on analyzed information and environmental data.

[0431] "Generative AI" is a system that uses artificial intelligence technology to create optimal operational management strategies based on evaluation reports.

[0432] "Operational management strategy" refers to specific work procedures and plans for achieving efficient operations within a logistics center.

[0433] The "means for transmitting and displaying" is a function for transmitting the created operation management strategy to the user's terminal and providing an interface for visually confirming the strategy.

[0434] The "execution results" are data indicating the status and deliverables after the user has performed the work.

[0435] The "means for evaluating effectiveness" is a function for analyzing and evaluating the effectiveness of the operation management strategy based on the user's execution results.

[0436] "Emotional state" is information that recognizes the psychological state and emotions of workers in real time.

[0437] The "means for adjusting the strategy presentation method" is a function for changing the content and presentation method of the presented operation management strategy according to the recognized emotional state of the worker.

[0438] To implement the present invention, each part of the system functions in the following steps.

[0439] 1. User's device

[0440] Users primarily use smartphones and tablets. These devices must have the following features:

[0441] Photography function: The user uses a camera to capture images of the situation inside the logistics center. Using this camera, the location and status of items are recorded as image and video data.

[0442] Data upload function: A dedicated application is used to upload captured image and video data to the server. Data can be sent quickly through this application.

[0443] Emotion recognition: Detects the user's emotional state in real time using the camera and microphone, using facial recognition and voice analysis technologies.

[0444] 2. Server

[0445] The server has the following features:

[0446] Data reception and storage function: The server receives image data, video data, and emotion data sent by users and stores them securely in a database. Storing this data is important for subsequent analysis and evaluation.

[0447] Image and video analysis function: The server is equipped with image analysis algorithms to analyze the received data, thereby identifying the type, location, and quantity of items within the logistics center, as well as the presence of obstacles.

[0448] Environmental data acquisition function: The server acquires environmental data from an external database, including temperature, humidity, lighting conditions, etc.

[0449] Assessment report generation function: The server integrates the analysis results and environmental data to generate an assessment report on the current status of the logistics center. This report clarifies the operational status and risks of the logistics center.

[0450] Strategy planning function using generative AI models: The server uses generative AI based on the evaluation report to automatically create optimal operation and management strategies, including material movement plans and work procedures.

[0451] Emotion-based strategy adjustment function: The server recognizes the user's emotional state and adjusts the way strategies are presented, thereby reducing stress and fatigue for workers and promoting efficient work.

[0452] 3. Specific examples of applications

[0453] Consider a case where this system is used to optimize the operation and management of a specific area of ​​a logistics center. A user takes a picture of the specific area using a smartphone and uploads it to a server using a dedicated application.

[0454] Example prompt sentence:

[0455] Take an image of a specific area in your warehouse and run a query to generate an optimal management strategy. Use the following image path:

[0456] Image path: 'warehouse_section.jpg'

[0457] The generative AI model suggests optimal work procedures based on the location, type, and quantity of items, as well as the emotional state of the worker."

[0458] The server analyzes the received image data to identify the location, type, and quantity of items, as well as the presence of obstacles. An assessment report is generated based on the analysis results and environmental data, and an optimal operational management strategy is then created using a generative AI model. Furthermore, the system takes into account the emotional state of the worker and adjusts the strategy presentation method to improve work efficiency and reduce stress.

[0459] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0460] Step 1:

[0461] Users take pictures of the situation inside the logistics center using a smartphone or tablet device. The captured image and video data are uploaded to the server via a dedicated application. The input here is image and video data, and the output is data sent to the server.

[0462] Step 2:

[0463] The server receives image and video data sent by users and stores them securely in a database. The input is the data received from the user, and the output is storing it in the database. This involves specific operations to verify the consistency and safety of the data.

[0464] Step 3:

[0465] The server analyzes the stored image and video data. It uses image analysis algorithms to identify the type, location, and quantity of objects, as well as the presence of obstacles. The input is image and video data, and the output is the analysis results, such as the type, location, and quantity of objects, and the presence of obstacles. Specific operations include image processing using libraries such as OpenCV.

[0466] Step 4:

[0467] The server retrieves environmental data (temperature, humidity, lighting conditions, etc.) inside and outside the distribution center from an external database. The input is a query to the external database, and the output is the retrieved environmental data. This step includes specific operations to retrieve data, such as through API calls.

[0468] Step 5:

[0469] The server integrates the analysis results and the acquired environmental data to generate an assessment report. The input is the analysis results of the item and the environmental data, and the output is the assessment report. Specific operations include data integration and report generation algorithms.

[0470] Step 6:

[0471] The server uses a generative AI model based on the evaluation report to create an optimal operations management strategy. This strategy includes a material movement plan and work procedures. The input is the evaluation report, and the output is the operations management strategy. Specific operations include a strategy planning process using the generative AI model.

[0472] Step 7:

[0473] The server obtains emotional data from the user's device in real time and adjusts the strategy to be presented. The input is emotional data, and the output is an operational management strategy adjusted based on the emotion. Specific operations include obtaining emotional data through facial recognition and voice analysis and adjusting the strategy.

[0474] Step 8:

[0475] The server sends the created operation management strategy to the user's terminal and displays it through a visual interface. The input is the operation management strategy, and the output is the strategy information displayed on the user's terminal. Specific operations include data transmission and user interface display.

[0476] Step 9:

[0477] The user performs specific tasks based on the presented operation management strategy. The input is the operation management strategy, and the output is the work results. Specific actions include moving and organizing items within the warehouse and executing work procedures.

[0478] Step 10:

[0479] The user reports the results of their work to the server through a dedicated application. The input is the data of the work result, and the output is the transmission of data to the server. This includes uploading images and reports after the work.

[0480] Step 11:

[0481] The server receives and analyzes the execution result data sent by the user, thereby evaluating the effectiveness of the operation management strategy and readjusting the strategy as necessary. The input is the execution result data, and the output is an updated operation management strategy. Specific operations include data analysis and strategy readjustment.

[0482] Step 12:

[0483] The server notifies the user terminal of the updated strategy and the next proposal. The input is the updated management strategy, and the output is the notification information. Specific operations include the execution of the notification mechanism.

[0484] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0485] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0486] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0487] [Second embodiment]

[0488] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0489] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0490] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0491] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0492] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0493] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0494] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0495] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0496] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0497] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0498] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0499] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0500] The present invention relates to a system that receives image and video data acquired from users, analyzes them to evaluate the current state of a forest, and generates and presents an optimal forest management strategy. This system provides support, especially for novice forestry managers, to achieve sustainable forest management.

[0501] System Overview

[0502] The system receives image and video data taken by users of their own mountains and analyzes it to identify forest species, density, growth stage, and the presence of pests and diseases. It also collects meteorological data to generate a comprehensive assessment report and uses AI to develop an optimal forest management strategy. The strategy is then sent to the user's device, and the user can carry out specific tasks based on it, with the results then fed back into the system.

[0503] System configuration

[0504] 1. User's device

[0505] Photography function: Users can use their smartphones or cameras to capture image and video data of the forest.

[0506] Data upload function: Upload captured data to the system via a dedicated application.

[0507] 2. Server

[0508] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database.

[0509] Analysis features:

[0510] Image analysis algorithms are used to identify tree species, density, growth stage, and the presence of pests and diseases.

[0511] Use the API to obtain local climate information from a weather database.

[0512] Assessment report generation function: Integrates analyzed information with meteorological data to generate assessment reports on forest health and risks.

[0513] Strategic planning function using generative AI: Automatically creates optimal forest management strategies based on assessment reports.

[0514] Strategy notification function: The created management strategy is sent to the user's terminal and presented to the user in a visual interface.

[0515] Feedback analysis function: Receives execution results from users, reevaluates the effectiveness of the strategy, and adjusts it as necessary.

[0516] Program processing

[0517] Below, we will explain in detail an example of how the system's programs are used and the processing flow.

[0518] 1. Uploading image and video data

[0519] To understand the current state of their mountain, users take multiple photos and videos using their smartphone or camera and upload them to the system via the application.

[0520] 2. Receipt and storage of data

[0521] The server receives the data uploaded by the user and stores it securely in a database.

[0522] 3. Data Analysis

[0523] The server passes the received data through image analysis algorithms to identify tree species, density, growth stage and the presence of pests and diseases.

[0524] The server also retrieves climate information for the target area from a weather database via an API.

[0525] 4. Generate an evaluation report

[0526] The server integrates image analysis results with meteorological data to generate assessment reports on forest health and risks.

[0527] 5. Generating a management strategy

[0528] Based on the assessment report, the server uses generative AI to create an optimal forest management strategy, including a felling plan, a planting plan, and the timing of maintenance.

[0529] 6. Informing and implementing control strategies

[0530] The server transmits the created management strategy to the user's terminal and presents it to the user through a visual interface.

[0531] The user carries out specific tasks based on the presented strategies.

[0532] 7. Feedback on execution results

[0533] The user reports the results of their work to the system through the application, which includes photos of the work and a status report.

[0534] 8. Evaluate and recalibrate your strategy

[0535] The server receives the execution results from the user, evaluates the effectiveness of the strategy, adjusts the strategy if necessary, and makes next suggestions.

[0536] This system makes it easy for even new forest owners to achieve sustainable forest management. By continuing to implement appropriate forest management through specific instructions and feedback, environmentally friendly forest management becomes possible.

[0537] The processing flow will be explained below.

[0538] Step 1:

[0539] Users take pictures of their mountain using a smartphone or camera. In order to collect sufficient information, they take multiple images and videos from various angles.

[0540] Step 2:

[0541] The user launches a dedicated application and uploads the captured image and video data to the application, which then transmits the data to the system.

[0542] Step 3:

[0543] The server receives the image and video data sent by the user and stores the received data securely in a database.

[0544] Step 4:

[0545] The server then passes the received data to an image analysis algorithm that uses computer vision technology to identify tree species, density, growth stage, and the presence of pests and diseases.

[0546] Step 5:

[0547] The server connects to a weather database and retrieves local weather information through an API, including past weather conditions and future forecast data.

[0548] Step 6:

[0549] The server combines image analysis results with meteorological data to generate forest health and risk assessment reports, which contain detailed information about the condition of the forest.

[0550] Step 7:

[0551] The server uses generative AI to create an optimal forest management strategy based on the assessment report, including optimal harvesting plans, planting plans, and maintenance timing.

[0552] Step 8:

[0553] The server then sends the created forest management strategy to the user's device, where the strategy can be viewed through a visual interface.

[0554] Step 9:

[0555] Users are then presented with a management strategy on their device, which involves carrying out specific tasks, such as cutting trees, planting trees, and tending them at set times.

[0556] Step 10:

[0557] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[0558] Step 11:

[0559] The server again receives and analyzes the execution result data from the user, thereby evaluating the effectiveness of the strategy and readjusting it if necessary.

[0560] Step 12:

[0561] The server notifies the user's device of updated strategies and next proposals, allowing the user to continuously implement appropriate forest management.

[0562] In this way, the system provides continuous support to users in achieving sustainable forest management.

[0563] Example 1

[0564] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0565] Traditional forest management requires specialized knowledge and experience, making it difficult for novice forest managers. Accurately assessing forest health and risks and formulating appropriate management strategies requires advanced technology and a great deal of time and effort. Furthermore, it is difficult to effectively utilize meteorological data, making it difficult to develop optimal felling and planting plans. This makes it difficult to achieve sustainable forest management.

[0566] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0567] In this invention, the server includes means for receiving image data and video data of a forest captured by a user, means for analyzing the received image data and video data using an image analysis algorithm to identify the forest type, density, growth stage, and presence of pests and diseases, means for generating a forest assessment report based on the analyzed information and meteorological data obtained from an external database, means for creating an optimal forest management strategy using a generative AI model based on the assessment report, means for sending the created forest management strategy to the user's device and displaying it in a visual interface, and means for receiving the user's execution results again and evaluating the effectiveness of the forest management strategy. This enables sustainable forest management even for novice foresters without advanced expertise.

[0568] A "user" is a person who uses the system to provide image and video data related to forest management, and receives and implements analysis results and management strategies.

[0569] "Image and video data" refers to photographs and video information taken by users of the current state of the forest, and is used to identify the type and density of trees, their growth stage, and the presence of pests and diseases.

[0570] The "means for receiving" refers to the process and mechanism by which the server acquires and stores image data and video data uploaded by users.

[0571] "Image analysis algorithm" means a mathematical and technical method for analyzing received image and video data to identify tree species, density, growth stage, and the presence of pests and diseases.

[0572] "Weather data" refers to local climate information obtained from external databases and is data that is integrated when generating forest assessment reports.

[0573] The "assessment report" is a report that evaluates the health and risks of a forest based on image analysis results and meteorological data.

[0574] A "generative AI model" is an artificial intelligence model that automatically creates optimal forest management strategies based on assessment reports.

[0575] A "forest management strategy" is a specific plan for managing forests sustainably, including optimal felling plans, planting plans, and maintenance timing.

[0576] The "visual interface" refers to the screen display and operation means that allow the server to display the forest management strategy generated by the server in an easy-to-understand manner for the user.

[0577] "Execution results" are the results of work carried out by the user based on the forest management strategy presented, and are re-evaluated.

[0578] The "means for evaluating effectiveness" refers to the process and techniques by which the server receives the execution results from the user and re-evaluates the effectiveness of the existing forest management strategy.

[0579] The present invention relates to a system that receives image and video data captured by users, analyzes them to evaluate the current state of a forest, and generates and presents an optimal forest management strategy. This system provides support, especially for novice foresters, to achieve sustainable forest management.

[0580] System Overview

[0581] The system receives image and video data taken by users of their own mountains and analyzes it to identify forest species, density, growth stage, and the presence of pests and diseases. It also collects meteorological data to generate a comprehensive assessment report and uses a generative AI model to develop an optimal forest management strategy. The strategy is then sent to the user's device, and the user can carry out specific tasks based on it, with the results then fed back into the system.

[0582] Hardware and Software Configuration

[0583] 1. User's device

[0584] Photography function: Users can use their smartphones or digital cameras to capture image and video data of the forest.

[0585] Data upload function: Upload captured data to the system through a dedicated application (e.g., "ForestCare" or "TreeHealth").

[0586] 2. Server

[0587] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database (e.g., MySQL, MongoDB).

[0588] Image analysis algorithms: Image analysis algorithms (e.g., TensorFlow, PyTorch) are used to identify tree species, density, growth stage, and the presence of pests and diseases.

[0589] Weather data acquisition: Use an API (e.g., OpenWeatherMap API) to obtain local climate information from a weather database.

[0590] Assessment report generation: Use Jupyter Notebook and Pandas to integrate the analyzed information with meteorological data and generate assessment reports on forest health and risks.

[0591] Strategy planning using generative AI: Based on the evaluation report, a generative AI model (e.g., GPT-4) is used to create an optimal forest management strategy.

[0592] Strategy notification: The created management strategy is sent to the user's terminal and presented to the user in a visual interface (e.g., a web application).

[0593] Feedback analysis: Receive execution results from users, reassess the effectiveness of the strategy, and readjust the strategy if necessary.

[0594] Specific examples

[0595] For example, a novice forestry manager might follow these steps to understand the current state of his or her mountain:

[0596] 1. Capture and upload data:

[0597] Users use their smartphones to take photos and videos of the mountain from multiple angles.

[0598] Launch the dedicated app "ForestCare" and upload the captured data to the system.

[0599] 2. Data analysis and evaluation:

[0600] The server passes the received data to a TensorFlow image analysis algorithm, which identifies tree species, density, growth stage, and the presence of pests and diseases.

[0601] Obtain weather data for the target area from OpenWeatherMap via API.

[0602] 3. Generate an assessment report:

[0603] The server integrates the image analysis results and meteorological data using Pandas and creates an evaluation report.

[0604] Create a report in Jupyter Notebook and save it as a PDF.

[0605] 4. Generate a management strategy:

[0606] The server uses a generative AI model (GPT-4) to input the following prompt based on the evaluation report: "Please tell me the appropriate tree-cutting and planting plan for this area."

[0607] Format the strategy proposed by GPT-4 and notify the user.

[0608] 5. Notice and Execution:

[0609] The server pushes the created management strategy to the user's app.

[0610] Users can check the strategy through the app and carry out the tasks presented.

[0611] 6. Feedback and reassessment of results:

[0612] Users report their work results to the app and upload photos and status reports.

[0613] The server receives and analyzes the reports and readjusts the strategy as needed.

[0614] In this way, this system enables even novice foresters to achieve sustainable forest management without specialized knowledge.

[0615] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0616] Step 1:

[0617] To understand the current state of their mountain, users take image and video data of the forest using a smartphone or digital camera. The input is the captured image and video data, and the output is the saving of this data on the user's device. Specifically, users take photos of the overall view from multiple angles, the condition of nearby trees, and the state of damage caused by pests and diseases.

[0618] Step 2:

[0619] The user launches a dedicated mobile application (e.g., "ForestCare" or "TreeHealth") and uploads the captured data to the system. The input is image and video data stored on the user's device, and the output is the data being sent to the system's server. Specifically, the user taps the "Upload Data" button on the application, selects the captured data, and uploads it.

[0620] Step 3:

[0621] The server immediately receives image and video data uploaded by users and securely stores it in a database (e.g., MySQL, MongoDB). The input is the image and video data received from users, and the output is the data stored in the database. Specifically, the server receives the data uploaded via an HTTP request and records it in the database.

[0622] Step 4:

[0623] The server passes the received image and video data to an image analysis algorithm (e.g., TensorFlow, PyTorch) and performs the analysis. The input is the image and video data stored in the database, and the output is the analysis results regarding the tree species, density, growth stage, and the presence of pests and diseases. Specifically, the server inputs the data into the image analysis algorithm, extracts specific features, and evaluates them.

[0624] Step 5:

[0625] The server obtains the analysis results and uses an API (e.g., OpenWeatherMap API) to obtain weather data for the target area. The input is the analysis results and information about the target area, and the output is the obtained weather data. Specifically, the server sends a request to the API endpoint to obtain the weather data.

[0626] Step 6:

[0627] The server integrates the image analysis results with meteorological data to generate an assessment report on forest health and risks. The input is the image analysis results and meteorological data, and the output is an assessment report. Specifically, the server integrates the analysis results and meteorological data using Pandas, creates an assessment report in Jupyter Notebook, and saves it in PDF format.

[0628] Step 7:

[0629] The server uses a generative AI model (e.g., GPT-4) to create an optimal forest management strategy based on the evaluation report. The input is the evaluation report and a prompt (e.g., "Please tell me the appropriate logging and planting plan for this area."), and the output is the generated forest management strategy. Specifically, the server inputs the evaluation report as a prompt to GPT-4 and obtains the generated strategy.

[0630] Step 8:

[0631] The server sends the created forest management strategy to the user's device and presents it to the user through a visual interface. The input is the created forest management strategy, and the output is the strategy displayed on the user's device. Specifically, the server pushes the strategy to the user's app in JSON format, and the strategy is displayed visually on the application.

[0632] Step 9:

[0633] The user carries out specific tasks based on the presented forest management strategy. The input is the forest management strategy displayed on the user's terminal, and the output is the results of the executed tasks. As specific actions, the user performs tasks such as cutting trees, planting, and maintenance according to the strategy.

[0634] Step 10:

[0635] Users report the results of their work through the application and upload photos and status reports after the work is completed to the system. The input is data about the work performed and its results, and the output is feedback sent to the system. Specifically, users upload photos and reports using the app's "Work Completion Report" function.

[0636] Step 11:

[0637] The server receives feedback from the user, analyzes it again, and evaluates the effectiveness of the forest management strategy. It readjusts the strategy as needed and presents the next proposal to the user. The input is the received feedback data, and the output is the evaluated effectiveness of the strategy and the readjusted management strategy. Specifically, the server passes the feedback data to an analysis algorithm for evaluation, and if necessary, uses a generative AI model to generate a new strategy.

[0638] This process allows even novice foresters to achieve sustainable forest management. The process of specific instructions and feedback is repeated, allowing for more precise management.

[0639] (Application example 1)

[0640] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0641] Maintaining and managing factory equipment and production lines requires a great deal of time and effort, and conventional methods have the problem of making it difficult to detect abnormalities early and formulate appropriate maintenance plans.The present invention aims to improve the operational efficiency of equipment and reduce unexpected downtime by efficiently evaluating the health of factory equipment and accurately proposing necessary maintenance.

[0642] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0643] In this invention, the server includes means for receiving image data and video data acquired from a user, means for analyzing the received image data and video data to identify the type of equipment, its operating status, and the presence or absence of abnormalities, and means for generating an equipment evaluation report based on the analyzed information and environmental data acquired from the manufacturing environment database. This makes it possible to evaluate the health of the equipment in real time and present optimal maintenance plans, replacement plans, and maintenance timing.

[0644] The "means for receiving image data and video data acquired from the user" is a function for transmitting image data and video data captured by the user to a server via the Internet and receiving the data.

[0645] "Means for analyzing received image data and video data to identify the type of equipment, its operating status, and whether or not there are any abnormalities" refers to a function that analyzes received image data and video data using computer vision technology and machine learning algorithms, etc., to identify the type of specific equipment, its operating status, and signs of abnormalities.

[0646] "Means for generating an equipment evaluation report based on the analyzed information and environmental data obtained from the manufacturing environment database" refers to a function for automatically generating an evaluation report by combining the analysis results with various environmental data (e.g., temperature, humidity, etc.) collected in advance.

[0647] "A means for creating an optimal asset management strategy using generative AI based on an evaluation report" is a function that utilizes a generative AI model to automatically consider and create optimal maintenance plans and strategies from the generated evaluation report.

[0648] "Means for sending and displaying the created facility management strategy to the user's device" refers to a function for sending the generated optimal management strategy to the user's smartphone, computer, etc., and displaying it in an intuitive interface.

[0649] "Means of receiving the user's execution results again and evaluating the effectiveness of the asset management strategy" refers to a function that receives the results of the maintenance work and maintenance that has been carried out again as feedback to the system, evaluates the results, and reflects them in future strategies.

[0650] "Equipment management strategy including optimal maintenance plan, replacement plan and maintenance timing" is a management strategy including the most appropriate maintenance work schedule, part replacement timing and maintenance method for maintaining the health of equipment.

[0651] "Facility management" refers to the planned maintenance, repair, and replacement work required to operate the equipment and machinery used in factories and production lines efficiently and effectively.

[0652] The present invention relates to a system that efficiently evaluates the health of equipment and manufacturing lines in a factory and proposes optimal maintenance plans and strategies. This system provides significant support, especially to factory managers who find equipment management difficult. The system is realized by the following procedure.

[0653] System configuration

[0654] 1. User's device

[0655] Photography function: Users can use their smartphones or cameras to capture image and video data of equipment and production lines.

[0656] Data upload function: Upload captured data to a server via a dedicated application.

[0657] 2. Server

[0658] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database.

[0659] Analysis features:

[0660] Image analysis algorithms are used to identify the type of equipment, its operating status, and whether or not there are any abnormalities.

[0661] Use the API to obtain environmental data from the manufacturing environment database.

[0662] Assessment report generation function: Integrates analyzed information with environmental data to generate assessment reports on the health and risk of facilities.

[0663] Generative AI strategy planning function: Based on the assessment report, generates an asset management strategy including optimal maintenance plans, replacement plans, and maintenance timing.

[0664] Strategy notification function: The created facility management strategy is sent to the user's device and presented to the user via a visual interface.

[0665] Feedback analysis function: Receives execution results from users, reevaluates the effectiveness of the strategy, and adjusts it as necessary.

[0666] Program processing explanation

[0667] Hardware:

[0668] Smartphones: Used as photography devices by factory robots.

[0669] Server: Cloud server (e.g. AWS, Google Cloud) for data analysis and management strategy generation.

[0670] software:

[0671] OpenCV: Used for image and video capture.

[0672] TensorFlow: Image analysis algorithms for anomaly detection and analysis.

[0673] REST API: Used for data upload and notifications (e.g. FastAPI).

[0674] Examples:

[0675] As an example, consider the case where the motor of equipment A is overheating. A user uses a smartphone to take pictures of equipment A and uploads the images and videos to the system. The server receives these and analyzes them using TensorFlow. Based on the analysis results, the generation AI creates an optimal maintenance plan and notifies the user again on their device. In this case, the following can be used as an example of a prompt text:

[0676] Example prompt sentence:

[0677] "Equipment A has been operating for a long time recently, and the motor temperature is rising sharply. Please detect this situation and generate an optimal maintenance plan."

[0678] The system helps reduce unplanned downtime while ensuring facility safety and efficiency.

[0679] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0680] Step 1:

[0681] Users use smartphones or cameras to take images and videos of the equipment and production lines in the factory. These become the input data. Specifically, images are taken from multiple viewpoints to record the equipment's condition in detail.

[0682] Step 2:

[0683] The user's device uploads the captured image and video data to the server via a dedicated application. The input data is the image and video data captured in the previous step, and the output is the completion of uploading to the server. Specifically, the data is sent using a data transfer protocol (e.g., HTTP).

[0684] Step 3:

[0685] The server receives uploaded image and video data and stores it securely in a database. The input data is the data uploaded by the user, and the output is the data that has been saved. Specifically, the received data is converted into an appropriate format and stored in the database.

[0686] Step 4:

[0687] The server passes the received data to an image analysis algorithm (using, for example, OpenCV or TensorFlow) to identify the type of equipment, its operating status, and whether or not there are any abnormalities. The input data is the stored image and video data, and the output is the analysis results (type of equipment, operating status, and whether or not there are any abnormalities). Specifically, the algorithm scans the image and detects specific patterns and abnormalities.

[0688] Step 5:

[0689] The server obtains environmental data (e.g., temperature, humidity) from the manufacturing environment database through the API. The input data is a request to the environment database, and the output is the obtained environmental data. Specifically, it accesses the API endpoint and retrieves the required data.

[0690] Step 6:

[0691] The server integrates the image analysis results with the environmental data to generate an assessment report on the health and risk of the facility. The input data are the analysis results and environmental data, and the output is an assessment report. Specifically, it runs an assessment algorithm that integrates both sets of data and generates the assessment results in text and graph format.

[0692] Step 7:

[0693] The server uses a generative AI model based on the evaluation report to create an equipment management strategy that includes optimal maintenance plans, replacement plans, and maintenance timing. The input data is the evaluation report, and the output is the management strategy. Specifically, the server inputs prompts into the generative AI model to generate the optimal strategy.

[0694] Step 8:

[0695] The server sends the created facility management strategy to the user's device and presents it to the user in a visual interface. The input data is the management strategy, and the output is a notification to the user's device. Specifically, the server encodes the management strategy as a message and sends it to the user's application.

[0696] Step 9:

[0697] The user performs specific maintenance work based on the transmitted management strategy and reports the results back to the system. The input data is the implementation results, and the output is feedback data. Specific operations include recording the status after the maintenance work using images and text, and reporting it through the application.

[0698] Step 10:

[0699] The server receives execution results from the user, analyzes them, and reevaluates the effectiveness of the asset management strategy. If necessary, it adjusts the strategy and makes the next proposal. The input data is feedback data, and the output is a new strategy. Specifically, it analyzes the execution results, reflects any necessary modifications in the generative AI model, and creates a new management strategy.

[0700] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0701] This invention relates to a system that receives image and video data acquired from users, analyzes them to evaluate the current state of a forest, and generates and presents an optimal forest management strategy. Furthermore, the system aims to enhance the effectiveness of the management strategy by incorporating an emotion engine that recognizes the user's emotions. This system provides support, especially for novice foresters, to achieve sustainable forest management.

[0702] System Overview

[0703] The system receives image and video data taken by users of their own mountains and analyzes it to identify forest species, density, growth stage, and the presence of pests and diseases. It also collects weather data to generate a comprehensive assessment report and uses generative AI to develop optimal forest management strategies. Furthermore, by utilizing an emotion engine that recognizes the user's emotions, the system adjusts the way strategies are presented to the user based on their emotional state, enabling them to implement strategies in a way that is more receptive to the user.

[0704] System configuration

[0705] 1. User's device

[0706] Photography function: Users can use their smartphones or cameras to capture image and video data of the forest.

[0707] Data upload function: Upload captured data to the system via a dedicated application.

[0708] Emotion recognition: Equipped with sensors and cameras to recognize the user's emotional state.

[0709] 2. Server

[0710] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database.

[0711] Analysis features:

[0712] Image analysis algorithms are used to identify tree species, density, growth stage, and the presence of pests and diseases.

[0713] Use the API to obtain local climate information from a weather database.

[0714] Assessment report generation function: Integrates analyzed information with meteorological data to generate assessment reports on forest health and risks.

[0715] Strategic planning function using generative AI: Automatically creates optimal forest management strategies based on assessment reports.

[0716] Sentiment visualization and strategy adjustment features:

[0717] Recognize user emotions in real time and adjust the presentation and content of management strategies.

[0718] Strategy notification function: The created management strategy is sent to the user's terminal and presented to the user in a visual interface.

[0719] Feedback analysis function: Receiving execution results from users, evaluating the effectiveness of strategies and adjusting them as necessary.

[0720] Program processing

[0721] Below, we will explain in detail an example of how the system's programs are used and the processing flow.

[0722] 1. Uploading image and video data

[0723] To understand the current state of their mountain, users take multiple photos and videos using their smartphone or camera and upload them to the system via the application.

[0724] 2. Collecting Emotional Data

[0725] The user's device uses techniques such as facial recognition and voice analysis to detect the user's emotional state in real time and transmits that data to a server.

[0726] 3. Receipt and storage of data

[0727] The server receives the image data, video data, and emotion data sent by the user and safely stores them in a database.

[0728] 4. Data Analysis

[0729] The server passes the received data through image analysis algorithms to identify tree species, density, growth stage and the presence of pests and diseases.

[0730] The server also retrieves climate information for the target area from a weather database via an API.

[0731] 5. Integrating evaluation reports and sentiment analysis

[0732] The server integrates the image analysis results, meteorological data, and user emotional data to generate an assessment report on the health and risks of the forest.

[0733] 6. Generating a management strategy

[0734] Based on the assessment report, the server uses generative AI to create an optimal forest management strategy, which includes a felling plan, a planting plan, and the timing of maintenance.

[0735] 7. Adjusting strategies based on emotional state

[0736] The server takes into account the user's emotional state and adjusts the content and presentation of the management strategies it presents, for example, providing encouraging messages or simplified instructions if the user is feeling anxious.

[0737] 8. Informing and implementing control strategies

[0738] The server transmits the created management strategy to the user's terminal, allowing the user to check the management strategy through a visual interface.

[0739] Users are then presented with a management strategy on their device, which involves carrying out specific tasks, such as cutting trees, planting trees, and tending them at set times.

[0740] 9. Feedback on execution results

[0741] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[0742] 10. Evaluate and recalibrate your strategy

[0743] The server again receives and analyzes the execution result data from the user, thereby evaluating the effectiveness of the strategy and readjusting it if necessary.

[0744] 11. Notification of next proposal

[0745] The server notifies the user's device of updated strategies and next proposals, allowing the user to continuously implement appropriate forest management.

[0746] This system allows even novice foresters to easily achieve sustainable forest management while taking into account the emotional state of the foresters. The introduction of the emotion engine is expected to improve user acceptance and lead to more effective management.

[0747] The processing flow will be explained below.

[0748] Step 1:

[0749] Users take pictures of their mountain using a smartphone or camera, and capture multiple image and video data to capture the overall picture and detailed parts of the forest.

[0750] Step 2:

[0751] The user starts the dedicated application and uploads the captured image data and video data to the application.

[0752] Step 3:

[0753] The user's device analyzes the user's facial expressions and voice in real time to recognize their emotional state, using an emotion recognition engine to distinguish emotions such as joy, surprise, sadness, and anger.

[0754] Step 4:

[0755] The user's terminal transmits the emotional state data and the captured image and video data to the server.

[0756] Step 5:

[0757] The server receives the image and video data and emotional state data sent by the user, and stores the received data securely in a database.

[0758] Step 6:

[0759] The server analyzes the received image and video data using an image analysis algorithm, specifically extracting the following information:

[0760] Tree types

[0761] Tree density

[0762] Tree growth stages

[0763] Presence of pests and diseases

[0764] Step 7:

[0765] The server uses an API to retrieve local weather information from a weather database, including historical and forecast weather conditions.

[0766] Step 8:

[0767] The server generates an assessment report on forest health and risks based on the image analysis results and acquired climate data.

[0768] Step 9:

[0769] The server uses generative AI to create an optimal forest management strategy based on the assessment report, including harvesting plans, planting plans, and maintenance timing.

[0770] Step 10:

[0771] The server tailors the presentation and content of management strategies based on the user's emotional state: for example, if the user is feeling anxious, it offers encouraging messages or simplified instructions.

[0772] Step 11:

[0773] The server transmits the created management strategy to the user's terminal, allowing the user to check the management strategy on a visual interface.

[0774] Step 12:

[0775] The user is then presented with a management strategy and performs specific tasks, including cutting, planting, and tending at specified times.

[0776] Step 13:

[0777] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[0778] Step 14:

[0779] The server again receives and analyzes the execution result data from the user, thereby evaluating the effectiveness of the strategy and readjusting it if necessary.

[0780] Step 15:

[0781] The server notifies the user's device of updated strategies and next proposals, allowing the user to continuously implement appropriate forest management.

[0782] In this way, a system incorporating an emotion engine can provide optimal strategies according to the user's emotional state, enabling even beginners to achieve sustainable forest management.

[0783] Example 2

[0784] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0785] Conventional forest management systems could use user-provided image and video data to identify tree species, density, growth stage, and the presence of pests and diseases. However, they lacked the ability to present appropriate management strategies that take the user's emotional state into account. This made it difficult for novice forest managers to receive appropriate guidance and implement sustainable forest management. The present invention aims to solve this problem and provide a more comprehensive and effective forest management strategy.

[0786] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving image data and video data acquired from a user, a means for analyzing the received image data and video data to identify the tree species, density, growth stage, and presence of pests and diseases, a means for generating a forest assessment report based on the analyzed information and climate data acquired from a weather database, a means for creating an optimal forest management strategy based on the assessment report using artificial intelligence, a means for acquiring user emotional data and adjusting the presentation method of the forest management strategy based on the data, a means for sending the created forest management strategy to the user's terminal and displaying it, and a means for receiving the user's execution results again and evaluating the effectiveness of the forest management strategy. This allows the server to provide an optimal forest management strategy while taking the user's emotional state into consideration, enabling even novice forestry managers to achieve sustainable forest management.

[0787] "User" refers to any person or entity that uses the system, and includes, in particular, novice foresters.

[0788] "Image data and video data" is visual information showing the state of the forest captured by a user using a smartphone or camera.

[0789] "Receiving" refers to the act of the server taking in data sent by the user.

[0790] "Analyzing" refers to the act of digitally processing received data using specific algorithms to extract useful information.

[0791] "Tree type, density, growth stage, and presence of pests and diseases" refer to various forest conditions and risk factors identified from the analyzed image and video data.

[0792] A "weather database" is an information system that stores regional climate information and is accessed through an API.

[0793] A "Forest Assessment Report" is a document that integrates analyzed information and meteorological data to assess the health and risks of a forest.

[0794] "Generative artificial intelligence" is a machine learning model or generative AI model designed to derive optimal solutions based on input data.

[0795] The "Forest Management Strategy" is a specific action plan for achieving sustainable forest management, created based on the assessment report.

[0796] "Emotion data" is information that indicates the user's psychological state, obtained from the user's facial expressions and voice.

[0797] "Terminal" refers to a device used by a user, and includes general computing devices such as smartphones, tablets, or personal computers.

[0798] "Adjusting the presentation method" means appropriately changing the content of the suggestion and its expression depending on the user's emotional state.

[0799] "Execution results" refer to the activities and deliverables that users actually perform based on the proposed management strategy.

[0800] "Evaluating effectiveness" is the act of assessing the extent to which the implementation results achieved the goals of the proposed management strategy.

[0801] The present invention relates to a system that allows users to understand the current state of their forests and generate and present optimal forest management strategies. This system provides support, particularly for novice forestry managers, to achieve sustainable forest management.

[0802] The components of the system and their operation are described below.

[0803] 1. User's device

[0804] Photography function: The user uses a smartphone or camera to capture image data and video data of the forest. For example, the user can take multiple photos of a mountain slope and shoot videos from various directions.

[0805] Data upload function: The user's device uploads the captured data to the system via a dedicated application. For example, by pressing the upload button in the app, images and videos are sent to the server.

[0806] Emotion recognition function: The user's device is equipped with sensors and functions for facial recognition and voice analysis, which detect the user's emotional state in real time and send it to the server. Facial recognition technology uses common face detection algorithms (e.g., EmoReact) and voice analysis technology (e.g., AWS Transcribe).

[0807] 2. Server

[0808] Data reception and storage function: The server receives image data, video data, and emotion data sent by the user and safely stores them in a database (e.g., MySQL).

[0809] Analysis features:

[0810] The server passes the received data to an image analysis algorithm (e.g., OpenCV) to identify tree species, density, growth stage, and the presence of pests and diseases.

[0811] The server obtains climate information for the target area from a weather database via an API (e.g., OpenWeatherMap API).

[0812] Assessment report generation function: The server integrates image analysis results with meteorological data to generate assessment reports on forest health and risks, including tree health, growth status, and pest and disease risk assessments.

[0813] Strategy planning function using generative AI: The server uses generative AI (e.g., GPT-4) to create optimal forest management strategies based on the evaluation report. The generated strategies include, for example, felling plans, planting plans, and maintenance timing.

[0814] Emotion visualization and strategy adjustment: The server recognizes the user's emotional state in real time and adjusts the content and method of management strategy presentation. For example, if the user is feeling anxious, it provides encouraging messages and simplified instructions.

[0815] Strategy notification function: The server sends the created management strategy to the user's terminal and presents it to the user through a visual interface.

[0816] Feedback analysis function: The server receives the execution results from the user again, evaluates the effectiveness of the strategy, and adjusts the strategy if necessary.

[0817] Specific examples

[0818] Below are some examples of specific prompt sentences.

[0819] "Analyze images and videos of a forest taken by a user and suggest optimal management strategies. This user is currently experiencing some anxiety."

[0820] This allows even novice forest managers to easily carry out sustainable forest management while taking into account emotional states. The introduction of the emotion engine is expected to improve user acceptance and lead to more effective management.

[0821] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0822] Step 1: Uploading image and video data

[0823] Users use their smartphones or cameras to capture images and video data of forests, for example, taking multiple photos and videos to cover a specific mountain slope or tree density.

[0824] Input: Forest image data and video data

[0825] The device uploads the captured data to the system via a dedicated application. When the user presses the upload button in the application, the data is sent to the server.

[0826] Output: Data is sent to the server

[0827] Step 2: Collecting emotion data

[0828] The device uses its built-in camera and microphone to analyze the user's facial expressions and voice in real time. The camera uses facial recognition technology (e.g., EmoReact), and the microphone uses voice analysis technology (e.g., AWS Transcribe).

[0829] Input: User's facial expression data and voice data

[0830] The terminal transmits the acquired emotion data to the server.

[0831] Output: Emotion data is sent to the server

[0832] Step 3: Receiving and storing data

[0833] The server receives the image data, video data, and emotion data sent from the device using a security protocol (e.g., HTTPS).

[0834] Input: Image data, video data, and emotion data sent from the device

[0835] The server securely stores the received data in a database (e.g. MySQL).

[0836] Output: Data stored in the database

[0837] Step 4: Analyze the data

[0838] The server passes the stored image and video data to an image analysis algorithm (e.g., OpenCV) to identify tree species, density, growth stage, and the presence of pests and diseases.

[0839] Input: Stored image and video data

[0840] The server uses an API (e.g., OpenWeatherMap API) to obtain climate information for the target area from a weather database.

[0841] Output: Tree species, density, growth stage, and pest and disease presence information, and acquired climate information

[0842] Step 5: Integrating evaluation reports and sentiment analysis

[0843] The server combines the image analysis results with meteorological data to generate an assessment report on forest health and risk, including tree health, growth status, and pest and disease risk assessment.

[0844] Input: Image analysis results, weather data, emotion data

[0845] The server also analyzes the user's emotional data and integrates this data.

[0846] Output: Consolidated evaluation report

[0847] Step 6: Generate a control strategy

[0848] Based on the evaluation report, the server uses generative AI (e.g., GPT-4) to create an optimal forest management strategy, including, for example, a felling plan, a planting plan, and the timing of maintenance.

[0849] Input: Consolidated Assessment Report

[0850] The server runs the generative AI model and generates a management strategy.

[0851] Output: Optimal forest management strategy

[0852] Step 7: Adjust your strategy based on your emotional state

[0853] The server takes into account the user's emotional state and adjusts the content and presentation of the management strategies it presents to them—for example, if the user is feeling anxious, it presents strategies that include encouraging messages.

[0854] Input: optimal forest management strategy, user's emotional state

[0855] The server adjusts how the strategy is presented based on the emotion data.

[0856] Output: Coordinated management strategy

[0857] Step 8: Inform and implement your control strategy

[0858] The server transmits the created management strategy to the user's terminal, allowing the user to check the strategy through a visual interface.

[0859] Input: Coordinated management strategies

[0860] The server uses the notification function to send the strategy to the user's terminal.

[0861] Output: The control strategy displayed to the user

[0862] Step 9: Feedback on execution results

[0863] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[0864] Input: Results of forest management operations

[0865] The user sends the result data from the terminal to the system.

[0866] Output: Execution results sent to the server

[0867] Step 10: Evaluate and recalibrate your strategy

[0868] The server again receives the execution result data from the user, evaluates the effectiveness of the strategy, and readjusts the strategy as necessary based on the evaluation results.

[0869] Input: Execution result data

[0870] The server analyzes again and makes the next proposal.

[0871] Output: Retuned management strategy

[0872] Step 11: Notification of next proposal

[0873] The server notifies the user's terminal of the updated strategy and the next proposal, allowing the user to continuously carry out appropriate forest management.

[0874] Input: Recalibrated management strategies

[0875] The server will notify the user of the next offer.

[0876] Output: Next suggestion notified to the user

[0877] (Application example 2)

[0878] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0879] Operational management at a logistics center requires efficient placement and movement of goods and work procedures, which requires a great deal of effort and experience. Furthermore, worker emotions and fatigue levels have a significant impact on productivity and work efficiency, but current systems make it difficult to manage and adjust these factors. Therefore, there is a need for a system that can accurately grasp the location, type, and quantity of goods and automatically propose optimal work procedures.

[0880] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for receiving image data and video data acquired from a user, a means for analyzing the received image data and video data to identify the type, location, quantity, and presence of obstacles of items, and a means for generating an evaluation report based on the analyzed information and environmental data acquired from an external database. This makes it possible to accurately grasp the location, type, and quantity of items in a logistics center and automatically develop and present optimal operation and management strategies. Furthermore, by recognizing the user's emotional state in real time and adjusting the strategy presentation method, worker stress and fatigue can be reduced, improving work efficiency.

[0881] "Users" are the workers and managers who use the system to manage the operation of the logistics center.

[0882] "Image data and video data" refers to data recorded in a visually identifiable format that shows the items, their locations, and their status within a logistics center.

[0883] The "receiving means" is a function for transmitting image data and video data captured by the user to the server and capturing the data.

[0884] "Type of goods" is information indicating the category or classification of products or items present in the logistics center.

[0885] "Position" is coordinate information that indicates the location of an item or an obstacle within a logistics center.

[0886] "Quantity" is information indicating how many of a particular type of item are present in the logistics center.

[0887] An "obstacle" is any material or structure that may affect the movement of goods or the efficiency of work within a logistics center.

[0888] "Means for analyzing" refers to algorithms or software that processes received image and video data to identify the type, location, and quantity of items, and the presence of obstacles.

[0889] An "external database" is a database that exists outside the distribution center and can be accessed and used by the server.

[0890] "Environmental data" refers to environmental information that may affect the operation of a logistics center, such as temperature, humidity, and lighting conditions inside and outside the logistics center.

[0891] An "assessment report" is a report that shows the current status and risks of a logistics center, generated based on analyzed information and environmental data.

[0892] "Generative AI" is a system that uses artificial intelligence technology to create optimal operational management strategies based on evaluation reports.

[0893] "Operational management strategy" refers to specific work procedures and plans for achieving efficient operations within a logistics center.

[0894] The "means for transmitting and displaying" is a function for transmitting the created operation management strategy to the user's terminal and providing an interface for visually confirming the strategy.

[0895] The "execution results" are data indicating the status and deliverables after the user has performed the work.

[0896] The "means for evaluating effectiveness" is a function for analyzing and evaluating the effectiveness of the operation management strategy based on the user's execution results.

[0897] "Emotional state" is information that recognizes the psychological state and emotions of workers in real time.

[0898] The "means for adjusting the strategy presentation method" is a function for changing the content and presentation method of the presented operation management strategy according to the recognized emotional state of the worker.

[0899] To implement the present invention, each part of the system functions in the following steps.

[0900] 1. User's device

[0901] Users primarily use smartphones and tablets. These devices must have the following features:

[0902] Photography function: The user uses a camera to capture images of the situation inside the logistics center. Using this camera, the location and status of items are recorded as image and video data.

[0903] Data upload function: A dedicated application is used to upload captured image and video data to the server. Data can be sent quickly through this application.

[0904] Emotion recognition: Detects the user's emotional state in real time using the camera and microphone, using facial recognition and voice analysis technologies.

[0905] 2. Server

[0906] The server has the following features:

[0907] Data reception and storage function: The server receives image data, video data, and emotion data sent by users and stores them securely in a database. Storing this data is important for subsequent analysis and evaluation.

[0908] Image and video analysis function: The server is equipped with image analysis algorithms to analyze the received data, thereby identifying the type, location, and quantity of items within the logistics center, as well as the presence of obstacles.

[0909] Environmental data acquisition function: The server acquires environmental data from an external database, including temperature, humidity, lighting conditions, etc.

[0910] Assessment report generation function: The server integrates the analysis results and environmental data to generate an assessment report on the current status of the logistics center. This report clarifies the operational status and risks of the logistics center.

[0911] Strategy planning function using generative AI models: The server uses generative AI based on the evaluation report to automatically create optimal operation and management strategies, including material movement plans and work procedures.

[0912] Emotion-based strategy adjustment function: The server recognizes the user's emotional state and adjusts the way strategies are presented, thereby reducing stress and fatigue for workers and promoting efficient work.

[0913] 3. Specific examples of applications

[0914] Consider a case where this system is used to optimize the operation and management of a specific area of ​​a logistics center. A user takes a picture of the specific area using a smartphone and uploads it to a server using a dedicated application.

[0915] Example prompt sentence:

[0916] Take an image of a specific area in your warehouse and run a query to generate an optimal management strategy. Use the following image path:

[0917] Image path: 'warehouse_section.jpg'

[0918] The generative AI model suggests optimal work procedures based on the location, type, and quantity of items, as well as the emotional state of the worker."

[0919] The server analyzes the received image data to identify the location, type, and quantity of items, as well as the presence of obstacles. An assessment report is generated based on the analysis results and environmental data, and an optimal operational management strategy is then created using a generative AI model. Furthermore, the system takes into account the emotional state of the worker and adjusts the strategy presentation method to improve work efficiency and reduce stress.

[0920] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0921] Step 1:

[0922] Users take pictures of the situation inside the logistics center using a smartphone or tablet device. The captured image and video data are uploaded to the server via a dedicated application. The input here is image and video data, and the output is data sent to the server.

[0923] Step 2:

[0924] The server receives image and video data sent by users and stores them securely in a database. The input is the data received from the user, and the output is storing it in the database. This involves specific operations to verify the consistency and safety of the data.

[0925] Step 3:

[0926] The server analyzes the stored image and video data. It uses image analysis algorithms to identify the type, location, and quantity of objects, as well as the presence of obstacles. The input is image and video data, and the output is the analysis results, such as the type, location, and quantity of objects, and the presence of obstacles. Specific operations include image processing using libraries such as OpenCV.

[0927] Step 4:

[0928] The server retrieves environmental data (temperature, humidity, lighting conditions, etc.) inside and outside the distribution center from an external database. The input is a query to the external database, and the output is the retrieved environmental data. This step includes specific operations to retrieve data, such as through API calls.

[0929] Step 5:

[0930] The server integrates the analysis results and the acquired environmental data to generate an assessment report. The input is the analysis results of the item and the environmental data, and the output is the assessment report. Specific operations include data integration and report generation algorithms.

[0931] Step 6:

[0932] The server uses a generative AI model based on the evaluation report to create an optimal operations management strategy. This strategy includes a material movement plan and work procedures. The input is the evaluation report, and the output is the operations management strategy. Specific operations include a strategy planning process using the generative AI model.

[0933] Step 7:

[0934] The server obtains emotional data from the user's device in real time and adjusts the strategy to be presented. The input is emotional data, and the output is an operational management strategy adjusted based on the emotion. Specific operations include obtaining emotional data through facial recognition and voice analysis and adjusting the strategy.

[0935] Step 8:

[0936] The server sends the created operation management strategy to the user's terminal and displays it through a visual interface. The input is the operation management strategy, and the output is the strategy information displayed on the user's terminal. Specific operations include data transmission and user interface display.

[0937] Step 9:

[0938] The user performs specific tasks based on the presented operation management strategy. The input is the operation management strategy, and the output is the work results. Specific actions include moving and organizing items within the warehouse and executing work procedures.

[0939] Step 10:

[0940] The user reports the results of their work to the server through a dedicated application. The input is the data of the work result, and the output is the transmission of data to the server. This includes uploading images and reports after the work.

[0941] Step 11:

[0942] The server receives and analyzes the execution result data sent by the user, thereby evaluating the effectiveness of the operation management strategy and readjusting the strategy as necessary. The input is the execution result data, and the output is an updated operation management strategy. Specific operations include data analysis and strategy readjustment.

[0943] Step 12:

[0944] The server notifies the user terminal of the updated strategy and the next proposal. The input is the updated management strategy, and the output is the notification information. Specific operations include the execution of the notification mechanism.

[0945] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0946] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0947] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0948] [Third embodiment]

[0949] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0950] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0951] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0952] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0953] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0954] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0955] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0956] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0957] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0958] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0959] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0960] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0961] The present invention relates to a system that receives image and video data acquired from users, analyzes them to evaluate the current state of a forest, and generates and presents an optimal forest management strategy. This system provides support, especially for novice forestry managers, to achieve sustainable forest management.

[0962] System Overview

[0963] The system receives image and video data taken by users of their own mountains and analyzes it to identify forest species, density, growth stage, and the presence of pests and diseases. It also collects meteorological data to generate a comprehensive assessment report and uses AI to develop an optimal forest management strategy. The strategy is then sent to the user's device, and the user can carry out specific tasks based on it, with the results then fed back into the system.

[0964] System configuration

[0965] 1. User's device

[0966] Photography function: Users can use their smartphones or cameras to capture image and video data of the forest.

[0967] Data upload function: Upload captured data to the system via a dedicated application.

[0968] 2. Server

[0969] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database.

[0970] Analysis features:

[0971] Image analysis algorithms are used to identify tree species, density, growth stage, and the presence of pests and diseases.

[0972] Use the API to obtain local climate information from a weather database.

[0973] Assessment report generation function: Integrates analyzed information with meteorological data to generate assessment reports on forest health and risks.

[0974] Strategic planning function using generative AI: Automatically creates optimal forest management strategies based on assessment reports.

[0975] Strategy notification function: The created management strategy is sent to the user's terminal and presented to the user in a visual interface.

[0976] Feedback analysis function: Receives execution results from users, reevaluates the effectiveness of the strategy, and adjusts it as necessary.

[0977] Program processing

[0978] Below, we will explain in detail an example of how the system's programs are used and the processing flow.

[0979] 1. Uploading image and video data

[0980] To understand the current state of their mountain, users take multiple photos and videos using their smartphone or camera and upload them to the system via the application.

[0981] 2. Receipt and storage of data

[0982] The server receives the data uploaded by the user and stores it securely in a database.

[0983] 3. Data Analysis

[0984] The server passes the received data through image analysis algorithms to identify tree species, density, growth stage and the presence of pests and diseases.

[0985] The server also retrieves climate information for the target area from a weather database via an API.

[0986] 4. Generate an evaluation report

[0987] The server integrates image analysis results with meteorological data to generate assessment reports on forest health and risks.

[0988] 5. Generating a management strategy

[0989] Based on the assessment report, the server uses generative AI to create an optimal forest management strategy, including a felling plan, a planting plan, and the timing of maintenance.

[0990] 6. Informing and implementing control strategies

[0991] The server transmits the created management strategy to the user's terminal and presents it to the user through a visual interface.

[0992] The user carries out specific tasks based on the presented strategies.

[0993] 7. Feedback on execution results

[0994] The user reports the results of their work to the system through the application, which includes photos of the work and a status report.

[0995] 8. Evaluate and recalibrate your strategy

[0996] The server receives the execution results from the user, evaluates the effectiveness of the strategy, adjusts the strategy if necessary, and makes next suggestions.

[0997] This system makes it easy for even new forest owners to achieve sustainable forest management. By continuing to implement appropriate forest management through specific instructions and feedback, environmentally friendly forest management becomes possible.

[0998] The processing flow will be explained below.

[0999] Step 1:

[1000] Users take pictures of their mountain using a smartphone or camera. In order to collect sufficient information, they take multiple images and videos from various angles.

[1001] Step 2:

[1002] The user launches a dedicated application and uploads the captured image and video data to the application, which then transmits the data to the system.

[1003] Step 3:

[1004] The server receives the image and video data sent by the user and stores the received data securely in a database.

[1005] Step 4:

[1006] The server then passes the received data to an image analysis algorithm that uses computer vision technology to identify tree species, density, growth stage, and the presence of pests and diseases.

[1007] Step 5:

[1008] The server connects to a weather database and retrieves local weather information through an API, including past weather conditions and future forecast data.

[1009] Step 6:

[1010] The server combines image analysis results with meteorological data to generate forest health and risk assessment reports, which contain detailed information about the condition of the forest.

[1011] Step 7:

[1012] The server uses generative AI to create an optimal forest management strategy based on the assessment report, including optimal harvesting plans, planting plans, and maintenance timing.

[1013] Step 8:

[1014] The server then sends the created forest management strategy to the user's device, where the strategy can be viewed through a visual interface.

[1015] Step 9:

[1016] Users are then presented with a management strategy on their device, which involves carrying out specific tasks, such as cutting trees, planting trees, and tending them at set times.

[1017] Step 10:

[1018] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[1019] Step 11:

[1020] The server again receives and analyzes the execution result data from the user, thereby evaluating the effectiveness of the strategy and readjusting it if necessary.

[1021] Step 12:

[1022] The server notifies the user's device of updated strategies and next proposals, allowing the user to continuously implement appropriate forest management.

[1023] In this way, the system provides continuous support to users in achieving sustainable forest management.

[1024] Example 1

[1025] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1026] Traditional forest management requires specialized knowledge and experience, making it difficult for novice forest managers. Accurately assessing forest health and risks and formulating appropriate management strategies requires advanced technology and a great deal of time and effort. Furthermore, it is difficult to effectively utilize meteorological data, making it difficult to develop optimal felling and planting plans. This makes it difficult to achieve sustainable forest management.

[1027] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1028] In this invention, the server includes means for receiving image data and video data of a forest captured by a user, means for analyzing the received image data and video data using an image analysis algorithm to identify the forest type, density, growth stage, and presence of pests and diseases, means for generating a forest assessment report based on the analyzed information and meteorological data obtained from an external database, means for creating an optimal forest management strategy using a generative AI model based on the assessment report, means for sending the created forest management strategy to the user's device and displaying it in a visual interface, and means for receiving the user's execution results again and evaluating the effectiveness of the forest management strategy. This enables sustainable forest management even for novice foresters without advanced expertise.

[1029] A "user" is a person who uses the system to provide image and video data related to forest management, and receives and implements analysis results and management strategies.

[1030] "Image and video data" refers to photographs and video information taken by users of the current state of the forest, and is used to identify the type and density of trees, their growth stage, and the presence of pests and diseases.

[1031] The "means for receiving" refers to the process and mechanism by which the server acquires and stores image data and video data uploaded by users.

[1032] "Image analysis algorithm" means a mathematical and technical method for analyzing received image and video data to identify tree species, density, growth stage, and the presence of pests and diseases.

[1033] "Weather data" refers to local climate information obtained from external databases and is data that is integrated when generating forest assessment reports.

[1034] The "assessment report" is a report that evaluates the health and risks of a forest based on image analysis results and meteorological data.

[1035] A "generative AI model" is an artificial intelligence model that automatically creates optimal forest management strategies based on assessment reports.

[1036] A "forest management strategy" is a specific plan for managing forests sustainably, including optimal felling plans, planting plans, and maintenance timing.

[1037] The "visual interface" refers to the screen display and operation means that allow the server to display the forest management strategy generated by the server in an easy-to-understand manner for the user.

[1038] "Execution results" are the results of work carried out by the user based on the forest management strategy presented, and are re-evaluated.

[1039] The "means for evaluating effectiveness" refers to the process and techniques by which the server receives the execution results from the user and re-evaluates the effectiveness of the existing forest management strategy.

[1040] The present invention relates to a system that receives image and video data captured by users, analyzes them to evaluate the current state of a forest, and generates and presents an optimal forest management strategy. This system provides support, especially for novice foresters, to achieve sustainable forest management.

[1041] System Overview

[1042] The system receives image and video data taken by users of their own mountains and analyzes it to identify forest species, density, growth stage, and the presence of pests and diseases. It also collects meteorological data to generate a comprehensive assessment report and uses a generative AI model to develop an optimal forest management strategy. The strategy is then sent to the user's device, and the user can carry out specific tasks based on it, with the results then fed back into the system.

[1043] Hardware and Software Configuration

[1044] 1. User's device

[1045] Photography function: Users can use their smartphones or digital cameras to capture image and video data of the forest.

[1046] Data upload function: Upload captured data to the system through a dedicated application (e.g., "ForestCare" or "TreeHealth").

[1047] 2. Server

[1048] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database (e.g., MySQL, MongoDB).

[1049] Image analysis algorithms: Image analysis algorithms (e.g., TensorFlow, PyTorch) are used to identify tree species, density, growth stage, and the presence of pests and diseases.

[1050] Weather data acquisition: Use an API (e.g., OpenWeatherMap API) to obtain local climate information from a weather database.

[1051] Assessment report generation: Use Jupyter Notebook and Pandas to integrate the analyzed information with meteorological data and generate assessment reports on forest health and risks.

[1052] Strategy planning using generative AI: Based on the evaluation report, a generative AI model (e.g., GPT-4) is used to create an optimal forest management strategy.

[1053] Strategy notification: The created management strategy is sent to the user's terminal and presented to the user in a visual interface (e.g., a web application).

[1054] Feedback analysis: Receive execution results from users, reassess the effectiveness of the strategy, and readjust the strategy if necessary.

[1055] Specific examples

[1056] For example, a novice forestry manager might follow these steps to understand the current state of his or her mountain:

[1057] 1. Capture and upload data:

[1058] Users use their smartphones to take photos and videos of the mountain from multiple angles.

[1059] Launch the dedicated app "ForestCare" and upload the captured data to the system.

[1060] 2. Data analysis and evaluation:

[1061] The server passes the received data to a TensorFlow image analysis algorithm, which identifies tree species, density, growth stage, and the presence of pests and diseases.

[1062] Obtain weather data for the target area from OpenWeatherMap via API.

[1063] 3. Generate an assessment report:

[1064] The server integrates the image analysis results and meteorological data using Pandas and creates an evaluation report.

[1065] Create a report in Jupyter Notebook and save it as a PDF.

[1066] 4. Generate a management strategy:

[1067] The server uses a generative AI model (GPT-4) to input the following prompt based on the evaluation report: "Please tell me the appropriate tree-cutting and planting plan for this area."

[1068] Format the strategy proposed by GPT-4 and notify the user.

[1069] 5. Notice and Execution:

[1070] The server pushes the created management strategy to the user's app.

[1071] Users can check the strategy through the app and carry out the tasks presented.

[1072] 6. Feedback and reassessment of results:

[1073] Users report their work results to the app and upload photos and status reports.

[1074] The server receives and analyzes the reports and readjusts the strategy as needed.

[1075] In this way, this system enables even novice foresters to achieve sustainable forest management without specialized knowledge.

[1076] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1077] Step 1:

[1078] To understand the current state of their mountain, users take image and video data of the forest using a smartphone or digital camera. The input is the captured image and video data, and the output is the saving of this data on the user's device. Specifically, users take photos of the overall view from multiple angles, the condition of nearby trees, and the state of damage caused by pests and diseases.

[1079] Step 2:

[1080] The user launches a dedicated mobile application (e.g., "ForestCare" or "TreeHealth") and uploads the captured data to the system. The input is image and video data stored on the user's device, and the output is the data being sent to the system's server. Specifically, the user taps the "Upload Data" button on the application, selects the captured data, and uploads it.

[1081] Step 3:

[1082] The server immediately receives image and video data uploaded by users and securely stores it in a database (e.g., MySQL, MongoDB). The input is the image and video data received from users, and the output is the data stored in the database. Specifically, the server receives the data uploaded via an HTTP request and records it in the database.

[1083] Step 4:

[1084] The server passes the received image and video data to an image analysis algorithm (e.g., TensorFlow, PyTorch) and performs the analysis. The input is the image and video data stored in the database, and the output is the analysis results regarding the tree species, density, growth stage, and the presence of pests and diseases. Specifically, the server inputs the data into the image analysis algorithm, extracts specific features, and evaluates them.

[1085] Step 5:

[1086] The server obtains the analysis results and uses an API (e.g., OpenWeatherMap API) to obtain weather data for the target area. The input is the analysis results and information about the target area, and the output is the obtained weather data. Specifically, the server sends a request to the API endpoint to obtain the weather data.

[1087] Step 6:

[1088] The server integrates the image analysis results with meteorological data to generate an assessment report on forest health and risks. The input is the image analysis results and meteorological data, and the output is an assessment report. Specifically, the server integrates the analysis results and meteorological data using Pandas, creates an assessment report in Jupyter Notebook, and saves it in PDF format.

[1089] Step 7:

[1090] The server uses a generative AI model (e.g., GPT-4) to create an optimal forest management strategy based on the evaluation report. The input is the evaluation report and a prompt (e.g., "Please tell me the appropriate logging and planting plan for this area."), and the output is the generated forest management strategy. Specifically, the server inputs the evaluation report as a prompt to GPT-4 and obtains the generated strategy.

[1091] Step 8:

[1092] The server sends the created forest management strategy to the user's device and presents it to the user through a visual interface. The input is the created forest management strategy, and the output is the strategy displayed on the user's device. Specifically, the server pushes the strategy to the user's app in JSON format, and the strategy is displayed visually on the application.

[1093] Step 9:

[1094] The user carries out specific tasks based on the presented forest management strategy. The input is the forest management strategy displayed on the user's terminal, and the output is the results of the executed tasks. As specific actions, the user performs tasks such as cutting trees, planting, and maintenance according to the strategy.

[1095] Step 10:

[1096] Users report the results of their work through the application and upload photos and status reports after the work is completed to the system. The input is data about the work performed and its results, and the output is feedback sent to the system. Specifically, users upload photos and reports using the app's "Work Completion Report" function.

[1097] Step 11:

[1098] The server receives feedback from the user, analyzes it again, and evaluates the effectiveness of the forest management strategy. It readjusts the strategy as needed and presents the next proposal to the user. The input is the received feedback data, and the output is the evaluated effectiveness of the strategy and the readjusted management strategy. Specifically, the server passes the feedback data to an analysis algorithm for evaluation, and if necessary, uses a generative AI model to generate a new strategy.

[1099] This process allows even novice foresters to achieve sustainable forest management. The process of specific instructions and feedback is repeated, allowing for more precise management.

[1100] (Application example 1)

[1101] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1102] Maintaining and managing factory equipment and production lines requires a great deal of time and effort, and conventional methods have the problem of making it difficult to detect abnormalities early and formulate appropriate maintenance plans.The present invention aims to improve the operational efficiency of equipment and reduce unexpected downtime by efficiently evaluating the health of factory equipment and accurately proposing necessary maintenance.

[1103] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1104] In this invention, the server includes means for receiving image data and video data acquired from a user, means for analyzing the received image data and video data to identify the type of equipment, its operating status, and the presence or absence of abnormalities, and means for generating an equipment evaluation report based on the analyzed information and environmental data acquired from the manufacturing environment database. This makes it possible to evaluate the health of the equipment in real time and present optimal maintenance plans, replacement plans, and maintenance timing.

[1105] The "means for receiving image data and video data acquired from the user" is a function for transmitting image data and video data captured by the user to a server via the Internet and receiving the data.

[1106] "Means for analyzing received image data and video data to identify the type of equipment, its operating status, and whether or not there are any abnormalities" refers to a function that analyzes received image data and video data using computer vision technology and machine learning algorithms, etc., to identify the type of specific equipment, its operating status, and signs of abnormalities.

[1107] "Means for generating an equipment evaluation report based on the analyzed information and environmental data obtained from the manufacturing environment database" refers to a function for automatically generating an evaluation report by combining the analysis results with various environmental data (e.g., temperature, humidity, etc.) collected in advance.

[1108] "A means for creating an optimal asset management strategy using generative AI based on an evaluation report" is a function that utilizes a generative AI model to automatically consider and create optimal maintenance plans and strategies from the generated evaluation report.

[1109] "Means for sending and displaying the created facility management strategy to the user's device" refers to a function for sending the generated optimal management strategy to the user's smartphone, computer, etc., and displaying it in an intuitive interface.

[1110] "Means of receiving the user's execution results again and evaluating the effectiveness of the asset management strategy" refers to a function that receives the results of the maintenance work and maintenance that has been carried out again as feedback to the system, evaluates the results, and reflects them in future strategies.

[1111] "Equipment management strategy including optimal maintenance plan, replacement plan and maintenance timing" is a management strategy including the most appropriate maintenance work schedule, part replacement timing and maintenance method for maintaining the health of equipment.

[1112] "Facility management" refers to the planned maintenance, repair, and replacement work required to operate the equipment and machinery used in factories and production lines efficiently and effectively.

[1113] The present invention relates to a system that efficiently evaluates the health of equipment and manufacturing lines in a factory and proposes optimal maintenance plans and strategies. This system provides significant support, especially to factory managers who find equipment management difficult. The system is realized by the following procedure.

[1114] System configuration

[1115] 1. User's device

[1116] Photography function: Users can use their smartphones or cameras to capture image and video data of equipment and production lines.

[1117] Data upload function: Upload captured data to a server via a dedicated application.

[1118] 2. Server

[1119] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database.

[1120] Analysis features:

[1121] Image analysis algorithms are used to identify the type of equipment, its operating status, and whether or not there are any abnormalities.

[1122] Use the API to obtain environmental data from the manufacturing environment database.

[1123] Assessment report generation function: Integrates analyzed information with environmental data to generate assessment reports on the health and risk of facilities.

[1124] Generative AI strategy planning function: Based on the assessment report, generates an asset management strategy including optimal maintenance plans, replacement plans, and maintenance timing.

[1125] Strategy notification function: The created facility management strategy is sent to the user's device and presented to the user via a visual interface.

[1126] Feedback analysis function: Receives execution results from users, reevaluates the effectiveness of the strategy, and adjusts it as necessary.

[1127] Program processing explanation

[1128] Hardware:

[1129] Smartphones: Used as photography devices by factory robots.

[1130] Server: Cloud server (e.g. AWS, Google Cloud) for data analysis and management strategy generation.

[1131] software:

[1132] OpenCV: Used for image and video capture.

[1133] TensorFlow: Image analysis algorithms for anomaly detection and analysis.

[1134] REST API: Used for data upload and notifications (e.g. FastAPI).

[1135] Examples:

[1136] As an example, consider the case where the motor of equipment A is overheating. A user uses a smartphone to take pictures of equipment A and uploads the images and videos to the system. The server receives these and analyzes them using TensorFlow. Based on the analysis results, the generation AI creates an optimal maintenance plan and notifies the user again on their device. In this case, the following can be used as an example of a prompt text:

[1137] Example prompt sentence:

[1138] "Equipment A has been operating for a long time recently, and the motor temperature is rising sharply. Please detect this situation and generate an optimal maintenance plan."

[1139] The system helps reduce unplanned downtime while ensuring facility safety and efficiency.

[1140] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1141] Step 1:

[1142] Users use smartphones or cameras to take images and videos of the equipment and production lines in the factory. These become the input data. Specifically, images are taken from multiple viewpoints to record the equipment's condition in detail.

[1143] Step 2:

[1144] The user's device uploads the captured image and video data to the server via a dedicated application. The input data is the image and video data captured in the previous step, and the output is the completion of uploading to the server. Specifically, the data is sent using a data transfer protocol (e.g., HTTP).

[1145] Step 3:

[1146] The server receives uploaded image and video data and stores it securely in a database. The input data is the data uploaded by the user, and the output is the data that has been saved. Specifically, the received data is converted into an appropriate format and stored in the database.

[1147] Step 4:

[1148] The server passes the received data to an image analysis algorithm (using, for example, OpenCV or TensorFlow) to identify the type of equipment, its operating status, and whether or not there are any abnormalities. The input data is the stored image and video data, and the output is the analysis results (type of equipment, operating status, and whether or not there are any abnormalities). Specifically, the algorithm scans the image and detects specific patterns and abnormalities.

[1149] Step 5:

[1150] The server obtains environmental data (e.g., temperature, humidity) from the manufacturing environment database through the API. The input data is a request to the environment database, and the output is the obtained environmental data. Specifically, it accesses the API endpoint and retrieves the required data.

[1151] Step 6:

[1152] The server integrates the image analysis results with the environmental data to generate an assessment report on the health and risk of the facility. The input data are the analysis results and environmental data, and the output is an assessment report. Specifically, it runs an assessment algorithm that integrates both sets of data and generates the assessment results in text and graph format.

[1153] Step 7:

[1154] The server uses a generative AI model based on the evaluation report to create an equipment management strategy that includes optimal maintenance plans, replacement plans, and maintenance timing. The input data is the evaluation report, and the output is the management strategy. Specifically, the server inputs prompts into the generative AI model to generate the optimal strategy.

[1155] Step 8:

[1156] The server sends the created facility management strategy to the user's device and presents it to the user in a visual interface. The input data is the management strategy, and the output is a notification to the user's device. Specifically, the server encodes the management strategy as a message and sends it to the user's application.

[1157] Step 9:

[1158] The user performs specific maintenance work based on the transmitted management strategy and reports the results back to the system. The input data is the implementation results, and the output is feedback data. Specific operations include recording the status after the maintenance work using images and text, and reporting it through the application.

[1159] Step 10:

[1160] The server receives execution results from the user, analyzes them, and reevaluates the effectiveness of the asset management strategy. If necessary, it adjusts the strategy and makes the next proposal. The input data is feedback data, and the output is a new strategy. Specifically, it analyzes the execution results, reflects any necessary modifications in the generative AI model, and creates a new management strategy.

[1161] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1162] This invention relates to a system that receives image and video data acquired from users, analyzes them to evaluate the current state of a forest, and generates and presents an optimal forest management strategy. Furthermore, the system aims to enhance the effectiveness of the management strategy by incorporating an emotion engine that recognizes the user's emotions. This system provides support, especially for novice foresters, to achieve sustainable forest management.

[1163] System Overview

[1164] The system receives image and video data taken by users of their own mountains and analyzes it to identify forest species, density, growth stage, and the presence of pests and diseases. It also collects weather data to generate a comprehensive assessment report and uses generative AI to develop optimal forest management strategies. Furthermore, by utilizing an emotion engine that recognizes the user's emotions, the system adjusts the way strategies are presented to the user based on their emotional state, enabling them to implement strategies in a way that is more receptive to the user.

[1165] System configuration

[1166] 1. User's device

[1167] Photography function: Users can use their smartphones or cameras to capture image and video data of the forest.

[1168] Data upload function: Upload captured data to the system via a dedicated application.

[1169] Emotion recognition: Equipped with sensors and cameras to recognize the user's emotional state.

[1170] 2. Server

[1171] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database.

[1172] Analysis features:

[1173] Image analysis algorithms are used to identify tree species, density, growth stage, and the presence of pests and diseases.

[1174] Use the API to obtain local climate information from a weather database.

[1175] Assessment report generation function: Integrates analyzed information with meteorological data to generate assessment reports on forest health and risks.

[1176] Strategic planning function using generative AI: Automatically creates optimal forest management strategies based on assessment reports.

[1177] Sentiment visualization and strategy adjustment features:

[1178] Recognize user emotions in real time and adjust the presentation and content of management strategies.

[1179] Strategy notification function: The created management strategy is sent to the user's terminal and presented to the user in a visual interface.

[1180] Feedback analysis function: Receiving execution results from users, evaluating the effectiveness of strategies and adjusting them as necessary.

[1181] Program processing

[1182] Below, we will explain in detail an example of how the system's programs are used and the processing flow.

[1183] 1. Uploading image and video data

[1184] To understand the current state of their mountain, users take multiple photos and videos using their smartphone or camera and upload them to the system via the application.

[1185] 2. Collecting Emotional Data

[1186] The user's device uses techniques such as facial recognition and voice analysis to detect the user's emotional state in real time and transmits that data to a server.

[1187] 3. Receipt and storage of data

[1188] The server receives the image data, video data, and emotion data sent by the user and safely stores them in a database.

[1189] 4. Data Analysis

[1190] The server passes the received data through image analysis algorithms to identify tree species, density, growth stage and the presence of pests and diseases.

[1191] The server also retrieves climate information for the target area from a weather database via an API.

[1192] 5. Integrating evaluation reports and sentiment analysis

[1193] The server integrates the image analysis results, meteorological data, and user emotional data to generate an assessment report on the health and risks of the forest.

[1194] 6. Generating a management strategy

[1195] Based on the assessment report, the server uses generative AI to create an optimal forest management strategy, which includes a felling plan, a planting plan, and the timing of maintenance.

[1196] 7. Adjusting strategies based on emotional state

[1197] The server takes into account the user's emotional state and adjusts the content and presentation of the management strategies it presents, for example, providing encouraging messages or simplified instructions if the user is feeling anxious.

[1198] 8. Informing and implementing control strategies

[1199] The server transmits the created management strategy to the user's terminal, allowing the user to check the management strategy through a visual interface.

[1200] Users are then presented with a management strategy on their device, which involves carrying out specific tasks, such as cutting trees, planting trees, and tending them at set times.

[1201] 9. Feedback on execution results

[1202] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[1203] 10. Evaluate and recalibrate your strategy

[1204] The server again receives and analyzes the execution result data from the user, thereby evaluating the effectiveness of the strategy and readjusting it if necessary.

[1205] 11. Notification of next proposal

[1206] The server notifies the user's device of updated strategies and next proposals, allowing the user to continuously implement appropriate forest management.

[1207] This system allows even novice foresters to easily achieve sustainable forest management while taking into account the emotional state of the foresters. The introduction of the emotion engine is expected to improve user acceptance and lead to more effective management.

[1208] The processing flow will be explained below.

[1209] Step 1:

[1210] Users take pictures of their mountain using a smartphone or camera, and capture multiple image and video data to capture the overall picture and detailed parts of the forest.

[1211] Step 2:

[1212] The user starts the dedicated application and uploads the captured image data and video data to the application.

[1213] Step 3:

[1214] The user's device analyzes the user's facial expressions and voice in real time to recognize their emotional state, using an emotion recognition engine to distinguish emotions such as joy, surprise, sadness, and anger.

[1215] Step 4:

[1216] The user's terminal transmits the emotional state data and the captured image and video data to the server.

[1217] Step 5:

[1218] The server receives the image and video data and emotional state data sent by the user, and stores the received data securely in a database.

[1219] Step 6:

[1220] The server analyzes the received image and video data using an image analysis algorithm, specifically extracting the following information:

[1221] Tree types

[1222] Tree density

[1223] Tree growth stages

[1224] Presence of pests and diseases

[1225] Step 7:

[1226] The server uses an API to retrieve local weather information from a weather database, including historical and forecast weather conditions.

[1227] Step 8:

[1228] The server generates an assessment report on forest health and risks based on the image analysis results and acquired climate data.

[1229] Step 9:

[1230] The server uses generative AI to create an optimal forest management strategy based on the assessment report, including harvesting plans, planting plans, and maintenance timing.

[1231] Step 10:

[1232] The server tailors the presentation and content of management strategies based on the user's emotional state: for example, if the user is feeling anxious, it offers encouraging messages or simplified instructions.

[1233] Step 11:

[1234] The server transmits the created management strategy to the user's terminal, allowing the user to check the management strategy on a visual interface.

[1235] Step 12:

[1236] The user is then presented with a management strategy and performs specific tasks, including cutting, planting, and tending at specified times.

[1237] Step 13:

[1238] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[1239] Step 14:

[1240] The server again receives and analyzes the execution result data from the user, thereby evaluating the effectiveness of the strategy and readjusting it if necessary.

[1241] Step 15:

[1242] The server notifies the user's device of updated strategies and next proposals, allowing the user to continuously implement appropriate forest management.

[1243] In this way, a system incorporating an emotion engine can provide optimal strategies according to the user's emotional state, enabling even beginners to achieve sustainable forest management.

[1244] Example 2

[1245] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1246] Conventional forest management systems could use user-provided image and video data to identify tree species, density, growth stage, and the presence of pests and diseases. However, they lacked the ability to present appropriate management strategies that take the user's emotional state into account. This made it difficult for novice forest managers to receive appropriate guidance and implement sustainable forest management. The present invention aims to solve this problem and provide a more comprehensive and effective forest management strategy.

[1247] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving image data and video data acquired from a user, a means for analyzing the received image data and video data to identify the tree species, density, growth stage, and presence of pests and diseases, a means for generating a forest assessment report based on the analyzed information and climate data acquired from a weather database, a means for creating an optimal forest management strategy based on the assessment report using artificial intelligence, a means for acquiring user emotional data and adjusting the presentation method of the forest management strategy based on the data, a means for sending the created forest management strategy to the user's terminal and displaying it, and a means for receiving the user's execution results again and evaluating the effectiveness of the forest management strategy. This allows the server to provide an optimal forest management strategy while taking the user's emotional state into consideration, enabling even novice forestry managers to achieve sustainable forest management.

[1248] "User" refers to any person or entity that uses the system, and includes, in particular, novice foresters.

[1249] "Image data and video data" is visual information showing the state of the forest captured by a user using a smartphone or camera.

[1250] "Receiving" refers to the act of the server taking in data sent by the user.

[1251] "Analyzing" refers to the act of digitally processing received data using specific algorithms to extract useful information.

[1252] "Tree type, density, growth stage, and presence of pests and diseases" refer to various forest conditions and risk factors identified from the analyzed image and video data.

[1253] A "weather database" is an information system that stores regional climate information and is accessed through an API.

[1254] A "Forest Assessment Report" is a document that integrates analyzed information and meteorological data to assess the health and risks of a forest.

[1255] "Generative artificial intelligence" is a machine learning model or generative AI model designed to derive optimal solutions based on input data.

[1256] The "Forest Management Strategy" is a specific action plan for achieving sustainable forest management, created based on the assessment report.

[1257] "Emotion data" is information that indicates the user's psychological state, obtained from the user's facial expressions and voice.

[1258] "Terminal" refers to a device used by a user, and includes general computing devices such as smartphones, tablets, or personal computers.

[1259] "Adjusting the presentation method" means appropriately changing the content of the suggestion and its expression depending on the user's emotional state.

[1260] "Execution results" refer to the activities and deliverables that users actually perform based on the proposed management strategy.

[1261] "Evaluating effectiveness" is the act of assessing the extent to which the implementation results achieved the goals of the proposed management strategy.

[1262] The present invention relates to a system that allows users to understand the current state of their forests and generate and present optimal forest management strategies. This system provides support, particularly for novice forestry managers, to achieve sustainable forest management.

[1263] The components of the system and their operation are described below.

[1264] 1. User's device

[1265] Photography function: The user uses a smartphone or camera to capture image data and video data of the forest. For example, the user can take multiple photos of a mountain slope and shoot videos from various directions.

[1266] Data upload function: The user's device uploads the captured data to the system via a dedicated application. For example, by pressing the upload button in the app, images and videos are sent to the server.

[1267] Emotion recognition function: The user's device is equipped with sensors and functions for facial recognition and voice analysis, which detect the user's emotional state in real time and send it to the server. Facial recognition technology uses common face detection algorithms (e.g., EmoReact) and voice analysis technology (e.g., AWS Transcribe).

[1268] 2. Server

[1269] Data reception and storage function: The server receives image data, video data, and emotion data sent by the user and safely stores them in a database (e.g., MySQL).

[1270] Analysis features:

[1271] The server passes the received data to an image analysis algorithm (e.g., OpenCV) to identify tree species, density, growth stage, and the presence of pests and diseases.

[1272] The server obtains climate information for the target area from a weather database via an API (e.g., OpenWeatherMap API).

[1273] Assessment report generation function: The server integrates image analysis results with meteorological data to generate assessment reports on forest health and risks, including tree health, growth status, and pest and disease risk assessments.

[1274] Strategy planning function using generative AI: The server uses generative AI (e.g., GPT-4) to create optimal forest management strategies based on the evaluation report. The generated strategies include, for example, felling plans, planting plans, and maintenance timing.

[1275] Emotion visualization and strategy adjustment: The server recognizes the user's emotional state in real time and adjusts the content and method of management strategy presentation. For example, if the user is feeling anxious, it provides encouraging messages and simplified instructions.

[1276] Strategy notification function: The server sends the created management strategy to the user's terminal and presents it to the user through a visual interface.

[1277] Feedback analysis function: The server receives the execution results from the user again, evaluates the effectiveness of the strategy, and adjusts the strategy if necessary.

[1278] Specific examples

[1279] Below are some examples of specific prompt sentences.

[1280] "Analyze images and videos of a forest taken by a user and suggest optimal management strategies. This user is currently experiencing some anxiety."

[1281] This allows even novice forest managers to easily carry out sustainable forest management while taking into account emotional states. The introduction of the emotion engine is expected to improve user acceptance and lead to more effective management.

[1282] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1283] Step 1: Uploading image and video data

[1284] Users use their smartphones or cameras to capture images and video data of forests, for example, taking multiple photos and videos to cover a specific mountain slope or tree density.

[1285] Input: Forest image data and video data

[1286] The device uploads the captured data to the system via a dedicated application. When the user presses the upload button in the application, the data is sent to the server.

[1287] Output: Data is sent to the server

[1288] Step 2: Collecting emotion data

[1289] The device uses its built-in camera and microphone to analyze the user's facial expressions and voice in real time. The camera uses facial recognition technology (e.g., EmoReact), and the microphone uses voice analysis technology (e.g., AWS Transcribe).

[1290] Input: User's facial expression data and voice data

[1291] The terminal transmits the acquired emotion data to the server.

[1292] Output: Emotion data is sent to the server

[1293] Step 3: Receiving and storing data

[1294] The server receives the image data, video data, and emotion data sent from the device using a security protocol (e.g., HTTPS).

[1295] Input: Image data, video data, and emotion data sent from the device

[1296] The server securely stores the received data in a database (e.g. MySQL).

[1297] Output: Data stored in the database

[1298] Step 4: Analyze the data

[1299] The server passes the stored image and video data to an image analysis algorithm (e.g., OpenCV) to identify tree species, density, growth stage, and the presence of pests and diseases.

[1300] Input: Stored image and video data

[1301] The server uses an API (e.g., OpenWeatherMap API) to obtain climate information for the target area from a weather database.

[1302] Output: Tree species, density, growth stage, and pest and disease presence information, and acquired climate information

[1303] Step 5: Integrating evaluation reports and sentiment analysis

[1304] The server combines the image analysis results with meteorological data to generate an assessment report on forest health and risk, including tree health, growth status, and pest and disease risk assessment.

[1305] Input: Image analysis results, weather data, emotion data

[1306] The server also analyzes the user's emotional data and integrates this data.

[1307] Output: Consolidated evaluation report

[1308] Step 6: Generate a control strategy

[1309] Based on the evaluation report, the server uses generative AI (e.g., GPT-4) to create an optimal forest management strategy, including, for example, a felling plan, a planting plan, and the timing of maintenance.

[1310] Input: Consolidated Assessment Report

[1311] The server runs the generative AI model and generates a management strategy.

[1312] Output: Optimal forest management strategy

[1313] Step 7: Adjust your strategy based on your emotional state

[1314] The server takes into account the user's emotional state and adjusts the content and presentation of the management strategies it presents to them—for example, if the user is feeling anxious, it presents strategies that include encouraging messages.

[1315] Input: optimal forest management strategy, user's emotional state

[1316] The server adjusts how the strategy is presented based on the emotion data.

[1317] Output: Coordinated management strategy

[1318] Step 8: Inform and implement your control strategy

[1319] The server transmits the created management strategy to the user's terminal, allowing the user to check the strategy through a visual interface.

[1320] Input: Coordinated management strategies

[1321] The server uses the notification function to send the strategy to the user's terminal.

[1322] Output: The control strategy displayed to the user

[1323] Step 9: Feedback on execution results

[1324] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[1325] Input: Results of forest management operations

[1326] The user sends the result data from the terminal to the system.

[1327] Output: Execution results sent to the server

[1328] Step 10: Evaluate and recalibrate your strategy

[1329] The server again receives the execution result data from the user, evaluates the effectiveness of the strategy, and readjusts the strategy as necessary based on the evaluation results.

[1330] Input: Execution result data

[1331] The server analyzes again and makes the next proposal.

[1332] Output: Retuned management strategy

[1333] Step 11: Notification of next proposal

[1334] The server notifies the user's terminal of the updated strategy and the next proposal, allowing the user to continuously carry out appropriate forest management.

[1335] Input: Recalibrated management strategies

[1336] The server will notify the user of the next offer.

[1337] Output: Next suggestion notified to the user

[1338] (Application example 2)

[1339] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1340] Operational management at a logistics center requires efficient placement and movement of goods and work procedures, which requires a great deal of effort and experience. Furthermore, worker emotions and fatigue levels have a significant impact on productivity and work efficiency, but current systems make it difficult to manage and adjust these factors. Therefore, there is a need for a system that can accurately grasp the location, type, and quantity of goods and automatically propose optimal work procedures.

[1341] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for receiving image data and video data acquired from a user, a means for analyzing the received image data and video data to identify the type, location, quantity, and presence of obstacles of items, and a means for generating an evaluation report based on the analyzed information and environmental data acquired from an external database. This makes it possible to accurately grasp the location, type, and quantity of items in a logistics center and automatically develop and present optimal operation and management strategies. Furthermore, by recognizing the user's emotional state in real time and adjusting the strategy presentation method, worker stress and fatigue can be reduced, improving work efficiency.

[1342] "Users" are the workers and managers who use the system to manage the operation of the logistics center.

[1343] "Image data and video data" refers to data recorded in a visually identifiable format that shows the items, their locations, and their status within a logistics center.

[1344] The "receiving means" is a function for transmitting image data and video data captured by the user to the server and capturing the data.

[1345] "Type of goods" is information indicating the category or classification of products or items present in the logistics center.

[1346] "Position" is coordinate information that indicates the location of an item or an obstacle within a logistics center.

[1347] "Quantity" is information indicating how many of a particular type of item are present in the logistics center.

[1348] An "obstacle" is any material or structure that may affect the movement of goods or the efficiency of work within a logistics center.

[1349] "Means for analyzing" refers to algorithms or software that processes received image and video data to identify the type, location, and quantity of items, and the presence of obstacles.

[1350] An "external database" is a database that exists outside the distribution center and can be accessed and used by the server.

[1351] "Environmental data" refers to environmental information that may affect the operation of a logistics center, such as temperature, humidity, and lighting conditions inside and outside the logistics center.

[1352] An "assessment report" is a report that shows the current status and risks of a logistics center, generated based on analyzed information and environmental data.

[1353] "Generative AI" is a system that uses artificial intelligence technology to create optimal operational management strategies based on evaluation reports.

[1354] "Operational management strategy" refers to specific work procedures and plans for achieving efficient operations within a logistics center.

[1355] The "means for transmitting and displaying" is a function for transmitting the created operation management strategy to the user's terminal and providing an interface for visually confirming the strategy.

[1356] The "execution results" are data indicating the status and deliverables after the user has performed the work.

[1357] The "means for evaluating effectiveness" is a function for analyzing and evaluating the effectiveness of the operation management strategy based on the user's execution results.

[1358] "Emotional state" is information that recognizes the psychological state and emotions of workers in real time.

[1359] The "means for adjusting the strategy presentation method" is a function for changing the content and presentation method of the presented operation management strategy according to the recognized emotional state of the worker.

[1360] To implement the present invention, each part of the system functions in the following steps.

[1361] 1. User's device

[1362] Users primarily use smartphones and tablets. These devices must have the following features:

[1363] Photography function: The user uses a camera to capture images of the situation inside the logistics center. Using this camera, the location and status of items are recorded as image and video data.

[1364] Data upload function: A dedicated application is used to upload captured image and video data to the server. Data can be sent quickly through this application.

[1365] Emotion recognition: Detects the user's emotional state in real time using the camera and microphone, using facial recognition and voice analysis technologies.

[1366] 2. Server

[1367] The server has the following features:

[1368] Data reception and storage function: The server receives image data, video data, and emotion data sent by users and stores them securely in a database. Storing this data is important for subsequent analysis and evaluation.

[1369] Image and video analysis function: The server is equipped with image analysis algorithms to analyze the received data, thereby identifying the type, location, and quantity of items within the logistics center, as well as the presence of obstacles.

[1370] Environmental data acquisition function: The server acquires environmental data from an external database, including temperature, humidity, lighting conditions, etc.

[1371] Assessment report generation function: The server integrates the analysis results and environmental data to generate an assessment report on the current status of the logistics center. This report clarifies the operational status and risks of the logistics center.

[1372] Strategy planning function using generative AI models: The server uses generative AI based on the evaluation report to automatically create optimal operation and management strategies, including material movement plans and work procedures.

[1373] Emotion-based strategy adjustment function: The server recognizes the user's emotional state and adjusts the way strategies are presented, thereby reducing stress and fatigue for workers and promoting efficient work.

[1374] 3. Specific examples of applications

[1375] Consider a case where this system is used to optimize the operation and management of a specific area of ​​a logistics center. A user takes a picture of the specific area using a smartphone and uploads it to a server using a dedicated application.

[1376] Example prompt sentence:

[1377] Take an image of a specific area in your warehouse and run a query to generate an optimal management strategy. Use the following image path:

[1378] Image path: 'warehouse_section.jpg'

[1379] The generative AI model suggests optimal work procedures based on the location, type, and quantity of items, as well as the emotional state of the worker."

[1380] The server analyzes the received image data to identify the location, type, and quantity of items, as well as the presence of obstacles. An assessment report is generated based on the analysis results and environmental data, and an optimal operational management strategy is then created using a generative AI model. Furthermore, the system takes into account the emotional state of the worker and adjusts the strategy presentation method to improve work efficiency and reduce stress.

[1381] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1382] Step 1:

[1383] Users take pictures of the situation inside the logistics center using a smartphone or tablet device. The captured image and video data are uploaded to the server via a dedicated application. The input here is image and video data, and the output is data sent to the server.

[1384] Step 2:

[1385] The server receives image and video data sent by users and stores them securely in a database. The input is the data received from the user, and the output is storing it in the database. This involves specific operations to verify the consistency and safety of the data.

[1386] Step 3:

[1387] The server analyzes the stored image and video data. It uses image analysis algorithms to identify the type, location, and quantity of objects, as well as the presence of obstacles. The input is image and video data, and the output is the analysis results, such as the type, location, and quantity of objects, and the presence of obstacles. Specific operations include image processing using libraries such as OpenCV.

[1388] Step 4:

[1389] The server retrieves environmental data (temperature, humidity, lighting conditions, etc.) inside and outside the distribution center from an external database. The input is a query to the external database, and the output is the retrieved environmental data. This step includes specific operations to retrieve data, such as through API calls.

[1390] Step 5:

[1391] The server integrates the analysis results and the acquired environmental data to generate an assessment report. The input is the analysis results of the item and the environmental data, and the output is the assessment report. Specific operations include data integration and report generation algorithms.

[1392] Step 6:

[1393] The server uses a generative AI model based on the evaluation report to create an optimal operations management strategy. This strategy includes a material movement plan and work procedures. The input is the evaluation report, and the output is the operations management strategy. Specific operations include a strategy planning process using the generative AI model.

[1394] Step 7:

[1395] The server obtains emotional data from the user's device in real time and adjusts the strategy to be presented. The input is emotional data, and the output is an operational management strategy adjusted based on the emotion. Specific operations include obtaining emotional data through facial recognition and voice analysis and adjusting the strategy.

[1396] Step 8:

[1397] The server sends the created operation management strategy to the user's terminal and displays it through a visual interface. The input is the operation management strategy, and the output is the strategy information displayed on the user's terminal. Specific operations include data transmission and user interface display.

[1398] Step 9:

[1399] The user performs specific tasks based on the presented operation management strategy. The input is the operation management strategy, and the output is the work results. Specific actions include moving and organizing items within the warehouse and executing work procedures.

[1400] Step 10:

[1401] The user reports the results of their work to the server through a dedicated application. The input is the data of the work result, and the output is the transmission of data to the server. This includes uploading images and reports after the work.

[1402] Step 11:

[1403] The server receives and analyzes the execution result data sent by the user, thereby evaluating the effectiveness of the operation management strategy and readjusting the strategy as necessary. The input is the execution result data, and the output is an updated operation management strategy. Specific operations include data analysis and strategy readjustment.

[1404] Step 12:

[1405] The server notifies the user terminal of the updated strategy and the next proposal. The input is the updated management strategy, and the output is the notification information. Specific operations include the execution of the notification mechanism.

[1406] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1407] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1408] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1409] [Fourth embodiment]

[1410] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1411] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1412] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1413] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1414] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1415] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1416] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1417] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1418] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1419] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1420] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1421] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1422] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1423] The present invention relates to a system that receives image and video data acquired from users, analyzes them to evaluate the current state of a forest, and generates and presents an optimal forest management strategy. This system provides support, especially for novice forestry managers, to achieve sustainable forest management.

[1424] System Overview

[1425] The system receives image and video data taken by users of their own mountains and analyzes it to identify forest species, density, growth stage, and the presence of pests and diseases. It also collects meteorological data to generate a comprehensive assessment report and uses AI to develop an optimal forest management strategy. The strategy is then sent to the user's device, and the user can carry out specific tasks based on it, with the results then fed back into the system.

[1426] System configuration

[1427] 1. User's device

[1428] Photography function: Users can use their smartphones or cameras to capture image and video data of the forest.

[1429] Data upload function: Upload captured data to the system via a dedicated application.

[1430] 2. Server

[1431] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database.

[1432] Analysis features:

[1433] Image analysis algorithms are used to identify tree species, density, growth stage, and the presence of pests and diseases.

[1434] Use the API to obtain local climate information from a weather database.

[1435] Assessment report generation function: Integrates analyzed information with meteorological data to generate assessment reports on forest health and risks.

[1436] Strategic planning function using generative AI: Automatically creates optimal forest management strategies based on assessment reports.

[1437] Strategy notification function: The created management strategy is sent to the user's terminal and presented to the user in a visual interface.

[1438] Feedback analysis function: Receives execution results from users, reevaluates the effectiveness of the strategy, and adjusts it as necessary.

[1439] Program processing

[1440] Below, we will explain in detail an example of how the system's programs are used and the processing flow.

[1441] 1. Uploading image and video data

[1442] To understand the current state of their mountain, users take multiple photos and videos using their smartphone or camera and upload them to the system via the application.

[1443] 2. Receipt and storage of data

[1444] The server receives the data uploaded by the user and stores it securely in a database.

[1445] 3. Data Analysis

[1446] The server passes the received data through image analysis algorithms to identify tree species, density, growth stage and the presence of pests and diseases.

[1447] The server also retrieves climate information for the target area from a weather database via an API.

[1448] 4. Generate an evaluation report

[1449] The server integrates image analysis results with meteorological data to generate assessment reports on forest health and risks.

[1450] 5. Generating a management strategy

[1451] Based on the assessment report, the server uses generative AI to create an optimal forest management strategy, including a felling plan, a planting plan, and the timing of maintenance.

[1452] 6. Informing and implementing control strategies

[1453] The server transmits the created management strategy to the user's terminal and presents it to the user through a visual interface.

[1454] The user carries out specific tasks based on the presented strategies.

[1455] 7. Feedback on execution results

[1456] The user reports the results of their work to the system through the application, which includes photos of the work and a status report.

[1457] 8. Evaluate and recalibrate your strategy

[1458] The server receives the execution results from the user, evaluates the effectiveness of the strategy, adjusts the strategy if necessary, and makes next suggestions.

[1459] This system makes it easy for even new forest owners to achieve sustainable forest management. By continuing to implement appropriate forest management through specific instructions and feedback, environmentally friendly forest management becomes possible.

[1460] The processing flow will be explained below.

[1461] Step 1:

[1462] Users take pictures of their mountain using a smartphone or camera. In order to collect sufficient information, they take multiple images and videos from various angles.

[1463] Step 2:

[1464] The user launches a dedicated application and uploads the captured image and video data to the application, which then transmits the data to the system.

[1465] Step 3:

[1466] The server receives the image and video data sent by the user and stores the received data securely in a database.

[1467] Step 4:

[1468] The server then passes the received data to an image analysis algorithm that uses computer vision technology to identify tree species, density, growth stage, and the presence of pests and diseases.

[1469] Step 5:

[1470] The server connects to a weather database and retrieves local weather information through an API, including past weather conditions and future forecast data.

[1471] Step 6:

[1472] The server combines image analysis results with meteorological data to generate forest health and risk assessment reports, which contain detailed information about the condition of the forest.

[1473] Step 7:

[1474] The server uses generative AI to create an optimal forest management strategy based on the assessment report, including optimal harvesting plans, planting plans, and maintenance timing.

[1475] Step 8:

[1476] The server then sends the created forest management strategy to the user's device, where the strategy can be viewed through a visual interface.

[1477] Step 9:

[1478] Users are then presented with a management strategy on their device, which involves carrying out specific tasks, such as cutting trees, planting trees, and tending them at set times.

[1479] Step 10:

[1480] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[1481] Step 11:

[1482] The server again receives and analyzes the execution result data from the user, thereby evaluating the effectiveness of the strategy and readjusting it if necessary.

[1483] Step 12:

[1484] The server notifies the user's device of updated strategies and next proposals, allowing the user to continuously implement appropriate forest management.

[1485] In this way, the system provides continuous support to users in achieving sustainable forest management.

[1486] Example 1

[1487] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1488] Traditional forest management requires specialized knowledge and experience, making it difficult for novice forest managers. Accurately assessing forest health and risks and formulating appropriate management strategies requires advanced technology and a great deal of time and effort. Furthermore, it is difficult to effectively utilize meteorological data, making it difficult to develop optimal felling and planting plans. This makes it difficult to achieve sustainable forest management.

[1489] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1490] In this invention, the server includes means for receiving image data and video data of a forest captured by a user, means for analyzing the received image data and video data using an image analysis algorithm to identify the forest type, density, growth stage, and presence of pests and diseases, means for generating a forest assessment report based on the analyzed information and meteorological data obtained from an external database, means for creating an optimal forest management strategy using a generative AI model based on the assessment report, means for sending the created forest management strategy to the user's device and displaying it in a visual interface, and means for receiving the user's execution results again and evaluating the effectiveness of the forest management strategy. This enables sustainable forest management even for novice foresters without advanced expertise.

[1491] A "user" is a person who uses the system to provide image and video data related to forest management, and receives and implements analysis results and management strategies.

[1492] "Image and video data" refers to photographs and video information taken by users of the current state of the forest, and is used to identify the type and density of trees, their growth stage, and the presence of pests and diseases.

[1493] The "means for receiving" refers to the process and mechanism by which the server acquires and stores image data and video data uploaded by users.

[1494] "Image analysis algorithm" means a mathematical and technical method for analyzing received image and video data to identify tree species, density, growth stage, and the presence of pests and diseases.

[1495] "Weather data" refers to local climate information obtained from external databases and is data that is integrated when generating forest assessment reports.

[1496] The "assessment report" is a report that evaluates the health and risks of a forest based on image analysis results and meteorological data.

[1497] A "generative AI model" is an artificial intelligence model that automatically creates optimal forest management strategies based on assessment reports.

[1498] A "forest management strategy" is a specific plan for managing forests sustainably, including optimal felling plans, planting plans, and maintenance timing.

[1499] The "visual interface" refers to the screen display and operation means that allow the server to display the forest management strategy generated by the server in an easy-to-understand manner for the user.

[1500] "Execution results" are the results of work carried out by the user based on the forest management strategy presented, and are re-evaluated.

[1501] The "means for evaluating effectiveness" refers to the process and techniques by which the server receives the execution results from the user and re-evaluates the effectiveness of the existing forest management strategy.

[1502] The present invention relates to a system that receives image and video data captured by users, analyzes them to evaluate the current state of a forest, and generates and presents an optimal forest management strategy. This system provides support, especially for novice foresters, to achieve sustainable forest management.

[1503] System Overview

[1504] The system receives image and video data taken by users of their own mountains and analyzes it to identify forest species, density, growth stage, and the presence of pests and diseases. It also collects meteorological data to generate a comprehensive assessment report and uses a generative AI model to develop an optimal forest management strategy. The strategy is then sent to the user's device, and the user can carry out specific tasks based on it, with the results then fed back into the system.

[1505] Hardware and Software Configuration

[1506] 1. User's device

[1507] Photography function: Users can use their smartphones or digital cameras to capture image and video data of the forest.

[1508] Data upload function: Upload captured data to the system through a dedicated application (e.g., "ForestCare" or "TreeHealth").

[1509] 2. Server

[1510] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database (e.g., MySQL, MongoDB).

[1511] Image analysis algorithms: Image analysis algorithms (e.g., TensorFlow, PyTorch) are used to identify tree species, density, growth stage, and the presence of pests and diseases.

[1512] Weather data acquisition: Use an API (e.g., OpenWeatherMap API) to obtain local climate information from a weather database.

[1513] Assessment report generation: Use Jupyter Notebook and Pandas to integrate the analyzed information with meteorological data and generate assessment reports on forest health and risks.

[1514] Strategy planning using generative AI: Based on the evaluation report, a generative AI model (e.g., GPT-4) is used to create an optimal forest management strategy.

[1515] Strategy notification: The created management strategy is sent to the user's terminal and presented to the user in a visual interface (e.g., a web application).

[1516] Feedback analysis: Receive execution results from users, reassess the effectiveness of the strategy, and readjust the strategy if necessary.

[1517] Specific examples

[1518] For example, a novice forestry manager might follow these steps to understand the current state of his or her mountain:

[1519] 1. Capture and upload data:

[1520] Users use their smartphones to take photos and videos of the mountain from multiple angles.

[1521] Launch the dedicated app "ForestCare" and upload the captured data to the system.

[1522] 2. Data analysis and evaluation:

[1523] The server passes the received data to a TensorFlow image analysis algorithm, which identifies tree species, density, growth stage, and the presence of pests and diseases.

[1524] Obtain weather data for the target area from OpenWeatherMap via API.

[1525] 3. Generate an assessment report:

[1526] The server integrates the image analysis results and meteorological data using Pandas and creates an evaluation report.

[1527] Create a report in Jupyter Notebook and save it as a PDF.

[1528] 4. Generate a management strategy:

[1529] The server uses a generative AI model (GPT-4) to input the following prompt based on the evaluation report: "Please tell me the appropriate tree-cutting and planting plan for this area."

[1530] Format the strategy proposed by GPT-4 and notify the user.

[1531] 5. Notice and Execution:

[1532] The server pushes the created management strategy to the user's app.

[1533] Users can check the strategy through the app and carry out the tasks presented.

[1534] 6. Feedback and reassessment of results:

[1535] Users report their work results to the app and upload photos and status reports.

[1536] The server receives and analyzes the reports and readjusts the strategy as needed.

[1537] In this way, this system enables even novice foresters to achieve sustainable forest management without specialized knowledge.

[1538] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1539] Step 1:

[1540] To understand the current state of their mountain, users take image and video data of the forest using a smartphone or digital camera. The input is the captured image and video data, and the output is the saving of this data on the user's device. Specifically, users take photos of the overall view from multiple angles, the condition of nearby trees, and the state of damage caused by pests and diseases.

[1541] Step 2:

[1542] The user launches a dedicated mobile application (e.g., "ForestCare" or "TreeHealth") and uploads the captured data to the system. The input is image and video data stored on the user's device, and the output is the data being sent to the system's server. Specifically, the user taps the "Upload Data" button on the application, selects the captured data, and uploads it.

[1543] Step 3:

[1544] The server immediately receives image and video data uploaded by users and securely stores it in a database (e.g., MySQL, MongoDB). The input is the image and video data received from users, and the output is the data stored in the database. Specifically, the server receives the data uploaded via an HTTP request and records it in the database.

[1545] Step 4:

[1546] The server passes the received image and video data to an image analysis algorithm (e.g., TensorFlow, PyTorch) and performs the analysis. The input is the image and video data stored in the database, and the output is the analysis results regarding the tree species, density, growth stage, and the presence of pests and diseases. Specifically, the server inputs the data into the image analysis algorithm, extracts specific features, and evaluates them.

[1547] Step 5:

[1548] The server obtains the analysis results and uses an API (e.g., OpenWeatherMap API) to obtain weather data for the target area. The input is the analysis results and information about the target area, and the output is the obtained weather data. Specifically, the server sends a request to the API endpoint to obtain the weather data.

[1549] Step 6:

[1550] The server integrates the image analysis results with meteorological data to generate an assessment report on forest health and risks. The input is the image analysis results and meteorological data, and the output is an assessment report. Specifically, the server integrates the analysis results and meteorological data using Pandas, creates an assessment report in Jupyter Notebook, and saves it in PDF format.

[1551] Step 7:

[1552] The server uses a generative AI model (e.g., GPT-4) to create an optimal forest management strategy based on the evaluation report. The input is the evaluation report and a prompt (e.g., "Please tell me the appropriate logging and planting plan for this area."), and the output is the generated forest management strategy. Specifically, the server inputs the evaluation report as a prompt to GPT-4 and obtains the generated strategy.

[1553] Step 8:

[1554] The server sends the created forest management strategy to the user's device and presents it to the user through a visual interface. The input is the created forest management strategy, and the output is the strategy displayed on the user's device. Specifically, the server pushes the strategy to the user's app in JSON format, and the strategy is displayed visually on the application.

[1555] Step 9:

[1556] The user carries out specific tasks based on the presented forest management strategy. The input is the forest management strategy displayed on the user's terminal, and the output is the results of the executed tasks. As specific actions, the user performs tasks such as cutting trees, planting, and maintenance according to the strategy.

[1557] Step 10:

[1558] Users report the results of their work through the application and upload photos and status reports after the work is completed to the system. The input is data about the work performed and its results, and the output is feedback sent to the system. Specifically, users upload photos and reports using the app's "Work Completion Report" function.

[1559] Step 11:

[1560] The server receives feedback from the user, analyzes it again, and evaluates the effectiveness of the forest management strategy. It readjusts the strategy as needed and presents the next proposal to the user. The input is the received feedback data, and the output is the evaluated effectiveness of the strategy and the readjusted management strategy. Specifically, the server passes the feedback data to an analysis algorithm for evaluation, and if necessary, uses a generative AI model to generate a new strategy.

[1561] This process allows even novice foresters to achieve sustainable forest management. The process of specific instructions and feedback is repeated, allowing for more precise management.

[1562] (Application example 1)

[1563] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1564] Maintaining and managing factory equipment and production lines requires a great deal of time and effort, and conventional methods have the problem of making it difficult to detect abnormalities early and formulate appropriate maintenance plans.The present invention aims to improve the operational efficiency of equipment and reduce unexpected downtime by efficiently evaluating the health of factory equipment and accurately proposing necessary maintenance.

[1565] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1566] In this invention, the server includes means for receiving image data and video data acquired from a user, means for analyzing the received image data and video data to identify the type of equipment, its operating status, and the presence or absence of abnormalities, and means for generating an equipment evaluation report based on the analyzed information and environmental data acquired from the manufacturing environment database. This makes it possible to evaluate the health of the equipment in real time and present optimal maintenance plans, replacement plans, and maintenance timing.

[1567] The "means for receiving image data and video data acquired from the user" is a function for transmitting image data and video data captured by the user to a server via the Internet and receiving the data.

[1568] "Means for analyzing received image data and video data to identify the type of equipment, its operating status, and whether or not there are any abnormalities" refers to a function that analyzes received image data and video data using computer vision technology and machine learning algorithms, etc., to identify the type of specific equipment, its operating status, and signs of abnormalities.

[1569] "Means for generating an equipment evaluation report based on the analyzed information and environmental data obtained from the manufacturing environment database" refers to a function for automatically generating an evaluation report by combining the analysis results with various environmental data (e.g., temperature, humidity, etc.) collected in advance.

[1570] "A means for creating an optimal asset management strategy using generative AI based on an evaluation report" is a function that utilizes a generative AI model to automatically consider and create optimal maintenance plans and strategies from the generated evaluation report.

[1571] "Means for sending and displaying the created facility management strategy to the user's device" refers to a function for sending the generated optimal management strategy to the user's smartphone, computer, etc., and displaying it in an intuitive interface.

[1572] "Means of receiving the user's execution results again and evaluating the effectiveness of the asset management strategy" refers to a function that receives the results of the maintenance work and maintenance that has been carried out again as feedback to the system, evaluates the results, and reflects them in future strategies.

[1573] "Equipment management strategy including optimal maintenance plan, replacement plan and maintenance timing" is a management strategy including the most appropriate maintenance work schedule, part replacement timing and maintenance method for maintaining the health of equipment.

[1574] "Facility management" refers to the planned maintenance, repair, and replacement work required to operate the equipment and machinery used in factories and production lines efficiently and effectively.

[1575] The present invention relates to a system that efficiently evaluates the health of equipment and manufacturing lines in a factory and proposes optimal maintenance plans and strategies. This system provides significant support, especially to factory managers who find equipment management difficult. The system is realized by the following procedure.

[1576] System configuration

[1577] 1. User's device

[1578] Photography function: Users can use their smartphones or cameras to capture image and video data of equipment and production lines.

[1579] Data upload function: Upload captured data to a server via a dedicated application.

[1580] 2. Server

[1581] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database.

[1582] Analysis features:

[1583] Image analysis algorithms are used to identify the type of equipment, its operating status, and whether or not there are any abnormalities.

[1584] Use the API to obtain environmental data from the manufacturing environment database.

[1585] Assessment report generation function: Integrates analyzed information with environmental data to generate assessment reports on the health and risk of facilities.

[1586] Generative AI strategy planning function: Based on the assessment report, generates an asset management strategy including optimal maintenance plans, replacement plans, and maintenance timing.

[1587] Strategy notification function: The created facility management strategy is sent to the user's device and presented to the user via a visual interface.

[1588] Feedback analysis function: Receives execution results from users, reevaluates the effectiveness of the strategy, and adjusts it as necessary.

[1589] Program processing explanation

[1590] Hardware:

[1591] Smartphones: Used as photography devices by factory robots.

[1592] Server: Cloud server (e.g. AWS, Google Cloud) for data analysis and management strategy generation.

[1593] software:

[1594] OpenCV: Used for image and video capture.

[1595] TensorFlow: Image analysis algorithms for anomaly detection and analysis.

[1596] REST API: Used for data upload and notifications (e.g. FastAPI).

[1597] Examples:

[1598] As an example, consider the case where the motor of equipment A is overheating. A user uses a smartphone to take pictures of equipment A and uploads the images and videos to the system. The server receives these and analyzes them using TensorFlow. Based on the analysis results, the generation AI creates an optimal maintenance plan and notifies the user again on their device. In this case, the following can be used as an example of a prompt text:

[1599] Example prompt sentence:

[1600] "Equipment A has been operating for a long time recently, and the motor temperature is rising sharply. Please detect this situation and generate an optimal maintenance plan."

[1601] The system helps reduce unplanned downtime while ensuring facility safety and efficiency.

[1602] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1603] Step 1:

[1604] Users use smartphones or cameras to take images and videos of the equipment and production lines in the factory. These become the input data. Specifically, images are taken from multiple viewpoints to record the equipment's condition in detail.

[1605] Step 2:

[1606] The user's device uploads the captured image and video data to the server via a dedicated application. The input data is the image and video data captured in the previous step, and the output is the completion of uploading to the server. Specifically, the data is sent using a data transfer protocol (e.g., HTTP).

[1607] Step 3:

[1608] The server receives uploaded image and video data and stores it securely in a database. The input data is the data uploaded by the user, and the output is the data that has been saved. Specifically, the received data is converted into an appropriate format and stored in the database.

[1609] Step 4:

[1610] The server passes the received data to an image analysis algorithm (using, for example, OpenCV or TensorFlow) to identify the type of equipment, its operating status, and whether or not there are any abnormalities. The input data is the stored image and video data, and the output is the analysis results (type of equipment, operating status, and whether or not there are any abnormalities). Specifically, the algorithm scans the image and detects specific patterns and abnormalities.

[1611] Step 5:

[1612] The server obtains environmental data (e.g., temperature, humidity) from the manufacturing environment database through the API. The input data is a request to the environment database, and the output is the obtained environmental data. Specifically, it accesses the API endpoint and retrieves the required data.

[1613] Step 6:

[1614] The server integrates the image analysis results with the environmental data to generate an assessment report on the health and risk of the facility. The input data are the analysis results and environmental data, and the output is an assessment report. Specifically, it runs an assessment algorithm that integrates both sets of data and generates the assessment results in text and graph format.

[1615] Step 7:

[1616] The server uses a generative AI model based on the evaluation report to create an equipment management strategy that includes optimal maintenance plans, replacement plans, and maintenance timing. The input data is the evaluation report, and the output is the management strategy. Specifically, the server inputs prompts into the generative AI model to generate the optimal strategy.

[1617] Step 8:

[1618] The server sends the created facility management strategy to the user's device and presents it to the user in a visual interface. The input data is the management strategy, and the output is a notification to the user's device. Specifically, the server encodes the management strategy as a message and sends it to the user's application.

[1619] Step 9:

[1620] The user performs specific maintenance work based on the transmitted management strategy and reports the results back to the system. The input data is the implementation results, and the output is feedback data. Specific operations include recording the status after the maintenance work using images and text, and reporting it through the application.

[1621] Step 10:

[1622] The server receives execution results from the user, analyzes them, and reevaluates the effectiveness of the asset management strategy. If necessary, it adjusts the strategy and makes the next proposal. The input data is feedback data, and the output is a new strategy. Specifically, it analyzes the execution results, reflects any necessary modifications in the generative AI model, and creates a new management strategy.

[1623] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1624] This invention relates to a system that receives image and video data acquired from users, analyzes them to evaluate the current state of a forest, and generates and presents an optimal forest management strategy. Furthermore, the system aims to enhance the effectiveness of the management strategy by incorporating an emotion engine that recognizes the user's emotions. This system provides support, especially for novice foresters, to achieve sustainable forest management.

[1625] System Overview

[1626] The system receives image and video data taken by users of their own mountains and analyzes it to identify forest species, density, growth stage, and the presence of pests and diseases. It also collects weather data to generate a comprehensive assessment report and uses generative AI to develop optimal forest management strategies. Furthermore, by utilizing an emotion engine that recognizes the user's emotions, the system adjusts the way strategies are presented to the user based on their emotional state, enabling them to implement strategies in a way that is more receptive to the user.

[1627] System configuration

[1628] 1. User's device

[1629] Photography function: Users can use their smartphones or cameras to capture image and video data of the forest.

[1630] Data upload function: Upload captured data to the system via a dedicated application.

[1631] Emotion recognition: Equipped with sensors and cameras to recognize the user's emotional state.

[1632] 2. Server

[1633] Data reception and storage function: Receives image and video data sent by users and stores them securely in a database.

[1634] Analysis features:

[1635] Image analysis algorithms are used to identify tree species, density, growth stage, and the presence of pests and diseases.

[1636] Use the API to obtain local climate information from a weather database.

[1637] Assessment report generation function: Integrates analyzed information with meteorological data to generate assessment reports on forest health and risks.

[1638] Strategic planning function using generative AI: Automatically creates optimal forest management strategies based on assessment reports.

[1639] Sentiment visualization and strategy adjustment features:

[1640] Recognize user emotions in real time and adjust the presentation and content of management strategies.

[1641] Strategy notification function: The created management strategy is sent to the user's terminal and presented to the user in a visual interface.

[1642] Feedback analysis function: Receiving execution results from users, evaluating the effectiveness of strategies and adjusting them as necessary.

[1643] Program processing

[1644] Below, we will explain in detail an example of how the system's programs are used and the processing flow.

[1645] 1. Uploading image and video data

[1646] To understand the current state of their mountain, users take multiple photos and videos using their smartphone or camera and upload them to the system via the application.

[1647] 2. Collecting Emotional Data

[1648] The user's device uses techniques such as facial recognition and voice analysis to detect the user's emotional state in real time and transmits that data to a server.

[1649] 3. Receipt and storage of data

[1650] The server receives the image data, video data, and emotion data sent by the user and safely stores them in a database.

[1651] 4. Data Analysis

[1652] The server passes the received data through image analysis algorithms to identify tree species, density, growth stage and the presence of pests and diseases.

[1653] The server also retrieves climate information for the target area from a weather database via an API.

[1654] 5. Integrating evaluation reports and sentiment analysis

[1655] The server integrates the image analysis results, meteorological data, and user emotional data to generate an assessment report on the health and risks of the forest.

[1656] 6. Generating a management strategy

[1657] Based on the assessment report, the server uses generative AI to create an optimal forest management strategy, which includes a felling plan, a planting plan, and the timing of maintenance.

[1658] 7. Adjusting strategies based on emotional state

[1659] The server takes into account the user's emotional state and adjusts the content and presentation of the management strategies it presents, for example, providing encouraging messages or simplified instructions if the user is feeling anxious.

[1660] 8. Informing and implementing control strategies

[1661] The server transmits the created management strategy to the user's terminal, allowing the user to check the management strategy through a visual interface.

[1662] Users are then presented with a management strategy on their device, which involves carrying out specific tasks, such as cutting trees, planting trees, and tending them at set times.

[1663] 9. Feedback on execution results

[1664] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[1665] 10. Evaluate and recalibrate your strategy

[1666] The server again receives and analyzes the execution result data from the user, thereby evaluating the effectiveness of the strategy and readjusting it if necessary.

[1667] 11. Notification of next proposal

[1668] The server notifies the user's device of updated strategies and next proposals, allowing the user to continuously implement appropriate forest management.

[1669] This system allows even novice foresters to easily achieve sustainable forest management while taking into account the emotional state of the foresters. The introduction of the emotion engine is expected to improve user acceptance and lead to more effective management.

[1670] The processing flow will be explained below.

[1671] Step 1:

[1672] Users take pictures of their mountain using a smartphone or camera, and capture multiple image and video data to capture the overall picture and detailed parts of the forest.

[1673] Step 2:

[1674] The user starts the dedicated application and uploads the captured image data and video data to the application.

[1675] Step 3:

[1676] The user's device analyzes the user's facial expressions and voice in real time to recognize their emotional state, using an emotion recognition engine to distinguish emotions such as joy, surprise, sadness, and anger.

[1677] Step 4:

[1678] The user's terminal transmits the emotional state data and the captured image and video data to the server.

[1679] Step 5:

[1680] The server receives the image and video data and emotional state data sent by the user, and stores the received data securely in a database.

[1681] Step 6:

[1682] The server analyzes the received image and video data using an image analysis algorithm, specifically extracting the following information:

[1683] Tree types

[1684] Tree density

[1685] Tree growth stages

[1686] Presence of pests and diseases

[1687] Step 7:

[1688] The server uses an API to retrieve local weather information from a weather database, including historical and forecast weather conditions.

[1689] Step 8:

[1690] The server generates an assessment report on forest health and risks based on the image analysis results and acquired climate data.

[1691] Step 9:

[1692] The server uses generative AI to create an optimal forest management strategy based on the assessment report, including harvesting plans, planting plans, and maintenance timing.

[1693] Step 10:

[1694] The server tailors the presentation and content of management strategies based on the user's emotional state: for example, if the user is feeling anxious, it offers encouraging messages or simplified instructions.

[1695] Step 11:

[1696] The server transmits the created management strategy to the user's terminal, allowing the user to check the management strategy on a visual interface.

[1697] Step 12:

[1698] The user is then presented with a management strategy and performs specific tasks, including cutting, planting, and tending at specified times.

[1699] Step 13:

[1700] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[1701] Step 14:

[1702] The server again receives and analyzes the execution result data from the user, thereby evaluating the effectiveness of the strategy and readjusting it if necessary.

[1703] Step 15:

[1704] The server notifies the user's device of updated strategies and next proposals, allowing the user to continuously implement appropriate forest management.

[1705] In this way, a system incorporating an emotion engine can provide optimal strategies according to the user's emotional state, enabling even beginners to achieve sustainable forest management.

[1706] Example 2

[1707] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1708] Conventional forest management systems could use user-provided image and video data to identify tree species, density, growth stage, and the presence of pests and diseases. However, they lacked the ability to present appropriate management strategies that take the user's emotional state into account. This made it difficult for novice forest managers to receive appropriate guidance and implement sustainable forest management. The present invention aims to solve this problem and provide a more comprehensive and effective forest management strategy.

[1709] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving image data and video data acquired from a user, a means for analyzing the received image data and video data to identify the tree species, density, growth stage, and presence of pests and diseases, a means for generating a forest assessment report based on the analyzed information and climate data acquired from a weather database, a means for creating an optimal forest management strategy based on the assessment report using artificial intelligence, a means for acquiring user emotional data and adjusting the presentation method of the forest management strategy based on the data, a means for sending the created forest management strategy to the user's terminal and displaying it, and a means for receiving the user's execution results again and evaluating the effectiveness of the forest management strategy. This allows the server to provide an optimal forest management strategy while taking the user's emotional state into consideration, enabling even novice forestry managers to achieve sustainable forest management.

[1710] "User" refers to any person or entity that uses the system, and includes, in particular, novice foresters.

[1711] "Image data and video data" is visual information showing the state of the forest captured by a user using a smartphone or camera.

[1712] "Receiving" refers to the act of the server taking in data sent by the user.

[1713] "Analyzing" refers to the act of digitally processing received data using specific algorithms to extract useful information.

[1714] "Tree type, density, growth stage, and presence of pests and diseases" refer to various forest conditions and risk factors identified from the analyzed image and video data.

[1715] A "weather database" is an information system that stores regional climate information and is accessed through an API.

[1716] A "Forest Assessment Report" is a document that integrates analyzed information and meteorological data to assess the health and risks of a forest.

[1717] "Generative artificial intelligence" is a machine learning model or generative AI model designed to derive optimal solutions based on input data.

[1718] The "Forest Management Strategy" is a specific action plan for achieving sustainable forest management, created based on the assessment report.

[1719] "Emotion data" is information that indicates the user's psychological state, obtained from the user's facial expressions and voice.

[1720] "Terminal" refers to a device used by a user, and includes general computing devices such as smartphones, tablets, or personal computers.

[1721] "Adjusting the presentation method" means appropriately changing the content of the suggestion and its expression depending on the user's emotional state.

[1722] "Execution results" refer to the activities and deliverables that users actually perform based on the proposed management strategy.

[1723] "Evaluating effectiveness" is the act of assessing the extent to which the implementation results achieved the goals of the proposed management strategy.

[1724] The present invention relates to a system that allows users to understand the current state of their forests and generate and present optimal forest management strategies. This system provides support, particularly for novice forestry managers, to achieve sustainable forest management.

[1725] The components of the system and their operation are described below.

[1726] 1. User's device

[1727] Photography function: The user uses a smartphone or camera to capture image data and video data of the forest. For example, the user can take multiple photos of a mountain slope and shoot videos from various directions.

[1728] Data upload function: The user's device uploads the captured data to the system via a dedicated application. For example, by pressing the upload button in the app, images and videos are sent to the server.

[1729] Emotion recognition function: The user's device is equipped with sensors and functions for facial recognition and voice analysis, which detect the user's emotional state in real time and send it to the server. Facial recognition technology uses common face detection algorithms (e.g., EmoReact) and voice analysis technology (e.g., AWS Transcribe).

[1730] 2. Server

[1731] Data reception and storage function: The server receives image data, video data, and emotion data sent by the user and safely stores them in a database (e.g., MySQL).

[1732] Analysis features:

[1733] The server passes the received data to an image analysis algorithm (e.g., OpenCV) to identify tree species, density, growth stage, and the presence of pests and diseases.

[1734] The server obtains climate information for the target area from a weather database via an API (e.g., OpenWeatherMap API).

[1735] Assessment report generation function: The server integrates image analysis results with meteorological data to generate assessment reports on forest health and risks, including tree health, growth status, and pest and disease risk assessments.

[1736] Strategy planning function using generative AI: The server uses generative AI (e.g., GPT-4) to create optimal forest management strategies based on the evaluation report. The generated strategies include, for example, felling plans, planting plans, and maintenance timing.

[1737] Emotion visualization and strategy adjustment: The server recognizes the user's emotional state in real time and adjusts the content and method of management strategy presentation. For example, if the user is feeling anxious, it provides encouraging messages and simplified instructions.

[1738] Strategy notification function: The server sends the created management strategy to the user's terminal and presents it to the user through a visual interface.

[1739] Feedback analysis function: The server receives the execution results from the user again, evaluates the effectiveness of the strategy, and adjusts the strategy if necessary.

[1740] Specific examples

[1741] Below are some examples of specific prompt sentences.

[1742] "Analyze images and videos of a forest taken by a user and suggest optimal management strategies. This user is currently experiencing some anxiety."

[1743] This allows even novice forest managers to easily carry out sustainable forest management while taking into account emotional states. The introduction of the emotion engine is expected to improve user acceptance and lead to more effective management.

[1744] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1745] Step 1: Uploading image and video data

[1746] Users use their smartphones or cameras to capture images and video data of forests, for example, taking multiple photos and videos to cover a specific mountain slope or tree density.

[1747] Input: Forest image data and video data

[1748] The device uploads the captured data to the system via a dedicated application. When the user presses the upload button in the application, the data is sent to the server.

[1749] Output: Data is sent to the server

[1750] Step 2: Collecting emotion data

[1751] The device uses its built-in camera and microphone to analyze the user's facial expressions and voice in real time. The camera uses facial recognition technology (e.g., EmoReact), and the microphone uses voice analysis technology (e.g., AWS Transcribe).

[1752] Input: User's facial expression data and voice data

[1753] The terminal transmits the acquired emotion data to the server.

[1754] Output: Emotion data is sent to the server

[1755] Step 3: Receiving and storing data

[1756] The server receives the image data, video data, and emotion data sent from the device using a security protocol (e.g., HTTPS).

[1757] Input: Image data, video data, and emotion data sent from the device

[1758] The server securely stores the received data in a database (e.g. MySQL).

[1759] Output: Data stored in the database

[1760] Step 4: Analyze the data

[1761] The server passes the stored image and video data to an image analysis algorithm (e.g., OpenCV) to identify tree species, density, growth stage, and the presence of pests and diseases.

[1762] Input: Stored image and video data

[1763] The server uses an API (e.g., OpenWeatherMap API) to obtain climate information for the target area from a weather database.

[1764] Output: Tree species, density, growth stage, and pest and disease presence information, and acquired climate information

[1765] Step 5: Integrating evaluation reports and sentiment analysis

[1766] The server combines the image analysis results with meteorological data to generate an assessment report on forest health and risk, including tree health, growth status, and pest and disease risk assessment.

[1767] Input: Image analysis results, weather data, emotion data

[1768] The server also analyzes the user's emotional data and integrates this data.

[1769] Output: Consolidated evaluation report

[1770] Step 6: Generate a control strategy

[1771] Based on the evaluation report, the server uses generative AI (e.g., GPT-4) to create an optimal forest management strategy, including, for example, a felling plan, a planting plan, and the timing of maintenance.

[1772] Input: Consolidated Assessment Report

[1773] The server runs the generative AI model and generates a management strategy.

[1774] Output: Optimal forest management strategy

[1775] Step 7: Adjust your strategy based on your emotional state

[1776] The server takes into account the user's emotional state and adjusts the content and presentation of the management strategies it presents to them—for example, if the user is feeling anxious, it presents strategies that include encouraging messages.

[1777] Input: optimal forest management strategy, user's emotional state

[1778] The server adjusts how the strategy is presented based on the emotion data.

[1779] Output: Coordinated management strategy

[1780] Step 8: Inform and implement your control strategy

[1781] The server transmits the created management strategy to the user's terminal, allowing the user to check the strategy through a visual interface.

[1782] Input: Coordinated management strategies

[1783] The server uses the notification function to send the strategy to the user's terminal.

[1784] Output: The control strategy displayed to the user

[1785] Step 9: Feedback on execution results

[1786] The user reports the results of the work they have performed to the system through a dedicated application, including photos of the work and a description of the situation.

[1787] Input: Results of forest management operations

[1788] The user sends the result data from the terminal to the system.

[1789] Output: Execution results sent to the server

[1790] Step 10: Evaluate and recalibrate your strategy

[1791] The server again receives the execution result data from the user, evaluates the effectiveness of the strategy, and readjusts the strategy as necessary based on the evaluation results.

[1792] Input: Execution result data

[1793] The server analyzes again and makes the next proposal.

[1794] Output: Retuned management strategy

[1795] Step 11: Notification of next proposal

[1796] The server notifies the user's terminal of the updated strategy and the next proposal, allowing the user to continuously carry out appropriate forest management.

[1797] Input: Recalibrated management strategies

[1798] The server will notify the user of the next offer.

[1799] Output: Next suggestion notified to the user

[1800] (Application example 2)

[1801] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1802] Operational management at a logistics center requires efficient placement and movement of goods and work procedures, which requires a great deal of effort and experience. Furthermore, worker emotions and fatigue levels have a significant impact on productivity and work efficiency, but current systems make it difficult to manage and adjust these factors. Therefore, there is a need for a system that can accurately grasp the location, type, and quantity of goods and automatically propose optimal work procedures.

[1803] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for receiving image data and video data acquired from a user, a means for analyzing the received image data and video data to identify the type, location, quantity, and presence of obstacles of items, and a means for generating an evaluation report based on the analyzed information and environmental data acquired from an external database. This makes it possible to accurately grasp the location, type, and quantity of items in a logistics center and automatically develop and present optimal operation and management strategies. Furthermore, by recognizing the user's emotional state in real time and adjusting the strategy presentation method, worker stress and fatigue can be reduced, improving work efficiency.

[1804] "Users" are the workers and managers who use the system to manage the operation of the logistics center.

[1805] "Image data and video data" refers to data recorded in a visually identifiable format that shows the items, their locations, and their status within a logistics center.

[1806] The "receiving means" is a function for transmitting image data and video data captured by the user to the server and capturing the data.

[1807] "Type of goods" is information indicating the category or classification of products or items present in the logistics center.

[1808] "Position" is coordinate information that indicates the location of an item or an obstacle within a logistics center.

[1809] "Quantity" is information indicating how many of a particular type of item are present in the logistics center.

[1810] An "obstacle" is any material or structure that may affect the movement of goods or the efficiency of work within a logistics center.

[1811] "Means for analyzing" refers to algorithms or software that processes received image and video data to identify the type, location, and quantity of items, and the presence of obstacles.

[1812] An "external database" is a database that exists outside the distribution center and can be accessed and used by the server.

[1813] "Environmental data" refers to environmental information that may affect the operation of a logistics center, such as temperature, humidity, and lighting conditions inside and outside the logistics center.

[1814] An "assessment report" is a report that shows the current status and risks of a logistics center, generated based on analyzed information and environmental data.

[1815] "Generative AI" is a system that uses artificial intelligence technology to create optimal operational management strategies based on evaluation reports.

[1816] "Operational management strategy" refers to specific work procedures and plans for achieving efficient operations within a logistics center.

[1817] The "means for transmitting and displaying" is a function for transmitting the created operation management strategy to the user's terminal and providing an interface for visually confirming the strategy.

[1818] The "execution results" are data indicating the status and deliverables after the user has performed the work.

[1819] The "means for evaluating effectiveness" is a function for analyzing and evaluating the effectiveness of the operation management strategy based on the user's execution results.

[1820] "Emotional state" is information that recognizes the psychological state and emotions of workers in real time.

[1821] The "means for adjusting the strategy presentation method" is a function for changing the content and presentation method of the presented operation management strategy according to the recognized emotional state of the worker.

[1822] To implement the present invention, each part of the system functions in the following steps.

[1823] 1. User's device

[1824] Users primarily use smartphones and tablets. These devices must have the following features:

[1825] Photography function: The user uses a camera to capture images of the situation inside the logistics center. Using this camera, the location and status of items are recorded as image and video data.

[1826] Data upload function: A dedicated application is used to upload captured image and video data to the server. Data can be sent quickly through this application.

[1827] Emotion recognition: Detects the user's emotional state in real time using the camera and microphone, using facial recognition and voice analysis technologies.

[1828] 2. Server

[1829] The server has the following features:

[1830] Data reception and storage function: The server receives image data, video data, and emotion data sent by users and stores them securely in a database. Storing this data is important for subsequent analysis and evaluation.

[1831] Image and video analysis function: The server is equipped with image analysis algorithms to analyze the received data, thereby identifying the type, location, and quantity of items within the logistics center, as well as the presence of obstacles.

[1832] Environmental data acquisition function: The server acquires environmental data from an external database, including temperature, humidity, lighting conditions, etc.

[1833] Assessment report generation function: The server integrates the analysis results and environmental data to generate an assessment report on the current status of the logistics center. This report clarifies the operational status and risks of the logistics center.

[1834] Strategy planning function using generative AI models: The server uses generative AI based on the evaluation report to automatically create optimal operation and management strategies, including material movement plans and work procedures.

[1835] Emotion-based strategy adjustment function: The server recognizes the user's emotional state and adjusts the way strategies are presented, thereby reducing stress and fatigue for workers and promoting efficient work.

[1836] 3. Specific examples of applications

[1837] Consider a case where this system is used to optimize the operation and management of a specific area of ​​a logistics center. A user takes a picture of the specific area using a smartphone and uploads it to a server using a dedicated application.

[1838] Example prompt sentence:

[1839] Take an image of a specific area in your warehouse and run a query to generate an optimal management strategy. Use the following image path:

[1840] Image path: 'warehouse_section.jpg'

[1841] The generative AI model suggests optimal work procedures based on the location, type, and quantity of items, as well as the emotional state of the worker."

[1842] The server analyzes the received image data to identify the location, type, and quantity of items, as well as the presence of obstacles. An assessment report is generated based on the analysis results and environmental data, and an optimal operational management strategy is then created using a generative AI model. Furthermore, the system takes into account the emotional state of the worker and adjusts the strategy presentation method to improve work efficiency and reduce stress.

[1843] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1844] Step 1:

[1845] Users take pictures of the situation inside the logistics center using a smartphone or tablet device. The captured image and video data are uploaded to the server via a dedicated application. The input here is image and video data, and the output is data sent to the server.

[1846] Step 2:

[1847] The server receives image and video data sent by users and stores them securely in a database. The input is the data received from the user, and the output is storing it in the database. This involves specific operations to verify the consistency and safety of the data.

[1848] Step 3:

[1849] The server analyzes the stored image and video data. It uses image analysis algorithms to identify the type, location, and quantity of objects, as well as the presence of obstacles. The input is image and video data, and the output is the analysis results, such as the type, location, and quantity of objects, and the presence of obstacles. Specific operations include image processing using libraries such as OpenCV.

[1850] Step 4:

[1851] The server retrieves environmental data (temperature, humidity, lighting conditions, etc.) inside and outside the distribution center from an external database. The input is a query to the external database, and the output is the retrieved environmental data. This step includes specific operations to retrieve data, such as through API calls.

[1852] Step 5:

[1853] The server integrates the analysis results and the acquired environmental data to generate an assessment report. The input is the analysis results of the item and the environmental data, and the output is the assessment report. Specific operations include data integration and report generation algorithms.

[1854] Step 6:

[1855] The server uses a generative AI model based on the evaluation report to create an optimal operations management strategy. This strategy includes a material movement plan and work procedures. The input is the evaluation report, and the output is the operations management strategy. Specific operations include a strategy planning process using the generative AI model.

[1856] Step 7:

[1857] The server obtains emotional data from the user's device in real time and adjusts the strategy to be presented. The input is emotional data, and the output is an operational management strategy adjusted based on the emotion. Specific operations include obtaining emotional data through facial recognition and voice analysis and adjusting the strategy.

[1858] Step 8:

[1859] The server sends the created operation management strategy to the user's terminal and displays it through a visual interface. The input is the operation management strategy, and the output is the strategy information displayed on the user's terminal. Specific operations include data transmission and user interface display.

[1860] Step 9:

[1861] The user performs specific tasks based on the presented operation management strategy. The input is the operation management strategy, and the output is the work results. Specific actions include moving and organizing items within the warehouse and executing work procedures.

[1862] Step 10:

[1863] The user reports the results of their work to the server through a dedicated application. The input is the data of the work result, and the output is the transmission of data to the server. This includes uploading images and reports after the work.

[1864] Step 11:

[1865] The server receives and analyzes the execution result data sent by the user, thereby evaluating the effectiveness of the operation management strategy and readjusting the strategy as necessary. The input is the execution result data, and the output is an updated operation management strategy. Specific operations include data analysis and strategy readjustment.

[1866] Step 12:

[1867] The server notifies the user terminal of the updated strategy and the next proposal. The input is the updated management strategy, and the output is the notification information. Specific operations include the execution of the notification mechanism.

[1868] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 t...

Claims

1. means for receiving image data and video data acquired from a user; means for analyzing the received image data and video data to identify the type, density, growth stage, and presence of pests and diseases of trees; a means for generating a forest assessment report based on the analyzed information and climate data obtained from the meteorological database; A means to create optimal forest management strategies using AI generated based on the assessment report, and means for transmitting the created forest management strategy to a user's terminal and displaying it; and means for receiving the user's implementation results again and evaluating the effectiveness of the forest management strategy.

2. The system of claim 1 provides a forest management strategy including optimal felling plans, planting plans, and timing of care.

3. The system according to claim 1, further analyzing the results of the user's execution and making the next proposal.

Images