system

Abstract

Provide a system. A means for receiving satellite images and images captured by an unmanned aerial vehicle; means for preprocessing the received image data; means for analyzing the preprocessed image data to identify forest health and tree species; A means for determining optimal felling and planting locations based on the identified data; A means for visualizing the determined felling and planting locations on a map; A means of receiving user feedback and revising the plan; A means of providing revised plans to users and managing their execution and progress; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, forest felling and tree planting planning has become increasingly important from an environmental protection perspective. However, current methods require a lot of time and effort, and lack accuracy. Furthermore, it is extremely difficult to manually determine the appropriate felling and tree planting points and timing, making it difficult to create optimal plans that take sustainability and environmental protection into consideration. For this reason, there is a need for a system that automates efficient and accurate felling and tree planting planning. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means: a system including a means for receiving satellite images and images captured by unmanned aerial vehicles, a means for preprocessing the received image data, a means for analyzing the preprocessed image data to identify the forest health condition and tree species, a means for determining optimal logging and planting sites based on the identified data, a means for visualizing the determined logging and planting sites on a map, a means for receiving feedback from a user and revising the plan, and a means for providing the revised plan to the user and managing its execution and progress. Furthermore, by providing a means for determining optimal timing for logging and planting, taking environmental protection and sustainability into consideration, and a means for receiving additional data from the user and readjusting the plan, effective and sustainable forest management is realized.

[0006] "Satellite imagery" refers to images of the Earth's surface taken by artificial satellites, and provides wide-ranging geographical information and environmental data.

[0007] An "unmanned aerial vehicle" is an aircraft that flies under remote or automatic control and is used for a variety of purposes, including aerial photography and videography.

[0008] "Image data" refers to visual information stored in digital form, and includes various forms of digital images such as photographs and videos.

[0009] "Preprocessing" refers to the initial stage of data processing that takes place before data analysis or input into a machine learning model, and includes techniques such as noise removal and normalization.

[0010] "Forest health" is an indicator that shows the health of a forest's ecosystem, such as the state of vegetation growth and the extent of damage caused by pests and diseases.

[0011] "Tree types" are classifications of trees belonging to different botanical species, which determine the characteristics and uses of each type of tree.

[0012] "Harvesting site" refers to a location selected for felling trees growing in a particular area.

[0013] "Planting site" means a location selected for planting new trees.

[0014] "Map visualization" refers to the visual display of data or plans using geographic information systems or other map applications.

[0015] "Feedback" refers to opinions and requests for corrections provided by users, and is important input information for the system to create an optimal plan.

[0016] "Progress management" refers to the process of monitoring the execution of a plan and checking and managing whether it is progressing as planned.

[0017] "Environmental protection" refers to actions and strategies to protect and preserve the natural environment and maintain ecological sustainability.

[0018] "Sustainability" refers to the proper management of natural resources and keeping them available for future generations.

[0019] "Timing" refers to the time or period selected for a particular action or event. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] This invention is a system for planning and executing efficient and highly accurate forest felling and tree planting plans. The specific details of this system are described below.

[0042] System Overview

[0043] The system is primarily composed of a server and a terminal, and is operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal functions as an interface connecting the user and the server. The user operates the system via the terminal to collect data, confirm plans, and execute them.

[0044] Data collection and preprocessing

[0045] Users take aerial photographs (including satellite images) of forests using drones or unmanned aerial vehicles and store the image data on their devices. The devices then upload the image data to a server, which then performs preprocessing on the received image data. The preprocessing includes noise removal, normalization, and geometric transformation.

[0046] Data analysis and feature extraction

[0047] The server analyzes the preprocessed image data and extracts features such as forest health, tree species, and the location of water sources. This analysis uses machine learning models (e.g., convolutional neural networks). The server stores these features in a database and generates a risk map.

[0048] Planning of felling and planting trees

[0049] The server uses the risk map to identify optimal felling and planting locations. It considers the extent of diseased trees when identifying felling locations. It also considers soil data, rainfall information, and past success stories when selecting planting locations. It also uses seasonal information and rainfall forecasts to determine the optimal timing for felling and planting.

[0050] Plan visualization and feedback

[0051] The server sends the plan results to the device, which displays them visually on a map. The user provides feedback on the displayed plan, including specific instructions such as "I would like to postpone cutting trees in this area." The device then sends this feedback to the server, which then modifies the plan.

[0052] Implementing the plan and managing progress

[0053] Once the user has finalized the plan, the server will manage the progress based on this plan. The user will periodically report the execution status to the server via their terminal. The server will monitor the progress based on this data and adjust the plan as necessary.

[0054] Specific examples

[0055] For example, consider a user planning a tree planting project. First, the user takes aerial photos of the forest using a drone and uploads the image data from their device to a server. The server receives the data and pre-processes it. Next, the server analyzes it using machine learning models to identify soil quality and the health of existing trees. It then selects optimal planting locations and displays them on a map. The user reviews the plan and provides feedback. Finally, the user executes the plan and periodically reports progress to the server.

[0056] This system enables users to achieve efficient and sustainable forest management.

[0057] The processing flow will be explained below.

[0058] Program processing flow

[0059] Step 1: Collect data

[0060] Users take aerial and ground photographs of forests using drones or unmanned aerial vehicles, save the image data on their devices, and then upload it to a server via their devices.

[0061] Step 2: Receiving and Preprocessing Data

[0062] The server receives image data uploaded from the device. The server performs preprocessing such as noise removal, normalization, and geometric transformation on the received data. In this process, image distortion is corrected, improving the accuracy of analysis.

[0063] Step 3: Image analysis and feature extraction

[0064] The server applies machine learning models (e.g., convolutional neural networks) to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources. The server stores the analysis results in a database.

[0065] Step 4: Generate a risk map

[0066] The server generates a risk map based on the extracted features, visually indicating the extent of diseased trees, areas that need to be cut down, and areas where tree planting is appropriate.

[0067] Step 5: Identifying the harvest site

[0068] The server runs an algorithm based on the risk map to identify optimal felling sites, taking into account necessary environmental protection criteria such as the extent of diseased trees, and stores the list of felling sites.

[0069] Step 6: Select a planting site

[0070] The server selects the optimal planting location based on soil data, rainfall information, and past examples of successful planting. The server then displays the latitude and longitude information of the planting location overlaid on map data.

[0071] Step 7: Generate a time schedule

[0072] The server uses rainfall forecasts and seasonal information to determine the optimal timing for tree cutting and planting, and generates a time schedule that is recorded in a database.

[0073] Step 8: Visualize the plan

[0074] The server sends the optimized felling and planting plan along with map data to the terminal, which visualizes the plan on a map and displays it to the user.

[0075] Step 9: Gather user feedback

[0076] The user inputs feedback about the displayed plan via the terminal. For example, the user inputs an instruction such as "I would like to postpone the felling of this area." The terminal then sends the feedback to the server.

[0077] Step 10: Incorporate feedback and revise your plan

[0078] The server reconstructs the plan based on the received feedback, and the revised plan is sent back to the device and displayed to the user.

[0079] Step 11: Execute the plan

[0080] The user reviews the final plan and approves it for execution. The device sends the execution status to the server, allowing the progress of the plan to be monitored in real time.

[0081] Step 12: Monitor progress and improve

[0082] The server records the progress of the executed tasks in a database and adjusts the plan again if necessary. The user uploads additional data (e.g., information on new disease outbreaks) from their device to the server, and a new analysis is performed.

[0083] This is the flow of the program processing for this system, which enables users to achieve efficient and sustainable forest management based on data.

[0084] Example 1

[0085] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0086] Conventional forest management systems have faced the challenge of making efficient and accurate plans for felling and planting. Specifically, there was little centralized analysis of image data obtained from satellite images and aerial photography, selection of optimal felling and planting locations, or feedback and revision of plans. This resulted in a decrease in the accuracy of plans and the efficiency of their execution, making it difficult to achieve sustainable forest management.

[0087] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0088] In this invention, the server includes means for receiving satellite images and images captured by an aerial imaging device, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify forest health, tree species, and water source locations, means for determining optimal felling and planting sites based on the identified data, means for visualizing the determined felling and planting sites on a map, means for receiving feedback from a user and revising the plan, means for providing the revised plan to the user and managing its execution and progress, means for analyzing the image data using a machine learning model, and means for receiving additional data from the user and readjusting the plan, thereby enabling the efficient and accurate planning and execution of felling and planting plans.

[0089] "Satellite imagery" refers to images taken by satellites orbiting the Earth, and is data that provides information on a wide range of the Earth's surface.

[0090] An "aerial photography device" is a device, such as a drone or unmanned aerial vehicle, used to take photographs or videos of the ground from a high altitude.

[0091] "Preprocessing" refers to processes such as noise removal, normalization, and geometric transformation that are performed on image data to be analyzed, and is a process for improving the quality of the data.

[0092] "Forest health" refers to the overall state of the forest ecosystem, including the state of plant growth and the occurrence of pests and diseases.

[0093] "Tree species" refers to the types of trees that grow in a particular area or forest.

[0094] "Location of water sources" refers to location information of rivers, lakes, and other places where water exists within a forest.

[0095] "Optimal harvesting site" refers to a suitable location chosen to harvest trees efficiently and sustainably.

[0096] "Optimal planting site" refers to the suitable location selected for planting new trees.

[0097] "Visualization" refers to the graphical display of data and planning results on a map.

[0098] "Feedback" refers to information provided by users to modify and improve the plan through their opinions and instructions.

[0099] A "machine learning model" refers to an algorithm or model that learns patterns and features from data and predicts and analyzes future data.

[0100] "Additional data" refers to new information or data provided by the user that serves as the basis for revising or readjusting the plan.

[0101] "Progress" refers to the progress of checking the execution status and degree of achievement of a plan.

[0102] This invention is a system for planning and executing efficient and highly accurate forest felling and tree planting plans. The specific details of this system are described below.

[0103] The system is primarily composed of a server and a terminal, and is operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal functions as an interface connecting the user and the server. The user operates the system via the terminal to collect data, confirm plans, and execute them.

[0104] Data collection and preprocessing

[0105] Users use aerial photography equipment such as drones to take aerial or satellite images of forests and store the image data on their devices. The devices then upload the image data to a server. A high-speed Internet connection is required for uploading, and the HTTPS protocol is used. The server then performs preprocessing on the received image data. This preprocessing includes noise removal, normalization, and geometric transformation. OpenCV can be used for this.

[0106] Data analysis and feature extraction

[0107] The server analyzes the preprocessed image data and extracts features such as forest health, tree species, and the location of water sources. This analysis uses a convolutional neural network (CNN), a machine learning model. Specifically, Tensorflow (registered trademark) and PyTorch are used. The server stores these features in a database and generates a risk map. QGIS can be used to visualize the risk map.

[0108] Planning of felling and planting trees

[0109] The server identifies optimal felling and planting locations based on the risk map. The location of felling locations is determined by taking into account the extent of diseased trees and other risk factors. The planting locations are selected based on soil data, rainfall information, and past success stories. Seasonal information and rainfall forecast data are also used to determine the optimal timing for felling and planting. This process utilizes environmental databases and weather data APIs.

[0110] Plan visualization and feedback

[0111] The server sends the proposed tree-cutting and planting plan to the device, which then visually displays it on a map. Open-source JavaScript (registered trademark) libraries such as OpenLayers and Leaflet are used to display the map. The user checks the displayed plan and provides feedback such as "I would like to postpone tree-cutting in this area." The feedback is sent via the device to the server, which then modifies the plan.

[0112] Implementing the plan and managing progress

[0113] Once the user has finalized the plan, the server will manage progress based on this plan. The user periodically reports the execution status to the server via their terminal. The reports include the progress and any issues. The server will monitor the progress based on this data and readjust the plan as necessary. Project management tools such as Asana and JIRA can be used to manage progress.

[0114] Examples of specific examples and prompts

[0115] For example, consider a case where a user is creating a tree planting plan. The user first uses a drone to take aerial photos of the forest and uploads them from their device to the server. The server preprocesses the received data using OpenCV and then uses TensorFlow to identify soil quality and the health of existing trees. It then generates a risk map using QGIS. The server selects optimal planting sites and sends the plan to the device. The device displays the plan on the map, and the user provides feedback, such as "I would like to postpone tree felling in this area." This feedback is sent to the server through the Django framework, which then modifies the plan. Finally, the user finalizes the plan and periodically reports the progress of the work to the server. The server monitors the progress using Asana.

[0116] Example prompt sentence:

[0117] “You upload aerial photos of your forest. We preprocess the data and analyze it using machine learning models. We identify the best locations for cutting and planting trees and modify the plan based on your feedback.”

[0118] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0119] Step 1:

[0120] Data Acquisition

[0121] Users use drones or unmanned aerial vehicles to take aerial photographs or satellite images of forests, and the captured image data is stored in the drone's internal storage.

[0122] Specific actions

[0123] To take aerial photos of a forest, a user launches the drone and sets a designated flight path. The drone then flies according to the settings and takes photos of the designated area with its high-resolution camera.

[0124] Input and Output

[0125] Input: Flight path, shooting instructions

[0126] Output: Aerial photographs taken

[0127] Step 2:

[0128] Saving image data

[0129] The user transfers the captured image data from the drone to a device and stores it on a high-performance SSD.

[0130] Specific actions

[0131] The user connects a data transfer cable from the drone to the device and downloads the image data to the device, where it is saved in the device's storage.

[0132] Input and Output

[0133] Input: Image data from inside the drone

[0134] Output: Image data in the device

[0135] Step 3:

[0136] Uploading image data

[0137] The device uploads the stored image data to a server over a high-speed internet connection using the HTTPS protocol.

[0138] Specific actions

[0139] The user uploads image data to the server using a dedicated application on the device, and the progress of the data transfer is displayed until it is complete.

[0140] Input and Output

[0141] Input: Image data in the device

[0142] Output: Image data uploaded to the server

[0143] Step 4:

[0144] Image data preprocessing

[0145] The server performs preprocessing such as noise reduction, normalization, and geometric transformation on the received image data using OpenCV.

[0146] Specific actions

[0147] The server first applies a Gaussian filter to remove noise from the image, then scales the image's pixel values ​​to the range 0 to 1, and then performs a geometric transformation to adjust the image's angle and scale.

[0148] Input and Output

[0149] Input: Image data uploaded to the server

[0150] Output: Preprocessed image data

[0151] Step 5:

[0152] Image data analysis and feature extraction

[0153] The server then feeds the preprocessed image data into a machine learning model (e.g., a convolutional neural network) to extract features such as forest health, tree species, and the location of water sources. TensorFlow is used for the analysis.

[0154] Specific actions

[0155] The server runs the preprocessed image data through the TensorFlow environment and applies a trained neural network model that identifies forest health and other important features from the input image and outputs the results in a list format.

[0156] Input and Output

[0157] Input: Preprocessed image data

[0158] Output: Extracted feature data

[0159] Step 6:

[0160] Saving to the database and generating a risk map

[0161] The server stores the extracted feature data in a database and generates a risk map, which is visualized using QGIS.

[0162] Specific actions

[0163] The server inserts the extracted feature data into a database, runs the risk map generation algorithm, and uses QGIS to generate the risk map and save the results in a map format.

[0164] Input and Output

[0165] Input: extracted feature data

[0166] Output: Risk map

[0167] Step 7:

[0168] Deciding on felling and planting locations

[0169] The server determines the optimal locations for cutting and planting trees based on the risk map, taking into account seasonal information and rainfall forecast data.

[0170] Specific actions

[0171] The server takes into account the risk map along with additional environmental data (e.g., rainfall forecasts and seasonal information) and uses algorithms to identify optimal felling and planting locations. The results are stored in a list format and can be displayed later.

[0172] Input and Output

[0173] Input: Risk map, environmental data

[0174] Output: List of optimal cutting and planting locations

[0175] Step 8:

[0176] Sending planning results

[0177] The server sends the proposed tree-cutting and planting plan to the terminal, which includes geographic information and integrates it into the terminal's map system.

[0178] Specific actions

[0179] The server uses a dedicated API to send the list of determined felling and planting locations to the device, and after the sending process is completed, the device sends a message confirming receipt.

[0180] Input and Output

[0181] Input: List of optimal felling and planting sites

[0182] Output: Planning data sent to the terminal

[0183] Step 9:

[0184] Plan visualization

[0185] The device visually displays the received plan on a map, using open source JavaScript libraries such as OpenLayers and Leaflet to achieve detailed map display.

[0186] Specific actions

[0187] The terminal activates the geographic information system, imports the received planning data, and marks the planned felling and planting locations on a map and displays detailed information for the user's convenience.

[0188] Input and Output

[0189] Input: Planning data

[0190] Output: felling and planting plans displayed on a map

[0191] Step 10:

[0192] User Feedback

[0193] The user checks the displayed plan and provides feedback such as "I would like to postpone the felling of this area." The feedback is sent to the server via the terminal, and the server then modifies the plan.

[0194] Specific actions

[0195] The user checks the plan on the map, selects the part they want to correct, enters the feedback into the terminal, and presses the send button, which sends the correction request to the server.

[0196] Input and Output

[0197] Input: User feedback

[0198] Output: Feedback sent to the server

[0199] Step 11:

[0200] Revise and resubmit the plan

[0201] The server modifies the plan based on feedback from the user and retransmits the newly modified plan to the terminal.

[0202] Specific actions

[0203] The server analyzes the received feedback, applies a plan modification algorithm, and once the modification is complete, sends the plan back to the device.

[0204] Input and Output

[0205] Input: User feedback

[0206] Output: Corrected planning data

[0207] Step 12:

[0208] Implementing the plan and managing progress

[0209] Once the user has finalized the plan, the server will manage the progress based on this plan. The user will periodically report the execution status to the server via their terminal.

[0210] Specific actions

[0211] The user inputs the progress of the work into the terminal and presses the send button to report it to the server, which analyzes the received progress data and updates the progress status.

[0212] Input and Output

[0213] Input: Work progress data

[0214] Output: Latest progress data

[0215] Step 13:

[0216] Monitor progress and readjust

[0217] The server monitors the progress of the work and readjusts the plan as needed based on progress data reported by the user.

[0218] Specific actions

[0219] The server monitors progress data in real time, immediately issuing warnings if there are any problems with the progress or if new risks arise, and also makes any necessary corrections to the plan and notifies the device again.

[0220] Input and Output

[0221] Input: Work progress data

[0222] Output: Realigned planning data

[0223] The above are the processing steps of the program for this system. The techniques and specific operations used in each step have been explained in detail.

[0224] (Application example 1)

[0225] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0226] Conventional factory layout optimization systems are operated based on fixed layout designs and plans, making it difficult to change or optimize in real time. Furthermore, there was a lack of means to instantly understand and correct the efficiency of ongoing work in the actual environment. This resulted in a decline in work efficiency, making it difficult to improve productivity.

[0227] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0228] In this invention, the server includes means for receiving satellite images and images captured by unmanned aerial vehicles, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify the environmental health status and plant species, means for determining optimal work locations and layout plans based on the identified data, means for visualizing the determined work locations and layout plans on a map, means for receiving feedback from a user and revising the plan, means for providing the revised plan to the user and managing execution and progress, and means for monitoring the situation in real time using smart glasses and visually confirming the optimal layout, thereby enabling the optimal factory layout to be confirmed and revised in real time.

[0229] "Satellite imagery" refers to image data taken from Earth's satellites and is used to obtain information on the terrain and environment over a wide area.

[0230] An "unmanned aerial vehicle" is a remotely controlled or autonomously flown aircraft equipped with a specialized camera used to take high-resolution aerial photographs.

[0231] "Image data preprocessing" is a method of performing initial processing such as noise removal, normalization, and geometric transformation on acquired image data, and converting it into a format suitable for analysis.

[0232] "Environmental health" refers to the overall assessment of factors and elements (e.g., plant health, water quality) in a particular natural or man-made environment.

[0233] A "plant type" refers to a classification of plants present in a particular area, identified based on biological categories such as species, genus, or family.

[0234] "Work location" refers to the geographic location that is best suited to carrying out a particular task (e.g., cutting trees or planting trees).

[0235] A "layout plan" is a blueprint or scheme for a field, factory, etc., for planning and devising the optimal placement of specific tasks or machinery.

[0236] "Visualization" is the technique of visually displaying data and information, and converting it into a format that is easily understood and usable by users.

[0237] "Feedback" refers to the process of receiving opinions and requests from users and reflecting them in systems and plans.

[0238] "Smart glasses" are wearable devices that use augmented reality (AR) technology and are glasses-type devices that can display and operate information in real time.

[0239] This invention is a system for efficiently and accurately optimizing factory layouts. This system can improve work efficiency and productivity in real time. Specifically, optimization is performed through collaboration between servers, terminals, and users.

[0240] System configuration

[0241] The system mainly consists of the following components:

[0242] 1. Hardware

[0243] Server: A central processing unit that performs data analysis and planning.

[0244] Smart glasses: Used for real-time situation monitoring and visualization of optimization plans.

[0245] Unmanned aerial vehicle: Used to photograph the layout of the factory.

[0246] 2. Software

[0247] Python: Used for data analysis and running machine learning models.

[0248] OpenCV: A library for image processing.

[0249] Keras: Used to build and manage machine learning models.

[0250] Django: A web application framework.

[0251] Data collection and preprocessing

[0252] Users use unmanned aerial vehicles to take high-resolution images of their factory interiors and upload them to a server via their terminal. The server then performs preprocessing such as noise removal and normalization on the received image data, converting it into a format suitable for analysis. This removes noise from the image data and improves the accuracy of the analysis.

[0253] Data Analysis and Optimization

[0254] The server analyzes the preprocessed image data to determine the layout of equipment and work flow within the factory. A pre-trained machine learning model (generative AI model) is used for the analysis. This model extracts features from the input image data, such as the health status of the environment, the type of equipment, and its location. Based on these extracted features, the server then generates an optimal layout plan and work flow.

[0255] Visualization and feedback of optimization plans

[0256] The optimization plan generated by the server is displayed on the smart glasses. The user can check it in real time and provide feedback as needed. This feedback is sent from the device to the server, which then modifies the plan based on the received feedback. For example, specific instructions such as "I would like to change the equipment layout in this area" can be sent as feedback.

[0257] Implementing the plan and managing progress

[0258] Once the user has finalized the plan, the server will manage the progress based on this plan. Real-time monitoring and feedback makes it easy to revise and readjust the plan, improving work efficiency.

[0259] Specific examples

[0260] For example, consider the case of optimizing the layout of a factory. A user uses an unmanned aerial vehicle to take images of the inside of the factory and uploads this image data from their device to a server. The server preprocesses the image data and analyzes it using a machine learning model to generate an optimal layout plan and work flow. The optimized plan is then presented to the user via smart glasses. The user provides feedback on the plan while checking the actual situation on the site, and the server then modifies the plan, achieving optimization.

[0261] Example prompt sentence:

[0262] "Please build a system that can optimize the acquired factory layout images and visually confirm the results of the optimal layout."

[0263] This system enables the factory layout to be optimized in real time, establishing an efficient and sustainable production system.

[0264] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0265] Step 1:

[0266] The user uses an unmanned aerial vehicle to take high-resolution images of the factory and saves the image data on a terminal. The input is the image data taken by the unmanned aerial vehicle, and the output is an image file saved on the terminal.

[0267] Step 2:

[0268] A user uploads image data to a server via a terminal. The input is an image file stored on the terminal, and the output is the image data sent to the server.

[0269] Step 3:

[0270] The server performs preprocessing on the received image data. Specifically, it performs noise removal, normalization, and geometric transformation on the image data. The input is the image data sent to the server, and the output is the preprocessed image data.

[0271] Step 4:

[0272] The server analyzes the preprocessed image data and identifies the equipment layout and work flow within the factory. This analysis uses a machine learning model (generative AI model). The input is the preprocessed image data, and the output is the equipment layout and work flow data as the analysis results.

[0273] Step 5:

[0274] The server then formulates an optimal layout plan based on the generated equipment placement and work flow data. The input is the analysis results data, and the output is an optimized layout plan.

[0275] Step 6:

[0276] The server provides the optimized layout plan to the user through the smart glasses, where the input is the optimized layout plan and the output is the visual plan displayed on the smart glasses.

[0277] Step 7:

[0278] The user can use the smart glasses to check the layout plan in real time and provide feedback as needed. The input is the feedback information from the user, and the output is the feedback data sent to the server via the terminal.

[0279] Step 8:

[0280] The server modifies the layout plan based on the received user feedback and performs re-optimization. At this time, a new layout plan that reflects the user's opinions is generated. The input is the feedback data, and the output is the modified layout plan.

[0281] Step 9:

[0282] The user checks and approves the final layout plan. An example of a prompt is, "Optimize the acquired factory layout image and build a system that allows visual confirmation of the results of the optimal layout." The input is the revised layout plan, and the output is the approved final layout plan.

[0283] Step 10:

[0284] The server manages progress based on the final approved layout plan and provides a system where users can report the execution status of work in real time. The input is real-time execution status data, and the output is a work schedule with appropriately managed progress.

[0285] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0286] This invention is a system for planning and executing efficient and accurate forest felling and tree planting plans, and by combining it with an emotion engine that recognizes the user's emotions, it provides a more user-friendly interface.

[0287] System Overview

[0288] The system is primarily comprised of a server and a terminal, operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal acts as an interface between the user and the server. The emotion engine recognizes user emotions in real time and is used to optimize the user experience.

[0289] Data collection and preprocessing

[0290] Users use drones or unmanned aerial vehicles to take aerial and ground photographs of forests and store the image data on their devices. The users then upload the data to the server via their devices. The server then performs preprocessing on the received image data, including noise reduction, normalization, and geometric transformation.

[0291] Data analysis and feature extraction

[0292] The server applies machine learning models to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources, generating a risk map. The server then stores the analysis results in a database.

[0293] Planning of felling and planting trees

[0294] The server identifies optimal felling and planting locations based on a risk map. The server takes into account factors such as the extent of diseased trees when identifying felling locations. Soil data, rainfall information, and past success stories of plantings are taken into account when selecting planting locations. Additionally, the server uses seasonal information and rainfall forecasts to determine the optimal timing for felling and planting.

[0295] Emotion engine integration

[0296] The emotion engine analyzes the user's facial expressions and tone of voice in real time on the device to recognize the user's emotional state. For example, if the user looks anxious, the system will provide the user with more detailed explanations and advice. If the user looks satisfied, the system will determine that the proposed plan is acceptable.

[0297] Plan visualization and feedback

[0298] The server sends the optimized felling and planting plan along with map data to the device. The device visualizes the plan on a map and displays it to the user. The user provides feedback on the plan, which the device then sends to the server. The emotion engine optimizes the content and presentation of the feedback based on the user's emotions.

[0299] Implementing the plan and managing progress

[0300] Once the user finalizes the plan, the server manages progress based on this plan. The user periodically reports their execution status to the server via their device. The server monitors progress based on this and readjusts the plan as necessary. The emotion engine adjusts notifications and alarms at appropriate times when the user is feeling stressed.

[0301] Specific examples

[0302] For example, when a user creates a tree planting plan, they first use a drone to take aerial photos of the forest and upload the image data from their device to the server. The server receives the data and pre-processes it. Next, it uses a machine learning model to analyze it and identify the soil quality and the health of existing trees. The optimal planting site is then selected and displayed on a map. When the user reviews the plan and enters feedback, the emotion engine analyzes the user's emotional state and optimizes the feedback content. Finally, when the user executes the plan and reports progress to the server, the emotion engine notifies them at the appropriate time and helps manage progress.

[0303] In this way, the system enables efficient and sustainable forest management based on data, while providing an easy-to-use interface that takes into account the user's emotional state.

[0304] The processing flow will be explained below.

[0305] Program processing flow

[0306] Step 1: Collect data

[0307] Users take aerial and ground photographs of forests using drones or unmanned aerial vehicles, save the image data on their devices, and then upload it to a server via their devices.

[0308] Step 2: Receiving and Preprocessing Data

[0309] The server receives image data uploaded from the device. The server performs preprocessing such as noise removal, normalization, and geometric transformation on the received data. This removes distortion and noise from the image, making it easier to analyze.

[0310] Step 3: Image analysis and feature extraction

[0311] The server applies machine learning models (e.g., convolutional neural networks) to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources. The server stores the analysis results in a database.

[0312] Step 4: Generate a risk map

[0313] The server generates a risk map based on the extracted features, visually indicating the extent of diseased trees, areas that need to be cut down, and areas where tree planting is appropriate.

[0314] Step 5: Identifying the harvest site

[0315] The server runs an algorithm based on the risk map to identify optimal logging sites, taking into account the extent of diseased trees and other environmental protection criteria. The server then stores the list of logging sites.

[0316] Step 6: Select a planting site

[0317] The server selects the optimal planting location based on soil data, rainfall information, and past examples of successful planting. The server then displays the latitude and longitude information of the planting location overlaid on map data.

[0318] Step 7: Generate a time schedule

[0319] The server uses rainfall forecasts and seasonal information to determine the optimal timing for tree cutting and planting, and generates a time schedule that is recorded in a database.

[0320] Step 8: Recognizing user emotions with the emotion engine

[0321] The device analyzes the user's facial expressions and tone of voice in real time to recognize their emotional state, and when the user shows a certain emotion, it optimizes the feedback method according to that emotion.

[0322] Step 9: Visualize the plan

[0323] The server sends the optimized tree-cutting and planting plan along with map data to the device. The device visualizes the plan on the map and displays it to the user. The emotion engine analyzes the user's reaction and displays additional explanations or encouraging messages if necessary.

[0324] Step 10: Gather user feedback

[0325] The user inputs feedback about the displayed plan via the device. For example, they can enter specific instructions such as "I would like to postpone the felling of trees in this area." The device then sends the feedback to the server. The emotion engine analyzes the user's emotions and presents the feedback in an easy-to-understand format.

[0326] Step 11: Incorporate feedback and revise your plan

[0327] The server reconstructs the plan based on the received feedback. The revised plan is sent back to the device and displayed to the user. The emotion engine generates a response based on the user's emotions, encouraging the user to provide positive feedback.

[0328] Step 12: Execute the plan

[0329] The user reviews the final plan and approves it for execution. The device sends the execution status to the server, monitoring the progress of the plan in real time. The emotion engine displays encouraging messages at appropriate times to reduce the user's stress level.

[0330] Step 13: Monitor progress and improve

[0331] The server records the progress of the executed tasks in a database and adjusts the plan again if necessary. The user uploads additional data (e.g., information on new disease outbreaks) from their device to the server for new analysis. The emotion engine monitors the user's emotional state and provides appropriate feedback and responses.

[0332] The above is the specific flow of the program processing of this system, which enables users to achieve efficient and sustainable forest management based on data, while also increasing user satisfaction through the emotion engine.

[0333] Example 2

[0334] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0335] Conventional forest management systems lack the mechanisms for formulating and implementing efficient and accurate forest harvesting and planting plans. In particular, they lack the ability to adjust plans to take users' emotions into account, making it difficult to reduce their stress and anxiety. This poses a risk of reducing the sustainability and efficiency of forest management.

[0336] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0337] In this invention, the server includes means for receiving satellite images and images captured by unmanned aerial vehicles, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify the forest health and tree species, means for determining optimal felling and planting sites based on the identified data, means for visualizing the determined felling and planting sites on a map, means for analyzing the user's facial expressions and tone of voice in real time to recognize the user's emotional state, means for optimizing the interface in accordance with the recognized emotional state, receiving feedback from the user, and revising the plan, and means for providing the revised plan to the user and managing its execution and progress. This enables efficient and accurate forest management, and by taking the user's emotional state into consideration, it is possible to provide a user-friendly interface and reduce stress and anxiety.

[0338] "Satellite imagery" is image data taken of the Earth's surface from space.

[0339] An "unmanned aerial vehicle" is an aircraft that is operated by remote control or automatic pilot and has no personnel on board.

[0340] "Preprocessing" refers to the process of converting received data into a format that is easy to analyze and use, and specifically includes noise removal, normalization, geometric transformation, etc.

[0341] "Analysis" is the process of examining and breaking down data in detail to clarify its content and structure.

[0342] "Forest health" is an indicator of how healthy a forest is, including the growth status of trees and the presence of pests and diseases.

[0343] "Tree type" is a classification of the types of trees present in a particular forest.

[0344] "Cutting point" means [a specific location in the forest where trees should be cut].

[0345] "Planting site" means [a suitable location for planting new trees].

[0346] "Visualization" is a technique that makes data and information easier to understand by displaying them visually.

[0347] "Emotional state" refers to the user's current psychological and emotional state.

[0348] "Interface optimization" refers to adjusting the layout and functionality of an interface to improve the user experience.

[0349] "Feedback" is the process of feeding user reactions and opinions back into the system.

[0350] "Progress management" means monitoring the progress of plans and work and making adjustments or corrections as necessary.

[0351] This invention is a system for planning and executing efficient and accurate forest felling and planting plans, and by combining it with an emotion engine that recognizes user emotions, it provides a more user-friendly interface. The system is primarily composed of a server and a terminal, and is operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal functions as an interface connecting the user and the server. The emotion engine recognizes user emotions in real time and is used to optimize the user experience.

[0352] Data collection and preprocessing

[0353] Users use drones or unmanned aerial vehicles to take aerial and ground photographs of forests and store the image data on their devices. The users then upload the data to the server via their devices. The server then performs preprocessing on the received image data, including noise removal, normalization, and geometric transformation. OpenCV is used for noise removal, and Scikit-Image is used for normalization and geometric transformation.

[0354] Data analysis and feature extraction

[0355] The server applies machine learning models to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources, generating a risk map. The analysis is performed using TensorFlow and PyTorch, and the results are stored in a database.

[0356] Planning of felling and planting trees

[0357] The server uses the risk map to identify optimal felling and planting locations. The server takes into account factors such as the extent of diseased trees when identifying felling locations. Soil data, rainfall information, and past success stories of plantings are taken into account when selecting planting locations. The server also uses seasonal information and rainfall forecasts to determine the optimal timing for felling and planting. This is done using geographic information systems such as ArcGIS and QGIS.

[0358] Emotion engine integration

[0359] The emotion engine analyzes the user's facial expressions and tone of voice in real time on the device to recognize the user's emotional state. For example, it uses Microsoft® Azure® Cognitive Services and Affectiva. If the user looks anxious, the system will provide the user with more detailed explanations and advice.

[0360] Plan visualization and feedback

[0361] The server sends the optimized felling and planting plans along with map data to the device. The device visualizes the plans on a map and displays them to the user. For example, the device receives the data in GeoJSON format and visualizes it in a dedicated map app. The user provides feedback on the plans, which the device then sends to the server. The emotion engine optimizes the content and presentation of the feedback based on the user's emotions.

[0362] Implementing the plan and managing progress

[0363] Once the user finalizes the plan, the server manages progress based on this plan. The user periodically reports their execution status to the server via their device. The server monitors progress based on this and readjusts the plan as necessary. The emotion engine adjusts notifications and alarms at appropriate times when the user is feeling stressed.

[0364] Specific examples

[0365] For example, when a user creates a tree planting plan, they first use a drone to take aerial photos of the forest and upload the image data from their device to the server. The server receives the data and pre-processes it. Next, it uses a machine learning model to analyze it and identify the soil quality and the health of existing trees. The optimal planting site is then selected and displayed on a map. When the user reviews the plan and enters feedback, the emotion engine analyzes the user's emotional state and optimizes the feedback content. Finally, when the user executes the plan and reports progress to the server, the emotion engine notifies them at the appropriate time and helps manage progress.

[0366] Example prompt sentence:

[0367] "Please analyze aerial photos of the forest taken by a drone, identify the best locations for planting trees, and display the plan on a map. You should also develop a system that analyzes the user's facial expressions and provides appropriate feedback."

[0368] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0369] Step 1: Data collection and upload

[0370] A user takes aerial and ground photographs in a forest area using a drone or unmanned aerial vehicle. The captured image data (input) is saved on the device. The user then uploads the image data to a server via the device. The user uses a dedicated application on the device and presses the "upload" button to send the data (output).

[0371] Step 2: Preprocessing the data

[0372] The image data received by the server (input) is first denoised using OpenCV's Gaussian filter. Next, the image data is normalised to a range of 0 to 1 to maintain data consistency. Furthermore, geometric transformations such as resizing and rotation are performed to make the data suitable for analysis (output). This results in preprocessed image data.

[0373] Step 3: Data analysis and feature extraction

[0374] The server inputs preprocessed image data (input) into a TensorFlow or PyTorch machine learning model. The model then analyzes the data (data processing) to extract features such as forest health, tree species, and water source locations. The analysis results (output) are stored in a database in JSON format.

[0375] Step 4: Planning for felling and planting

[0376] The server generates a risk map and identifies optimal felling and planting locations based on the analysis results (input). The server also considers soil data, rainfall information, and past successful planting cases to create a plan (data calculation). It also determines the optimal timing based on seasonal information and rainfall forecasts (output). This is done using a geographic information system (GIS).

[0377] Step 5: Integrating the Emotion Engine

[0378] The device's camera and microphone are activated, and the user's facial expressions and tone of voice are analyzed in real time (input). For example, Microsoft Azure Cognitive Services or Affectiva are used. Based on the analysis results (output), the device dynamically adjusts the interface and provides detailed explanations and advice to the user.

[0379] Step 6: Visualize and feedback the plan

[0380] The server sends the optimized felling and planting plan (input) along with map data to the device. The device visualizes the plan received in GeoJSON format on a map and displays it to the user (output). The user checks the plan and fills in a feedback form. The device sends the user's feedback to the server, and the emotion engine optimizes the way the feedback is presented based on the user's emotions.

[0381] Step 7: Implement the plan and track progress

[0382] The user finalizes the plan and presses the "Start Execution" button on the device (input). The server starts managing progress based on the plan and monitors it in real time (output). The user periodically reports the execution status via the device, and the server checks and adjusts the progress. The emotion engine analyzes the user's stress state and issues notifications and alarms at appropriate times.

[0383] Specific examples

[0384] For example, if a user were to create a tree planting plan, the process would be as follows: The user would take aerial photos of the forest with a drone, save them on their device, and upload them to the server. The server would use OpenCV to remove noise from the images and a TensorFlow model to identify soil quality and the health of existing trees. Using this information, the optimal planting locations would be identified and displayed on a map. The user would review the plan and provide feedback, and the emotion engine would analyze that feedback and optimize the presentation. Finally, the user would execute the plan, and the server would monitor progress in real time.

[0385] (Application example 2)

[0386] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0387] In conventional forest management systems, it is necessary to consider not only forest health and tree planting optimization, but also the psychological state of workers. In particular, to improve the efficiency of workers who are susceptible to fatigue and stress, a system that can grasp their emotional state and provide appropriate feedback is needed.

[0388] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0389] In this invention, the server includes means for receiving satellite images and images captured by unmanned aerial vehicles, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify the forest health and tree species, means for determining optimal felling and planting sites based on the identified data, means for visualizing the determined felling and planting sites on a map, means for recognizing user emotions in real time and optimizing the content and presentation method of the feedback, means for receiving user feedback and revising the plan, and means for providing the revised plan to the user and managing its execution and progress. This enables a user-friendly system that takes into account the psychological state of workers while improving the efficiency and accuracy of forest management.

[0390] "Satellite imagery" refers to image data taken from an artificial satellite in orbit around the Earth.

[0391] An "unmanned aerial vehicle" is an aircraft that flies remotely or autonomously without a human on board.

[0392] "Preprocessing" refers to processes such as noise removal, normalization, and geometric transformation of image data before analysis.

[0393] "Forest health" refers to the state of the forest, indicating whether the trees and vegetation within it are growing healthily.

[0394] "Tree types" refers to the classification of the various trees present in the forest.

[0395] A "harvesting point" is a location within a forest where selected trees are recommended for felling.

[0396] A "planting site" is a location selected for planting new trees.

[0397] "Visualization" refers to the visual representation of numbers and data in an easy-to-understand way.

[0398] "User emotion" refers to the emotional state felt by the person using the system at that moment.

[0399] "Real-time recognition" means that the system processes data as soon as it receives it and provides analysis results immediately.

[0400] "Feedback" refers to the opinions and requests provided by users and any resulting adjustments to the system.

[0401] "Plan modification" means receiving feedback from users and modifying an existing plan based on this feedback.

[0402] Progress management is the process of monitoring the progress of a plan and making adjustments as needed.

[0403] The system for realizing this invention uses a server, a terminal, and, if necessary, multiple input devices. The server collects data, preprocesses it, analyzes it, creates plans, and manages progress, while the terminal links the user and the server and provides an interface. Specifically, the system is implemented as follows:

[0404] First, the user takes aerial and ground photographs of the forest using an unmanned aerial vehicle or satellite. This image data is stored on the device and the user uploads it to the server via the device. The server performs preprocessing on the received image data, such as noise removal, normalization, and geometric transformation. Libraries such as OpenCV and TensorFlow are used for image processing.

[0405] Next, the server applies machine learning models to the preprocessed image data, extracting features such as forest health, tree species, and the location of water sources. A pre-trained deep learning model (TensorFlow / Keras) is used for feature extraction, generating a risk map.

[0406] The server then uses the risk map to identify optimal felling and planting locations. The server takes into account factors such as diseased trees to identify felling locations, and soil data, rainfall information, and past successful plantings to select planting locations. Seasonal information and rainfall forecasts are also used to determine the optimal timing for felling and planting.

[0407] The server also incorporates an emotion engine that recognizes the user's emotions in real time. This emotion engine analyzes the user's facial expressions and tone of voice on the device to identify the user's emotional state. For example, if the user is feeling tired or stressed, the system will notify them at the appropriate time to take a break. EmotionRecognition is often used as an emotion recognition library.

[0408] The server then receives feedback from the user, analyzes it, and modifies the plan. The modified plan is then reflected on the map and sent to the device. The user then checks the plan on the device and reports the final execution status to the server. The server uses this information to monitor progress and make adjustments as necessary.

[0409] As a concrete example, when a user creates a tree planting plan, they first take aerial photos of the forest using an unmanned aerial vehicle and upload the image data from their device to a server. The server preprocesses this data and analyzes it using machine learning models to identify soil quality and the health of existing trees. The server then selects the optimal planting site and displays it on a map. When the user reviews the plan and enters feedback, the emotion engine analyzes the user's emotional state and optimizes the feedback content. Finally, when the user executes the plan and reports progress to the server, the emotion engine notifies them at the appropriate time and helps manage progress.

[0410] An example prompt might be, "This program is a forest management system installed in an autonomous vehicle. Write Python code that analyzes the following content and generates optimal felling and planting plans. Additionally, add a notification function to recognize the emotional state of the worker and reduce stress."

[0411] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0412] Step 1:

[0413] Users use unmanned aerial vehicles or satellites to take aerial and ground photographs of forests. These image data are stored on the device. The inputs are aerial and ground photographs, and the output is image data stored on the device. The user operates the image capture device to collect forest data over a wide area.

[0414] Step 2:

[0415] A user uploads image data to a server via a terminal. The input is the stored image data, and the output is the image data transferred to the server. The terminal uses an Internet connection to quickly transmit large amounts of image data to the server.

[0416] Step 3:

[0417] The server performs preprocessing on the received image data, such as noise removal, normalization, and geometric transformation. The input is the uploaded image data, and the output is the preprocessed image data. The OpenCV library is used to improve the image quality and make it suitable for analysis.

[0418] Step 4:

[0419] The server applies machine learning models based on the preprocessed image data to identify forest health, tree species, water source locations, etc. The input is the preprocessed image data, and the output is the identified feature data. A detailed analysis of the forest is performed using the TensorFlow / Keras library.

[0420] Step 5:

[0421] The server generates a risk map based on the identified data. The input is the identified feature data, and the output is a risk map. The server integrates the analysis results into a geographic information system (GIS) to visualize defects and dangerous areas.

[0422] Step 6:

[0423] The server identifies optimal felling and planting locations based on the risk map. The input is the risk map, and the output is optimized felling and planting locations. The server uses algorithms to generate an efficient work plan.

[0424] Step 7:

[0425] The server visualizes the optimal felling and planting locations on a map and sends it to the device. The input is the optimized location data, and the output is the visualized data on the map. The server uses a visualization tool to convert the data into a format that is easy to view on the device.

[0426] Step 8:

[0427] The user checks the plan contents and provides feedback using a device. The input is the plan data visualized on a map, and the output is the user's feedback data. The user interactively checks the plan on the device and suggests any necessary modifications.

[0428] Step 9:

[0429] The server receives user feedback and recognizes the user's emotional state in real time using an emotion engine. The inputs are feedback data and real-time emotion data, and the output is emotion recognition results. The server uses the EmotionRecognition library to understand the user's feelings.

[0430] Step 10:

[0431] The server modifies the plan based on the feedback and emotion recognition results. The inputs are user feedback and emotion recognition results, and the output is modified plan data. The server analyzes the feedback information and automatically makes necessary adjustments.

[0432] Step 11:

[0433] The revised plan is resent to the terminal and presented to the user. The input is the revised plan data, and the output is the final plan data presented to the user. The user confirms the revised plan on the terminal and starts implementing it.

[0434] Step 12:

[0435] The user executes the plan and reports the progress to the server. The input is data on the work being performed, and the output is progress report data. The user periodically sends progress information to the server using their terminal, allowing the progress of the work to be monitored in real time.

[0436] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0437] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0438] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0439] [Second embodiment]

[0440] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0441] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0442] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0443] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0444] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0445] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0446] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0447] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0448] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0449] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0450] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0451] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0452] This invention is a system for planning and executing efficient and highly accurate forest felling and tree planting plans. The specific details of this system are described below.

[0453] System Overview

[0454] The system is primarily composed of a server and a terminal, and is operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal functions as an interface connecting the user and the server. The user operates the system via the terminal to collect data, confirm plans, and execute them.

[0455] Data collection and preprocessing

[0456] Users take aerial photographs (including satellite images) of forests using drones or unmanned aerial vehicles and store the image data on their devices. The devices then upload the image data to a server, which then performs preprocessing on the received image data. The preprocessing includes noise removal, normalization, and geometric transformation.

[0457] Data analysis and feature extraction

[0458] The server analyzes the preprocessed image data and extracts features such as forest health, tree species, and the location of water sources. This analysis uses machine learning models (e.g., convolutional neural networks). The server stores these features in a database and generates a risk map.

[0459] Planning of felling and planting trees

[0460] The server uses the risk map to identify optimal felling and planting locations. It considers the extent of diseased trees when identifying felling locations. It also considers soil data, rainfall information, and past success stories when selecting planting locations. It also uses seasonal information and rainfall forecasts to determine the optimal timing for felling and planting.

[0461] Plan visualization and feedback

[0462] The server sends the plan results to the device, which displays them visually on a map. The user provides feedback on the displayed plan, including specific instructions such as "I would like to postpone cutting trees in this area." The device then sends this feedback to the server, which then modifies the plan.

[0463] Implementing the plan and managing progress

[0464] Once the user has finalized the plan, the server will manage the progress based on this plan. The user will periodically report the execution status to the server via their terminal. The server will monitor the progress based on this data and adjust the plan as necessary.

[0465] Specific examples

[0466] For example, consider a user planning a tree planting project. First, the user takes aerial photos of the forest using a drone and uploads the image data from their device to a server. The server receives the data and pre-processes it. Next, the server analyzes it using machine learning models to identify soil quality and the health of existing trees. It then selects optimal planting locations and displays them on a map. The user reviews the plan and provides feedback. Finally, the user executes the plan and periodically reports progress to the server.

[0467] This system enables users to achieve efficient and sustainable forest management.

[0468] The processing flow will be explained below.

[0469] Program processing flow

[0470] Step 1: Collect data

[0471] Users take aerial and ground photographs of forests using drones or unmanned aerial vehicles, save the image data on their devices, and then upload it to a server via their devices.

[0472] Step 2: Receiving and Preprocessing Data

[0473] The server receives image data uploaded from the device. The server performs preprocessing such as noise removal, normalization, and geometric transformation on the received data. In this process, image distortion is corrected, improving the accuracy of analysis.

[0474] Step 3: Image analysis and feature extraction

[0475] The server applies machine learning models (e.g., convolutional neural networks) to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources. The server stores the analysis results in a database.

[0476] Step 4: Generate a risk map

[0477] The server generates a risk map based on the extracted features, visually indicating the extent of diseased trees, areas that need to be cut down, and areas where tree planting is appropriate.

[0478] Step 5: Identifying the harvest site

[0479] The server runs an algorithm based on the risk map to identify optimal felling sites, taking into account necessary environmental protection criteria such as the extent of diseased trees, and stores the list of felling sites.

[0480] Step 6: Select a planting site

[0481] The server selects the optimal planting location based on soil data, rainfall information, and past examples of successful planting. The server then displays the latitude and longitude information of the planting location overlaid on map data.

[0482] Step 7: Generate a time schedule

[0483] The server uses rainfall forecasts and seasonal information to determine the optimal timing for tree cutting and planting, and generates a time schedule that is recorded in a database.

[0484] Step 8: Visualize the plan

[0485] The server sends the optimized felling and planting plan along with map data to the terminal, which visualizes the plan on a map and displays it to the user.

[0486] Step 9: Gather user feedback

[0487] The user inputs feedback about the displayed plan via the terminal. For example, the user inputs an instruction such as "I would like to postpone the felling of this area." The terminal then sends the feedback to the server.

[0488] Step 10: Incorporate feedback and revise your plan

[0489] The server reconstructs the plan based on the received feedback, and the revised plan is sent back to the device and displayed to the user.

[0490] Step 11: Execute the plan

[0491] The user reviews the final plan and approves it for execution. The device sends the execution status to the server, allowing the progress of the plan to be monitored in real time.

[0492] Step 12: Monitor progress and improve

[0493] The server records the progress of the executed tasks in a database and adjusts the plan again if necessary. The user uploads additional data (e.g., information on new disease outbreaks) from their device to the server, and a new analysis is performed.

[0494] This is the flow of the program processing for this system, which enables users to achieve efficient and sustainable forest management based on data.

[0495] Example 1

[0496] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0497] Conventional forest management systems have faced the challenge of making efficient and accurate plans for felling and planting. Specifically, there was little centralized analysis of image data obtained from satellite images and aerial photography, selection of optimal felling and planting locations, or feedback and revision of plans. This resulted in a decrease in the accuracy of plans and the efficiency of their execution, making it difficult to achieve sustainable forest management.

[0498] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0499] In this invention, the server includes means for receiving satellite images and images captured by an aerial imaging device, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify forest health, tree species, and water source locations, means for determining optimal felling and planting sites based on the identified data, means for visualizing the determined felling and planting sites on a map, means for receiving feedback from a user and revising the plan, means for providing the revised plan to the user and managing its execution and progress, means for analyzing the image data using a machine learning model, and means for receiving additional data from the user and readjusting the plan, thereby enabling the efficient and accurate planning and execution of felling and planting plans.

[0500] "Satellite imagery" refers to images taken by satellites orbiting the Earth, and is data that provides information on a wide range of the Earth's surface.

[0501] An "aerial photography device" is a device, such as a drone or unmanned aerial vehicle, used to take photographs or videos of the ground from a high altitude.

[0502] "Preprocessing" refers to processes such as noise removal, normalization, and geometric transformation that are performed on image data to be analyzed, and is a process for improving the quality of the data.

[0503] "Forest health" refers to the overall state of the forest ecosystem, including the state of plant growth and the occurrence of pests and diseases.

[0504] "Tree species" refers to the types of trees that grow in a particular area or forest.

[0505] "Location of water sources" refers to location information of rivers, lakes, and other places where water exists within a forest.

[0506] "Optimal harvesting site" refers to a suitable location chosen to harvest trees efficiently and sustainably.

[0507] "Optimal planting site" refers to the suitable location selected for planting new trees.

[0508] "Visualization" refers to the graphical display of data and planning results on a map.

[0509] "Feedback" refers to information provided by users to modify and improve the plan through their opinions and instructions.

[0510] A "machine learning model" refers to an algorithm or model that learns patterns and features from data and predicts and analyzes future data.

[0511] "Additional data" refers to new information or data provided by the user that serves as the basis for revising or readjusting the plan.

[0512] "Progress" refers to the progress of checking the execution status and degree of achievement of a plan.

[0513] This invention is a system for planning and executing efficient and highly accurate forest felling and tree planting plans. The specific details of this system are described below.

[0514] The system is primarily composed of a server and a terminal, and is operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal functions as an interface connecting the user and the server. The user operates the system via the terminal to collect data, confirm plans, and execute them.

[0515] Data collection and preprocessing

[0516] Users use aerial photography equipment such as drones to take aerial or satellite images of forests and store the image data on their devices. The devices then upload the image data to a server. A high-speed Internet connection is required for uploading, and the HTTPS protocol is used. The server then performs preprocessing on the received image data. This preprocessing includes noise removal, normalization, and geometric transformation. OpenCV can be used for this.

[0517] Data analysis and feature extraction

[0518] The server analyzes the preprocessed image data and extracts features such as forest health, tree species, and the location of water sources. This analysis uses a convolutional neural network (CNN), a machine learning model, using TensorFlow and PyTorch. The server stores these features in a database and generates a risk map. QGIS can be used to visualize the risk map.

[0519] Planning of felling and planting trees

[0520] The server identifies optimal felling and planting locations based on the risk map. The location of felling locations is determined by taking into account the extent of diseased trees and other risk factors. The planting locations are selected based on soil data, rainfall information, and past success stories. Seasonal information and rainfall forecast data are also used to determine the optimal timing for felling and planting. This process utilizes environmental databases and weather data APIs.

[0521] Plan visualization and feedback

[0522] The server sends the proposed tree-cutting and planting plan to the device, which then visually displays it on a map. Open-source JavaScript libraries such as OpenLayers and Leaflet are used to display the map. The user checks the displayed plan and provides feedback, such as "I would like to postpone tree-cutting in this area." The feedback is sent via the device to the server, which then modifies the plan.

[0523] Implementing the plan and managing progress

[0524] Once the user has finalized the plan, the server will manage progress based on this plan. The user periodically reports the execution status to the server via their terminal. The reports include the progress and any issues. The server will monitor the progress based on this data and readjust the plan as necessary. Project management tools such as Asana and JIRA can be used to manage progress.

[0525] Examples of specific examples and prompts

[0526] For example, consider a case where a user is creating a tree planting plan. The user first uses a drone to take aerial photos of the forest and uploads them from their device to the server. The server preprocesses the received data using OpenCV and then uses TensorFlow to identify soil quality and the health of existing trees. It then generates a risk map using QGIS. The server selects optimal planting sites and sends the plan to the device. The device displays the plan on the map, and the user provides feedback, such as "I would like to postpone tree felling in this area." This feedback is sent to the server through the Django framework, which then modifies the plan. Finally, the user finalizes the plan and periodically reports the progress of the work to the server. The server monitors the progress using Asana.

[0527] Example prompt sentence:

[0528] “You upload aerial photos of your forest. We preprocess the data and analyze it using machine learning models. We identify the best locations for cutting and planting trees and modify the plan based on your feedback.”

[0529] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0530] Step 1:

[0531] Data Acquisition

[0532] Users use drones or unmanned aerial vehicles to take aerial photographs or satellite images of forests, and the captured image data is stored in the drone's internal storage.

[0533] Specific actions

[0534] To take aerial photos of a forest, a user launches the drone and sets a designated flight path. The drone then flies according to the settings and takes photos of the designated area with its high-resolution camera.

[0535] Input and Output

[0536] Input: Flight path, shooting instructions

[0537] Output: Aerial photographs taken

[0538] Step 2:

[0539] Saving image data

[0540] The user transfers the captured image data from the drone to a device and stores it on a high-performance SSD.

[0541] Specific actions

[0542] The user connects a data transfer cable from the drone to the device and downloads the image data to the device, where it is saved in the device's storage.

[0543] Input and Output

[0544] Input: Image data from inside the drone

[0545] Output: Image data in the device

[0546] Step 3:

[0547] Uploading image data

[0548] The device uploads the stored image data to a server over a high-speed internet connection using the HTTPS protocol.

[0549] Specific actions

[0550] The user uploads image data to the server using a dedicated application on the device, and the progress of the data transfer is displayed until it is complete.

[0551] Input and Output

[0552] Input: Image data in the device

[0553] Output: Image data uploaded to the server

[0554] Step 4:

[0555] Image data preprocessing

[0556] The server performs preprocessing such as noise reduction, normalization, and geometric transformation on the received image data using OpenCV.

[0557] Specific actions

[0558] The server first applies a Gaussian filter to remove noise from the image, then scales the image's pixel values ​​to the range 0 to 1, and then performs a geometric transformation to adjust the image's angle and scale.

[0559] Input and Output

[0560] Input: Image data uploaded to the server

[0561] Output: Preprocessed image data

[0562] Step 5:

[0563] Image data analysis and feature extraction

[0564] The server then feeds the preprocessed image data into a machine learning model (e.g., a convolutional neural network) to extract features such as forest health, tree species, and the location of water sources. TensorFlow is used for the analysis.

[0565] Specific actions

[0566] The server runs the preprocessed image data through the TensorFlow environment and applies a trained neural network model that identifies forest health and other important features from the input image and outputs the results in a list format.

[0567] Input and Output

[0568] Input: Preprocessed image data

[0569] Output: Extracted feature data

[0570] Step 6:

[0571] Saving to the database and generating a risk map

[0572] The server stores the extracted feature data in a database and generates a risk map, which is visualized using QGIS.

[0573] Specific actions

[0574] The server inserts the extracted feature data into a database, runs the risk map generation algorithm, and uses QGIS to generate the risk map and save the results in a map format.

[0575] Input and Output

[0576] Input: extracted feature data

[0577] Output: Risk map

[0578] Step 7:

[0579] Deciding on felling and planting locations

[0580] The server determines the optimal locations for cutting and planting trees based on the risk map, taking into account seasonal information and rainfall forecast data.

[0581] Specific actions

[0582] The server takes into account the risk map along with additional environmental data (e.g., rainfall forecasts and seasonal information) and uses algorithms to identify optimal felling and planting locations. The results are stored in a list format and can be displayed later.

[0583] Input and Output

[0584] Input: Risk map, environmental data

[0585] Output: List of optimal cutting and planting locations

[0586] Step 8:

[0587] Sending planning results

[0588] The server sends the proposed tree-cutting and planting plan to the terminal, which includes geographic information and integrates it into the terminal's map system.

[0589] Specific actions

[0590] The server uses a dedicated API to send the list of determined felling and planting locations to the device, and after the sending process is completed, the device sends a message confirming receipt.

[0591] Input and Output

[0592] Input: List of optimal felling and planting sites

[0593] Output: Planning data sent to the terminal

[0594] Step 9:

[0595] Plan visualization

[0596] The device visually displays the received plan on a map, using open source JavaScript libraries such as OpenLayers and Leaflet to achieve detailed map display.

[0597] Specific actions

[0598] The terminal activates the geographic information system, imports the received planning data, and marks the planned felling and planting locations on a map and displays detailed information for the user's convenience.

[0599] Input and Output

[0600] Input: Planning data

[0601] Output: felling and planting plans displayed on a map

[0602] Step 10:

[0603] User Feedback

[0604] The user checks the displayed plan and provides feedback such as "I would like to postpone the felling of this area." The feedback is sent to the server via the terminal, and the server then modifies the plan.

[0605] Specific actions

[0606] The user checks the plan on the map, selects the part they want to correct, enters the feedback into the terminal, and presses the send button, which sends the correction request to the server.

[0607] Input and Output

[0608] Input: User feedback

[0609] Output: Feedback sent to the server

[0610] Step 11:

[0611] Revise and resubmit the plan

[0612] The server modifies the plan based on feedback from the user and retransmits the newly modified plan to the terminal.

[0613] Specific actions

[0614] The server analyzes the received feedback, applies a plan modification algorithm, and once the modification is complete, sends the plan back to the device.

[0615] Input and Output

[0616] Input: User feedback

[0617] Output: Corrected planning data

[0618] Step 12:

[0619] Implementing the plan and managing progress

[0620] Once the user has finalized the plan, the server will manage the progress based on this plan. The user will periodically report the execution status to the server via their terminal.

[0621] Specific actions

[0622] The user inputs the progress of the work into the terminal and presses the send button to report it to the server, which analyzes the received progress data and updates the progress status.

[0623] Input and Output

[0624] Input: Work progress data

[0625] Output: Latest progress data

[0626] Step 13:

[0627] Monitor progress and readjust

[0628] The server monitors the progress of the work and readjusts the plan as needed based on progress data reported by the user.

[0629] Specific actions

[0630] The server monitors progress data in real time, immediately issuing warnings if there are any problems with the progress or if new risks arise, and also makes any necessary corrections to the plan and notifies the device again.

[0631] Input and Output

[0632] Input: Work progress data

[0633] Output: Realigned planning data

[0634] The above are the processing steps of the program for this system. The techniques and specific operations used in each step have been explained in detail.

[0635] (Application example 1)

[0636] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0637] Conventional factory layout optimization systems are operated based on fixed layout designs and plans, making it difficult to change or optimize in real time. Furthermore, there was a lack of means to instantly understand and correct the efficiency of ongoing work in the actual environment. This resulted in a decline in work efficiency, making it difficult to improve productivity.

[0638] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0639] In this invention, the server includes means for receiving satellite images and images captured by unmanned aerial vehicles, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify the environmental health status and plant species, means for determining optimal work locations and layout plans based on the identified data, means for visualizing the determined work locations and layout plans on a map, means for receiving feedback from a user and revising the plan, means for providing the revised plan to the user and managing execution and progress, and means for monitoring the situation in real time using smart glasses and visually confirming the optimal layout, thereby enabling the optimal factory layout to be confirmed and revised in real time.

[0640] "Satellite imagery" refers to image data taken from Earth's satellites and is used to obtain information on the terrain and environment over a wide area.

[0641] An "unmanned aerial vehicle" is a remotely controlled or autonomously flown aircraft equipped with a specialized camera used to take high-resolution aerial photographs.

[0642] "Image data preprocessing" is a method of performing initial processing such as noise removal, normalization, and geometric transformation on acquired image data, and converting it into a format suitable for analysis.

[0643] "Environmental health" refers to the overall assessment of factors and elements (e.g., plant health, water quality) in a particular natural or man-made environment.

[0644] A "plant type" refers to a classification of plants present in a particular area, identified based on biological categories such as species, genus, or family.

[0645] "Work location" refers to the geographic location that is best suited to carrying out a particular task (e.g., cutting trees or planting trees).

[0646] A "layout plan" is a blueprint or scheme for a field, factory, etc., for planning and devising the optimal placement of specific tasks or machinery.

[0647] "Visualization" is the technique of visually displaying data and information, and converting it into a format that is easily understood and usable by users.

[0648] "Feedback" refers to the process of receiving opinions and requests from users and reflecting them in systems and plans.

[0649] "Smart glasses" are wearable devices that use augmented reality (AR) technology and are glasses-type devices that can display and operate information in real time.

[0650] This invention is a system for efficiently and accurately optimizing factory layouts. This system can improve work efficiency and productivity in real time. Specifically, optimization is performed through collaboration between servers, terminals, and users.

[0651] System configuration

[0652] The system mainly consists of the following components:

[0653] 1. Hardware

[0654] Server: A central processing unit that performs data analysis and planning.

[0655] Smart glasses: Used for real-time situation monitoring and visualization of optimization plans.

[0656] Unmanned aerial vehicle: Used to photograph the layout of the factory.

[0657] 2. Software

[0658] Python: Used for data analysis and running machine learning models.

[0659] OpenCV: A library for image processing.

[0660] Keras: Used to build and manage machine learning models.

[0661] Django: A web application framework.

[0662] Data collection and preprocessing

[0663] Users use unmanned aerial vehicles to take high-resolution images of their factory interiors and upload them to a server via their terminal. The server then performs preprocessing such as noise removal and normalization on the received image data, converting it into a format suitable for analysis. This removes noise from the image data and improves the accuracy of the analysis.

[0664] Data Analysis and Optimization

[0665] The server analyzes the preprocessed image data to determine the layout of equipment and work flow within the factory. A pre-trained machine learning model (generative AI model) is used for the analysis. This model extracts features from the input image data, such as the health status of the environment, the type of equipment, and its location. Based on these extracted features, the server then generates an optimal layout plan and work flow.

[0666] Visualization and feedback of optimization plans

[0667] The optimization plan generated by the server is displayed on the smart glasses. The user can check it in real time and provide feedback as needed. This feedback is sent from the device to the server, which then modifies the plan based on the received feedback. For example, specific instructions such as "I would like to change the equipment layout in this area" can be sent as feedback.

[0668] Implementing the plan and managing progress

[0669] Once the user has finalized the plan, the server will manage the progress based on this plan. Real-time monitoring and feedback makes it easy to revise and readjust the plan, improving work efficiency.

[0670] Specific examples

[0671] For example, consider the case of optimizing the layout of a factory. A user uses an unmanned aerial vehicle to take images of the inside of the factory and uploads this image data from their device to a server. The server preprocesses the image data and analyzes it using a machine learning model to generate an optimal layout plan and work flow. The optimized plan is then presented to the user via smart glasses. The user provides feedback on the plan while checking the actual situation on the site, and the server then modifies the plan, achieving optimization.

[0672] Example prompt sentence:

[0673] "Please build a system that can optimize the acquired factory layout images and visually confirm the results of the optimal layout."

[0674] This system enables the factory layout to be optimized in real time, establishing an efficient and sustainable production system.

[0675] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0676] Step 1:

[0677] The user uses an unmanned aerial vehicle to take high-resolution images of the factory and saves the image data on a terminal. The input is the image data taken by the unmanned aerial vehicle, and the output is an image file saved on the terminal.

[0678] Step 2:

[0679] A user uploads image data to a server via a terminal. The input is an image file stored on the terminal, and the output is the image data sent to the server.

[0680] Step 3:

[0681] The server performs preprocessing on the received image data. Specifically, it performs noise removal, normalization, and geometric transformation on the image data. The input is the image data sent to the server, and the output is the preprocessed image data.

[0682] Step 4:

[0683] The server analyzes the preprocessed image data and identifies the equipment layout and work flow within the factory. This analysis uses a machine learning model (generative AI model). The input is the preprocessed image data, and the output is the equipment layout and work flow data as the analysis results.

[0684] Step 5:

[0685] The server then formulates an optimal layout plan based on the generated equipment placement and work flow data. The input is the analysis results data, and the output is an optimized layout plan.

[0686] Step 6:

[0687] The server provides the optimized layout plan to the user through the smart glasses, where the input is the optimized layout plan and the output is the visual plan displayed on the smart glasses.

[0688] Step 7:

[0689] The user can use the smart glasses to check the layout plan in real time and provide feedback as needed. The input is the feedback information from the user, and the output is the feedback data sent to the server via the terminal.

[0690] Step 8:

[0691] The server modifies the layout plan based on the received user feedback and performs re-optimization. At this time, a new layout plan that reflects the user's opinions is generated. The input is the feedback data, and the output is the modified layout plan.

[0692] Step 9:

[0693] The user checks and approves the final layout plan. An example of a prompt is, "Optimize the acquired factory layout image and build a system that allows visual confirmation of the results of the optimal layout." The input is the revised layout plan, and the output is the approved final layout plan.

[0694] Step 10:

[0695] The server manages progress based on the final approved layout plan and provides a system where users can report the execution status of work in real time. The input is real-time execution status data, and the output is a work schedule with appropriately managed progress.

[0696] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0697] This invention is a system for planning and executing efficient and accurate forest felling and tree planting plans, and by combining it with an emotion engine that recognizes the user's emotions, it provides a more user-friendly interface.

[0698] System Overview

[0699] The system is primarily comprised of a server and a terminal, operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal acts as an interface between the user and the server. The emotion engine recognizes user emotions in real time and is used to optimize the user experience.

[0700] Data collection and preprocessing

[0701] Users use drones or unmanned aerial vehicles to take aerial and ground photographs of forests and store the image data on their devices. The users then upload the data to the server via their devices. The server then performs preprocessing on the received image data, including noise reduction, normalization, and geometric transformation.

[0702] Data analysis and feature extraction

[0703] The server applies machine learning models to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources, generating a risk map. The server then stores the analysis results in a database.

[0704] Planning of felling and planting trees

[0705] The server identifies optimal felling and planting locations based on a risk map. The server takes into account factors such as the extent of diseased trees when identifying felling locations. Soil data, rainfall information, and past success stories of plantings are taken into account when selecting planting locations. Additionally, the server uses seasonal information and rainfall forecasts to determine the optimal timing for felling and planting.

[0706] Emotion engine integration

[0707] The emotion engine analyzes the user's facial expressions and tone of voice in real time on the device to recognize the user's emotional state. For example, if the user looks anxious, the system will provide the user with more detailed explanations and advice. If the user looks satisfied, the system will determine that the proposed plan is acceptable.

[0708] Plan visualization and feedback

[0709] The server sends the optimized felling and planting plan along with map data to the device. The device visualizes the plan on a map and displays it to the user. The user provides feedback on the plan, which the device then sends to the server. The emotion engine optimizes the content and presentation of the feedback based on the user's emotions.

[0710] Implementing the plan and managing progress

[0711] Once the user finalizes the plan, the server manages progress based on this plan. The user periodically reports their execution status to the server via their device. The server monitors progress based on this and readjusts the plan as necessary. The emotion engine adjusts notifications and alarms at appropriate times when the user is feeling stressed.

[0712] Specific examples

[0713] For example, when a user creates a tree planting plan, they first use a drone to take aerial photos of the forest and upload the image data from their device to the server. The server receives the data and pre-processes it. Next, it uses a machine learning model to analyze it and identify the soil quality and the health of existing trees. The optimal planting site is then selected and displayed on a map. When the user reviews the plan and enters feedback, the emotion engine analyzes the user's emotional state and optimizes the feedback content. Finally, when the user executes the plan and reports progress to the server, the emotion engine notifies them at the appropriate time and helps manage progress.

[0714] In this way, the system enables efficient and sustainable forest management based on data, while providing an easy-to-use interface that takes into account the user's emotional state.

[0715] The processing flow will be explained below.

[0716] Program processing flow

[0717] Step 1: Collect data

[0718] Users take aerial and ground photographs of forests using drones or unmanned aerial vehicles, save the image data on their devices, and then upload it to a server via their devices.

[0719] Step 2: Receiving and Preprocessing Data

[0720] The server receives image data uploaded from the device. The server performs preprocessing such as noise removal, normalization, and geometric transformation on the received data. This removes distortion and noise from the image, making it easier to analyze.

[0721] Step 3: Image analysis and feature extraction

[0722] The server applies machine learning models (e.g., convolutional neural networks) to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources. The server stores the analysis results in a database.

[0723] Step 4: Generate a risk map

[0724] The server generates a risk map based on the extracted features, visually indicating the extent of diseased trees, areas that need to be cut down, and areas where tree planting is appropriate.

[0725] Step 5: Identifying the harvest site

[0726] The server runs an algorithm based on the risk map to identify optimal logging sites, taking into account the extent of diseased trees and other environmental protection criteria. The server then stores the list of logging sites.

[0727] Step 6: Select a planting site

[0728] The server selects the optimal planting location based on soil data, rainfall information, and past examples of successful planting. The server then displays the latitude and longitude information of the planting location overlaid on map data.

[0729] Step 7: Generate a time schedule

[0730] The server uses rainfall forecasts and seasonal information to determine the optimal timing for tree cutting and planting, and generates a time schedule that is recorded in a database.

[0731] Step 8: Recognizing user emotions with the emotion engine

[0732] The device analyzes the user's facial expressions and tone of voice in real time to recognize their emotional state, and when the user shows a certain emotion, it optimizes the feedback method according to that emotion.

[0733] Step 9: Visualize the plan

[0734] The server sends the optimized tree-cutting and planting plan along with map data to the device. The device visualizes the plan on the map and displays it to the user. The emotion engine analyzes the user's reaction and displays additional explanations or encouraging messages if necessary.

[0735] Step 10: Gather user feedback

[0736] The user inputs feedback about the displayed plan via the device. For example, they can enter specific instructions such as "I would like to postpone the felling of trees in this area." The device then sends the feedback to the server. The emotion engine analyzes the user's emotions and presents the feedback in an easy-to-understand format.

[0737] Step 11: Incorporate feedback and revise your plan

[0738] The server reconstructs the plan based on the received feedback. The revised plan is sent back to the device and displayed to the user. The emotion engine generates a response based on the user's emotions, encouraging the user to provide positive feedback.

[0739] Step 12: Execute the plan

[0740] The user reviews the final plan and approves it for execution. The device sends the execution status to the server, monitoring the progress of the plan in real time. The emotion engine displays encouraging messages at appropriate times to reduce the user's stress level.

[0741] Step 13: Monitor progress and improve

[0742] The server records the progress of the executed tasks in a database and adjusts the plan again if necessary. The user uploads additional data (e.g., information on new disease outbreaks) from their device to the server for new analysis. The emotion engine monitors the user's emotional state and provides appropriate feedback and responses.

[0743] The above is the specific flow of the program processing of this system, which enables users to achieve efficient and sustainable forest management based on data, while also increasing user satisfaction through the emotion engine.

[0744] Example 2

[0745] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0746] Conventional forest management systems lack the mechanisms for formulating and implementing efficient and accurate forest harvesting and planting plans. In particular, they lack the ability to adjust plans to take users' emotions into account, making it difficult to reduce their stress and anxiety. This poses a risk of reducing the sustainability and efficiency of forest management.

[0747] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0748] In this invention, the server includes means for receiving satellite images and images captured by unmanned aerial vehicles, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify the forest health and tree species, means for determining optimal felling and planting sites based on the identified data, means for visualizing the determined felling and planting sites on a map, means for analyzing the user's facial expressions and tone of voice in real time to recognize the user's emotional state, means for optimizing the interface in accordance with the recognized emotional state, receiving feedback from the user, and revising the plan, and means for providing the revised plan to the user and managing its execution and progress. This enables efficient and accurate forest management, and by taking the user's emotional state into consideration, it is possible to provide a user-friendly interface and reduce stress and anxiety.

[0749] "Satellite imagery" is image data taken of the Earth's surface from space.

[0750] An "unmanned aerial vehicle" is an aircraft that is operated by remote control or automatic pilot and has no personnel on board.

[0751] "Preprocessing" refers to the process of converting received data into a format that is easy to analyze and use, and specifically includes noise removal, normalization, geometric transformation, etc.

[0752] "Analysis" is the process of examining and breaking down data in detail to clarify its content and structure.

[0753] "Forest health" is an indicator of how healthy a forest is, including the growth status of trees and the presence of pests and diseases.

[0754] "Tree type" is a classification of the types of trees present in a particular forest.

[0755] "Cutting point" means [a specific location in the forest where trees should be cut].

[0756] "Planting site" means [a suitable location for planting new trees].

[0757] "Visualization" is a technique that makes data and information easier to understand by displaying them visually.

[0758] "Emotional state" refers to the user's current psychological and emotional state.

[0759] "Interface optimization" refers to adjusting the layout and functionality of an interface to improve the user experience.

[0760] "Feedback" is the process of feeding user reactions and opinions back into the system.

[0761] "Progress management" means monitoring the progress of plans and work and making adjustments or corrections as necessary.

[0762] This invention is a system for planning and executing efficient and accurate forest felling and planting plans, and by combining it with an emotion engine that recognizes user emotions, it provides a more user-friendly interface. The system is primarily composed of a server and a terminal, and is operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal functions as an interface connecting the user and the server. The emotion engine recognizes user emotions in real time and is used to optimize the user experience.

[0763] Data collection and preprocessing

[0764] Users use drones or unmanned aerial vehicles to take aerial and ground photographs of forests and store the image data on their devices. The users then upload the data to the server via their devices. The server then performs preprocessing on the received image data, including noise removal, normalization, and geometric transformation. OpenCV is used for noise removal, and Scikit-Image is used for normalization and geometric transformation.

[0765] Data analysis and feature extraction

[0766] The server applies machine learning models to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources, generating a risk map. The analysis is performed using TensorFlow and PyTorch, and the results are stored in a database.

[0767] Planning of felling and planting trees

[0768] The server uses the risk map to identify optimal felling and planting locations. The server takes into account factors such as the extent of diseased trees when identifying felling locations. Soil data, rainfall information, and past success stories of plantings are taken into account when selecting planting locations. The server also uses seasonal information and rainfall forecasts to determine the optimal timing for felling and planting. This is done using geographic information systems such as ArcGIS and QGIS.

[0769] Emotion engine integration

[0770] The emotion engine, powered by Microsoft Azure Cognitive Services and Affectiva, analyzes the user's facial expressions and tone of voice in real time on the device to recognize their emotional state. If the user looks anxious, the system will provide them with more detailed explanations and advice.

[0771] Plan visualization and feedback

[0772] The server sends the optimized felling and planting plans along with map data to the device. The device visualizes the plans on a map and displays them to the user. For example, the device receives the data in GeoJSON format and visualizes it in a dedicated map app. The user provides feedback on the plans, which the device then sends to the server. The emotion engine optimizes the content and presentation of the feedback based on the user's emotions.

[0773] Implementing the plan and managing progress

[0774] Once the user finalizes the plan, the server manages progress based on this plan. The user periodically reports their execution status to the server via their device. The server monitors progress based on this and readjusts the plan as necessary. The emotion engine adjusts notifications and alarms at appropriate times when the user is feeling stressed.

[0775] Specific examples

[0776] For example, when a user creates a tree planting plan, they first use a drone to take aerial photos of the forest and upload the image data from their device to the server. The server receives the data and pre-processes it. Next, it uses a machine learning model to analyze it and identify the soil quality and the health of existing trees. The optimal planting site is then selected and displayed on a map. When the user reviews the plan and enters feedback, the emotion engine analyzes the user's emotional state and optimizes the feedback content. Finally, when the user executes the plan and reports progress to the server, the emotion engine notifies them at the appropriate time and helps manage progress.

[0777] Example prompt sentence:

[0778] "Please analyze aerial photos of the forest taken by a drone, identify the best locations for planting trees, and display the plan on a map. You should also develop a system that analyzes the user's facial expressions and provides appropriate feedback."

[0779] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0780] Step 1: Data collection and upload

[0781] A user takes aerial and ground photographs in a forest area using a drone or unmanned aerial vehicle. The captured image data (input) is saved on the device. The user then uploads the image data to a server via the device. The user uses a dedicated application on the device and presses the "upload" button to send the data (output).

[0782] Step 2: Preprocessing the data

[0783] The image data received by the server (input) is first denoised using OpenCV's Gaussian filter. Next, the image data is normalised to a range of 0 to 1 to maintain data consistency. Furthermore, geometric transformations such as resizing and rotation are performed to make the data suitable for analysis (output). This results in preprocessed image data.

[0784] Step 3: Data analysis and feature extraction

[0785] The server inputs preprocessed image data (input) into a TensorFlow or PyTorch machine learning model. The model then analyzes the data (data processing) to extract features such as forest health, tree species, and water source locations. The analysis results (output) are stored in a database in JSON format.

[0786] Step 4: Planning for felling and planting

[0787] The server generates a risk map and identifies optimal felling and planting locations based on the analysis results (input). The server also considers soil data, rainfall information, and past successful planting cases to create a plan (data calculation). It also determines the optimal timing based on seasonal information and rainfall forecasts (output). This is done using a geographic information system (GIS).

[0788] Step 5: Integrating the Emotion Engine

[0789] The device's camera and microphone are activated, and the user's facial expressions and tone of voice are analyzed in real time (input). For example, Microsoft Azure Cognitive Services or Affectiva are used. Based on the analysis results (output), the device dynamically adjusts the interface and provides detailed explanations and advice to the user.

[0790] Step 6: Visualize and feedback the plan

[0791] The server sends the optimized felling and planting plan (input) along with map data to the device. The device visualizes the plan received in GeoJSON format on a map and displays it to the user (output). The user checks the plan and fills in a feedback form. The device sends the user's feedback to the server, and the emotion engine optimizes the way the feedback is presented based on the user's emotions.

[0792] Step 7: Implement the plan and track progress

[0793] The user finalizes the plan and presses the "Start Execution" button on the device (input). The server starts managing progress based on the plan and monitors it in real time (output). The user periodically reports the execution status via the device, and the server checks and adjusts the progress. The emotion engine analyzes the user's stress state and issues notifications and alarms at appropriate times.

[0794] Specific examples

[0795] For example, if a user were to create a tree planting plan, the process would be as follows: The user would take aerial photos of the forest with a drone, save them on their device, and upload them to the server. The server would use OpenCV to remove noise from the images and a TensorFlow model to identify soil quality and the health of existing trees. Using this information, the optimal planting locations would be identified and displayed on a map. The user would review the plan and provide feedback, and the emotion engine would analyze that feedback and optimize the presentation. Finally, the user would execute the plan, and the server would monitor progress in real time.

[0796] (Application example 2)

[0797] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0798] In conventional forest management systems, it is necessary to consider not only forest health and tree planting optimization, but also the psychological state of workers. In particular, to improve the efficiency of workers who are susceptible to fatigue and stress, a system that can grasp their emotional state and provide appropriate feedback is needed.

[0799] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0800] In this invention, the server includes means for receiving satellite images and images captured by unmanned aerial vehicles, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify the forest health and tree species, means for determining optimal felling and planting sites based on the identified data, means for visualizing the determined felling and planting sites on a map, means for recognizing user emotions in real time and optimizing the content and presentation method of the feedback, means for receiving user feedback and revising the plan, and means for providing the revised plan to the user and managing its execution and progress. This enables a user-friendly system that takes into account the psychological state of workers while improving the efficiency and accuracy of forest management.

[0801] "Satellite imagery" refers to image data taken from an artificial satellite in orbit around the Earth.

[0802] An "unmanned aerial vehicle" is an aircraft that flies remotely or autonomously without a human on board.

[0803] "Preprocessing" refers to processes such as noise removal, normalization, and geometric transformation of image data before analysis.

[0804] "Forest health" refers to the state of the forest, indicating whether the trees and vegetation within it are growing healthily.

[0805] "Tree types" refers to the classification of the various trees present in the forest.

[0806] A "harvesting point" is a location within a forest where selected trees are recommended for felling.

[0807] A "planting site" is a location selected for planting new trees.

[0808] "Visualization" refers to the visual representation of numbers and data in an easy-to-understand way.

[0809] "User emotion" refers to the emotional state felt by the person using the system at that moment.

[0810] "Real-time recognition" means that the system processes data as soon as it receives it and provides analysis results immediately.

[0811] "Feedback" refers to the opinions and requests provided by users and any resulting adjustments to the system.

[0812] "Plan modification" means receiving feedback from users and modifying an existing plan based on this feedback.

[0813] Progress management is the process of monitoring the progress of a plan and making adjustments as needed.

[0814] The system for realizing this invention uses a server, a terminal, and, if necessary, multiple input devices. The server collects data, preprocesses it, analyzes it, creates plans, and manages progress, while the terminal links the user and the server and provides an interface. Specifically, the system is implemented as follows:

[0815] First, the user takes aerial and ground photographs of the forest using an unmanned aerial vehicle or satellite. This image data is stored on the device and the user uploads it to the server via the device. The server performs preprocessing on the received image data, such as noise removal, normalization, and geometric transformation. Libraries such as OpenCV and TensorFlow are used for image processing.

[0816] Next, the server applies machine learning models to the preprocessed image data, extracting features such as forest health, tree species, and the location of water sources. A pre-trained deep learning model (TensorFlow / Keras) is used for feature extraction, generating a risk map.

[0817] The server then uses the risk map to identify optimal felling and planting locations. The server takes into account factors such as diseased trees to identify felling locations, and soil data, rainfall information, and past successful plantings to select planting locations. Seasonal information and rainfall forecasts are also used to determine the optimal timing for felling and planting.

[0818] The server also incorporates an emotion engine that recognizes the user's emotions in real time. This emotion engine analyzes the user's facial expressions and tone of voice on the device to identify the user's emotional state. For example, if the user is feeling tired or stressed, the system will notify them at the appropriate time to take a break. EmotionRecognition is often used as an emotion recognition library.

[0819] The server then receives feedback from the user, analyzes it, and modifies the plan. The modified plan is then reflected on the map and sent to the device. The user then checks the plan on the device and reports the final execution status to the server. The server uses this information to monitor progress and make adjustments as necessary.

[0820] As a concrete example, when a user creates a tree planting plan, they first take aerial photos of the forest using an unmanned aerial vehicle and upload the image data from their device to a server. The server preprocesses this data and analyzes it using machine learning models to identify soil quality and the health of existing trees. The server then selects the optimal planting site and displays it on a map. When the user reviews the plan and enters feedback, the emotion engine analyzes the user's emotional state and optimizes the feedback content. Finally, when the user executes the plan and reports progress to the server, the emotion engine notifies them at the appropriate time and helps manage progress.

[0821] An example prompt might be, "This program is a forest management system installed in an autonomous vehicle. Write Python code that analyzes the following content and generates optimal felling and planting plans. Additionally, add a notification function to recognize the emotional state of the worker and reduce stress."

[0822] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0823] Step 1:

[0824] Users use unmanned aerial vehicles or satellites to take aerial and ground photographs of forests. These image data are stored on the device. The inputs are aerial and ground photographs, and the output is image data stored on the device. The user operates the image capture device to collect forest data over a wide area.

[0825] Step 2:

[0826] A user uploads image data to a server via a terminal. The input is the stored image data, and the output is the image data transferred to the server. The terminal uses an Internet connection to quickly transmit large amounts of image data to the server.

[0827] Step 3:

[0828] The server performs preprocessing on the received image data, such as noise removal, normalization, and geometric transformation. The input is the uploaded image data, and the output is the preprocessed image data. The OpenCV library is used to improve the image quality and make it suitable for analysis.

[0829] Step 4:

[0830] The server applies machine learning models based on the preprocessed image data to identify forest health, tree species, water source locations, etc. The input is the preprocessed image data, and the output is the identified feature data. A detailed analysis of the forest is performed using the TensorFlow / Keras library.

[0831] Step 5:

[0832] The server generates a risk map based on the identified data. The input is the identified feature data, and the output is a risk map. The server integrates the analysis results into a geographic information system (GIS) to visualize defects and dangerous areas.

[0833] Step 6:

[0834] The server identifies optimal felling and planting locations based on the risk map. The input is the risk map, and the output is optimized felling and planting locations. The server uses algorithms to generate an efficient work plan.

[0835] Step 7:

[0836] The server visualizes the optimal felling and planting locations on a map and sends it to the device. The input is the optimized location data, and the output is the visualized data on the map. The server uses a visualization tool to convert the data into a format that is easy to view on the device.

[0837] Step 8:

[0838] The user checks the plan contents and provides feedback using a device. The input is the plan data visualized on a map, and the output is the user's feedback data. The user interactively checks the plan on the device and suggests any necessary modifications.

[0839] Step 9:

[0840] The server receives user feedback and recognizes the user's emotional state in real time using an emotion engine. The inputs are feedback data and real-time emotion data, and the output is emotion recognition results. The server uses the EmotionRecognition library to understand the user's feelings.

[0841] Step 10:

[0842] The server modifies the plan based on the feedback and emotion recognition results. The inputs are user feedback and emotion recognition results, and the output is modified plan data. The server analyzes the feedback information and automatically makes necessary adjustments.

[0843] Step 11:

[0844] The revised plan is resent to the terminal and presented to the user. The input is the revised plan data, and the output is the final plan data presented to the user. The user confirms the revised plan on the terminal and starts implementing it.

[0845] Step 12:

[0846] The user executes the plan and reports the progress to the server. The input is data on the work being performed, and the output is progress report data. The user periodically sends progress information to the server using their terminal, allowing the progress of the work to be monitored in real time.

[0847] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0848] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0849] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0850] [Third embodiment]

[0851] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0852] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0853] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0854] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0855] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0856] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0857] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0858] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0859] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0860] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0861] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0862] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0863] This invention is a system for planning and executing efficient and highly accurate forest felling and tree planting plans. The specific details of this system are described below.

[0864] System Overview

[0865] The system is primarily composed of a server and a terminal, and is operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal functions as an interface connecting the user and the server. The user operates the system via the terminal to collect data, confirm plans, and execute them.

[0866] Data collection and preprocessing

[0867] Users take aerial photographs (including satellite images) of forests using drones or unmanned aerial vehicles and store the image data on their devices. The devices then upload the image data to a server, which then performs preprocessing on the received image data. The preprocessing includes noise removal, normalization, and geometric transformation.

[0868] Data analysis and feature extraction

[0869] The server analyzes the preprocessed image data and extracts features such as forest health, tree species, and the location of water sources. This analysis uses machine learning models (e.g., convolutional neural networks). The server stores these features in a database and generates a risk map.

[0870] Planning of felling and planting trees

[0871] The server uses the risk map to identify optimal felling and planting locations. It considers the extent of diseased trees when identifying felling locations. It also considers soil data, rainfall information, and past success stories when selecting planting locations. It also uses seasonal information and rainfall forecasts to determine the optimal timing for felling and planting.

[0872] Plan visualization and feedback

[0873] The server sends the plan results to the device, which displays them visually on a map. The user provides feedback on the displayed plan, including specific instructions such as "I would like to postpone cutting trees in this area." The device then sends this feedback to the server, which then modifies the plan.

[0874] Implementing the plan and managing progress

[0875] Once the user has finalized the plan, the server will manage the progress based on this plan. The user will periodically report the execution status to the server via their terminal. The server will monitor the progress based on this data and adjust the plan as necessary.

[0876] Specific examples

[0877] For example, consider a user planning a tree planting project. First, the user takes aerial photos of the forest using a drone and uploads the image data from their device to a server. The server receives the data and pre-processes it. Next, the server analyzes it using machine learning models to identify soil quality and the health of existing trees. It then selects optimal planting locations and displays them on a map. The user reviews the plan and provides feedback. Finally, the user executes the plan and periodically reports progress to the server.

[0878] This system enables users to achieve efficient and sustainable forest management.

[0879] The processing flow will be explained below.

[0880] Program processing flow

[0881] Step 1: Collect data

[0882] Users take aerial and ground photographs of forests using drones or unmanned aerial vehicles, save the image data on their devices, and then upload it to a server via their devices.

[0883] Step 2: Receiving and Preprocessing Data

[0884] The server receives image data uploaded from the device. The server performs preprocessing such as noise removal, normalization, and geometric transformation on the received data. In this process, image distortion is corrected, improving the accuracy of analysis.

[0885] Step 3: Image analysis and feature extraction

[0886] The server applies machine learning models (e.g., convolutional neural networks) to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources. The server stores the analysis results in a database.

[0887] Step 4: Generate a risk map

[0888] The server generates a risk map based on the extracted features, visually indicating the extent of diseased trees, areas that need to be cut down, and areas where tree planting is appropriate.

[0889] Step 5: Identifying the harvest site

[0890] The server runs an algorithm based on the risk map to identify optimal felling sites, taking into account necessary environmental protection criteria such as the extent of diseased trees, and stores the list of felling sites.

[0891] Step 6: Select a planting site

[0892] The server selects the optimal planting location based on soil data, rainfall information, and past examples of successful planting. The server then displays the latitude and longitude information of the planting location overlaid on map data.

[0893] Step 7: Generate a time schedule

[0894] The server uses rainfall forecasts and seasonal information to determine the optimal timing for tree cutting and planting, and generates a time schedule that is recorded in a database.

[0895] Step 8: Visualize the plan

[0896] The server sends the optimized felling and planting plan along with map data to the terminal, which visualizes the plan on a map and displays it to the user.

[0897] Step 9: Gather user feedback

[0898] The user inputs feedback about the displayed plan via the terminal. For example, the user inputs an instruction such as "I would like to postpone the felling of this area." The terminal then sends the feedback to the server.

[0899] Step 10: Incorporate feedback and revise your plan

[0900] The server reconstructs the plan based on the received feedback, and the revised plan is sent back to the device and displayed to the user.

[0901] Step 11: Execute the plan

[0902] The user reviews the final plan and approves it for execution. The device sends the execution status to the server, allowing the progress of the plan to be monitored in real time.

[0903] Step 12: Monitor progress and improve

[0904] The server records the progress of the executed tasks in a database and adjusts the plan again if necessary. The user uploads additional data (e.g., information on new disease outbreaks) from their device to the server, and a new analysis is performed.

[0905] This is the flow of the program processing for this system, which enables users to achieve efficient and sustainable forest management based on data.

[0906] Example 1

[0907] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0908] Conventional forest management systems have faced the challenge of making efficient and accurate plans for felling and planting. Specifically, there was little centralized analysis of image data obtained from satellite images and aerial photography, selection of optimal felling and planting locations, or feedback and revision of plans. This resulted in a decrease in the accuracy of plans and the efficiency of their execution, making it difficult to achieve sustainable forest management.

[0909] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0910] In this invention, the server includes means for receiving satellite images and images captured by an aerial imaging device, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify forest health, tree species, and water source locations, means for determining optimal felling and planting sites based on the identified data, means for visualizing the determined felling and planting sites on a map, means for receiving feedback from a user and revising the plan, means for providing the revised plan to the user and managing its execution and progress, means for analyzing the image data using a machine learning model, and means for receiving additional data from the user and readjusting the plan, thereby enabling the efficient and accurate planning and execution of felling and planting plans.

[0911] "Satellite imagery" refers to images taken by satellites orbiting the Earth, and is data that provides information on a wide range of the Earth's surface.

[0912] An "aerial photography device" is a device, such as a drone or unmanned aerial vehicle, used to take photographs or videos of the ground from a high altitude.

[0913] "Preprocessing" refers to processes such as noise removal, normalization, and geometric transformation that are performed on image data to be analyzed, and is a process for improving the quality of the data.

[0914] "Forest health" refers to the overall state of the forest ecosystem, including the state of plant growth and the occurrence of pests and diseases.

[0915] "Tree species" refers to the types of trees that grow in a particular area or forest.

[0916] "Location of water sources" refers to location information of rivers, lakes, and other places where water exists within a forest.

[0917] "Optimal harvesting site" refers to a suitable location chosen to harvest trees efficiently and sustainably.

[0918] "Optimal planting site" refers to the suitable location selected for planting new trees.

[0919] "Visualization" refers to the graphical display of data and planning results on a map.

[0920] "Feedback" refers to information provided by users to modify and improve the plan through their opinions and instructions.

[0921] A "machine learning model" refers to an algorithm or model that learns patterns and features from data and predicts and analyzes future data.

[0922] "Additional data" refers to new information or data provided by the user that serves as the basis for revising or readjusting the plan.

[0923] "Progress" refers to the progress of checking the execution status and degree of achievement of a plan.

[0924] This invention is a system for planning and executing efficient and highly accurate forest felling and tree planting plans. The specific details of this system are described below.

[0925] The system is primarily composed of a server and a terminal, and is operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal functions as an interface connecting the user and the server. The user operates the system via the terminal to collect data, confirm plans, and execute them.

[0926] Data collection and preprocessing

[0927] Users use aerial photography equipment such as drones to take aerial or satellite images of forests and store the image data on their devices. The devices then upload the image data to a server. A high-speed Internet connection is required for uploading, and the HTTPS protocol is used. The server then performs preprocessing on the received image data. This preprocessing includes noise removal, normalization, and geometric transformation. OpenCV can be used for this.

[0928] Data analysis and feature extraction

[0929] The server analyzes the preprocessed image data and extracts features such as forest health, tree species, and the location of water sources. This analysis uses a convolutional neural network (CNN), a machine learning model, using TensorFlow and PyTorch. The server stores these features in a database and generates a risk map. QGIS can be used to visualize the risk map.

[0930] Planning of felling and planting trees

[0931] The server identifies optimal felling and planting locations based on the risk map. The location of felling locations is determined by taking into account the extent of diseased trees and other risk factors. The planting locations are selected based on soil data, rainfall information, and past success stories. Seasonal information and rainfall forecast data are also used to determine the optimal timing for felling and planting. This process utilizes environmental databases and weather data APIs.

[0932] Plan visualization and feedback

[0933] The server sends the proposed tree-cutting and planting plan to the device, which then visually displays it on a map. Open-source JavaScript libraries such as OpenLayers and Leaflet are used to display the map. The user checks the displayed plan and provides feedback, such as "I would like to postpone tree-cutting in this area." The feedback is sent via the device to the server, which then modifies the plan.

[0934] Implementing the plan and managing progress

[0935] Once the user has finalized the plan, the server will manage progress based on this plan. The user periodically reports the execution status to the server via their terminal. The reports include the progress and any issues. The server will monitor the progress based on this data and readjust the plan as necessary. Project management tools such as Asana and JIRA can be used to manage progress.

[0936] Examples of concrete examples and prompts

[0937] For example, consider a case where a user is creating a tree planting plan. The user first uses a drone to take aerial photos of the forest and uploads them from their device to the server. The server preprocesses the received data using OpenCV and then uses TensorFlow to identify soil quality and the health of existing trees. It then generates a risk map using QGIS. The server selects optimal planting sites and sends the plan to the device. The device displays the plan on the map, and the user provides feedback, such as "I would like to postpone tree felling in this area." This feedback is sent to the server through the Django framework, which then modifies the plan. Finally, the user finalizes the plan and periodically reports the progress of the work to the server. The server monitors the progress using Asana.

[0938] Example prompt sentence:

[0939] “You upload aerial photos of your forest. We preprocess the data and analyze it using machine learning models. We identify the best locations for cutting and planting trees and modify the plan based on your feedback.”

[0940] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0941] Step 1:

[0942] Data Acquisition

[0943] Users use drones or unmanned aerial vehicles to take aerial photographs or satellite images of forests, and the captured image data is stored in the drone's internal storage.

[0944] Specific actions

[0945] To take aerial photos of a forest, a user launches the drone and sets a designated flight path. The drone then flies according to the settings and takes photos of the designated area with its high-resolution camera.

[0946] Input and Output

[0947] Input: Flight path, shooting instructions

[0948] Output: Aerial photographs taken

[0949] Step 2:

[0950] Saving image data

[0951] The user transfers the captured image data from the drone to a device and stores it on a high-performance SSD.

[0952] Specific actions

[0953] The user connects a data transfer cable from the drone to the device and downloads the image data to the device, where it is saved in the device's storage.

[0954] Input and Output

[0955] Input: Image data from inside the drone

[0956] Output: Image data in the device

[0957] Step 3:

[0958] Uploading image data

[0959] The device uploads the stored image data to a server over a high-speed internet connection using the HTTPS protocol.

[0960] Specific actions

[0961] The user uploads image data to the server using a dedicated application on the device, and the progress of the data transfer is displayed until it is complete.

[0962] Input and Output

[0963] Input: Image data in the device

[0964] Output: Image data uploaded to the server

[0965] Step 4:

[0966] Image data preprocessing

[0967] The server performs preprocessing such as noise reduction, normalization, and geometric transformation on the received image data using OpenCV.

[0968] Specific actions

[0969] The server first applies a Gaussian filter to remove noise from the image, then scales the image's pixel values ​​to the range 0 to 1, and then performs a geometric transformation to adjust the image's angle and scale.

[0970] Input and Output

[0971] Input: Image data uploaded to the server

[0972] Output: Preprocessed image data

[0973] Step 5:

[0974] Image data analysis and feature extraction

[0975] The server then feeds the preprocessed image data into a machine learning model (e.g., a convolutional neural network) to extract features such as forest health, tree species, and the location of water sources. TensorFlow is used for the analysis.

[0976] Specific actions

[0977] The server runs the preprocessed image data through the TensorFlow environment and applies a trained neural network model that identifies forest health and other important features from the input image and outputs the results in a list format.

[0978] Input and Output

[0979] Input: Preprocessed image data

[0980] Output: Extracted feature data

[0981] Step 6:

[0982] Saving to the database and generating a risk map

[0983] The server stores the extracted feature data in a database and generates a risk map, which is visualized using QGIS.

[0984] Specific actions

[0985] The server inserts the extracted feature data into a database, runs the risk map generation algorithm, and uses QGIS to generate the risk map and save the results in a map format.

[0986] Input and Output

[0987] Input: extracted feature data

[0988] Output: Risk map

[0989] Step 7:

[0990] Deciding on felling and planting locations

[0991] The server determines the optimal locations for cutting and planting trees based on the risk map, taking into account seasonal information and rainfall forecast data.

[0992] Specific actions

[0993] The server takes into account the risk map along with additional environmental data (e.g., rainfall forecasts and seasonal information) and uses algorithms to identify optimal felling and planting locations. The results are stored in a list format and can be displayed later.

[0994] Input and Output

[0995] Input: Risk map, environmental data

[0996] Output: List of optimal cutting and planting locations

[0997] Step 8:

[0998] Sending planning results

[0999] The server sends the proposed tree-cutting and planting plan to the terminal, which includes geographic information and integrates it into the terminal's map system.

[1000] Specific actions

[1001] The server uses a dedicated API to send the list of determined felling and planting locations to the device, and after the sending process is completed, the device sends a message confirming receipt.

[1002] Input and Output

[1003] Input: List of optimal felling and planting sites

[1004] Output: Planning data sent to the terminal

[1005] Step 9:

[1006] Plan visualization

[1007] The device visually displays the received plan on a map, using open source JavaScript libraries such as OpenLayers and Leaflet to achieve detailed map display.

[1008] Specific actions

[1009] The terminal activates the geographic information system, imports the received planning data, and marks the planned felling and planting locations on a map and displays detailed information for the user's convenience.

[1010] Input and Output

[1011] Input: Planning data

[1012] Output: felling and planting plans displayed on a map

[1013] Step 10:

[1014] User Feedback

[1015] The user checks the displayed plan and provides feedback such as "I would like to postpone the felling of this area." The feedback is sent to the server via the terminal, and the server then modifies the plan.

[1016] Specific actions

[1017] The user checks the plan on the map, selects the part they want to correct, enters the feedback into the terminal, and presses the send button, which sends the correction request to the server.

[1018] Input and Output

[1019] Input: User feedback

[1020] Output: Feedback sent to the server

[1021] Step 11:

[1022] Revise and resubmit the plan

[1023] The server modifies the plan based on feedback from the user and retransmits the newly modified plan to the terminal.

[1024] Specific actions

[1025] The server analyzes the received feedback, applies a plan modification algorithm, and once the modification is complete, sends the plan back to the device.

[1026] Input and Output

[1027] Input: User feedback

[1028] Output: Corrected planning data

[1029] Step 12:

[1030] Implementing the plan and managing progress

[1031] Once the user has finalized the plan, the server will manage the progress based on this plan. The user will periodically report the execution status to the server via their terminal.

[1032] Specific actions

[1033] The user inputs the progress of the work into the terminal and presses the send button to report it to the server, which analyzes the received progress data and updates the progress status.

[1034] Input and Output

[1035] Input: Work progress data

[1036] Output: Latest progress data

[1037] Step 13:

[1038] Monitor progress and readjust

[1039] The server monitors the progress of the work and readjusts the plan as needed based on progress data reported by the user.

[1040] Specific actions

[1041] The server monitors progress data in real time, immediately issuing warnings if there are any problems with the progress or if new risks arise, and also makes any necessary corrections to the plan and notifies the device again.

[1042] Input and Output

[1043] Input: Work progress data

[1044] Output: Realigned planning data

[1045] The above are the processing steps of the program for this system. The techniques and specific operations used in each step have been explained in detail.

[1046] (Application example 1)

[1047] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1048] Conventional factory layout optimization systems are operated based on fixed layout designs and plans, making it difficult to change or optimize in real time. Furthermore, there was a lack of means to instantly understand and correct the efficiency of ongoing work in the actual environment. This resulted in a decline in work efficiency, making it difficult to improve productivity.

[1049] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1050] In this invention, the server includes means for receiving satellite images and images captured by unmanned aerial vehicles, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify the environmental health status and plant species, means for determining optimal work locations and layout plans based on the identified data, means for visualizing the determined work locations and layout plans on a map, means for receiving feedback from a user and revising the plan, means for providing the revised plan to the user and managing execution and progress, and means for monitoring the situation in real time using smart glasses and visually confirming the optimal layout, thereby enabling the optimal factory layout to be confirmed and revised in real time.

[1051] "Satellite imagery" refers to image data taken from Earth's satellites and is used to obtain information on the terrain and environment over a wide area.

[1052] An "unmanned aerial vehicle" is a remotely controlled or autonomously flown aircraft equipped with a specialized camera used to take high-resolution aerial photographs.

[1053] "Image data preprocessing" is a method of performing initial processing such as noise removal, normalization, and geometric transformation on acquired image data, and converting it into a format suitable for analysis.

[1054] "Environmental health" refers to the overall assessment of factors and elements (e.g., plant health, water quality) in a particular natural or man-made environment.

[1055] A "plant type" refers to a classification of plants present in a particular area, identified based on biological categories such as species, genus, or family.

[1056] "Work location" refers to the geographic location that is best suited to carrying out a particular task (e.g., cutting trees or planting trees).

[1057] A "layout plan" is a blueprint or scheme for a field, factory, etc., for planning and devising the optimal placement of specific tasks or machinery.

[1058] "Visualization" is the technique of visually displaying data and information, and converting it into a format that is easily understood and usable by users.

[1059] "Feedback" refers to the process of receiving opinions and requests from users and reflecting them in systems and plans.

[1060] "Smart glasses" are wearable devices that use augmented reality (AR) technology and are glasses-type devices that can display and operate information in real time.

[1061] This invention is a system for efficiently and accurately optimizing factory layouts. This system can improve work efficiency and productivity in real time. Specifically, optimization is performed through collaboration between servers, terminals, and users.

[1062] System configuration

[1063] The system mainly consists of the following components:

[1064] 1. Hardware

[1065] Server: A central processing unit that performs data analysis and planning.

[1066] Smart glasses: Used for real-time situation monitoring and visualization of optimization plans.

[1067] Unmanned aerial vehicle: Used to photograph the layout of the factory.

[1068] 2. Software

[1069] Python: Used for data analysis and running machine learning models.

[1070] OpenCV: A library for image processing.

[1071] Keras: Used to build and manage machine learning models.

[1072] Django: A web application framework.

[1073] Data collection and preprocessing

[1074] Users use unmanned aerial vehicles to take high-resolution images of their factory interiors and upload them to a server via their terminal. The server then performs preprocessing such as noise removal and normalization on the received image data, converting it into a format suitable for analysis. This removes noise from the image data and improves the accuracy of the analysis.

[1075] Data Analysis and Optimization

[1076] The server analyzes the preprocessed image data to determine the layout of equipment and work flow within the factory. A pre-trained machine learning model (generative AI model) is used for the analysis. This model extracts features from the input image data, such as the health status of the environment, the type of equipment, and its location. Based on these extracted features, the server then generates an optimal layout plan and work flow.

[1077] Visualization and feedback of optimization plans

[1078] The optimization plan generated by the server is displayed on the smart glasses. The user can check it in real time and provide feedback as needed. This feedback is sent from the device to the server, which then modifies the plan based on the received feedback. For example, specific instructions such as "I would like to change the equipment layout in this area" can be sent as feedback.

[1079] Implementing the plan and managing progress

[1080] Once the user has finalized the plan, the server will manage the progress based on this plan. Real-time monitoring and feedback makes it easy to revise and readjust the plan, improving work efficiency.

[1081] Specific examples

[1082] For example, consider the case of optimizing the layout of a factory. A user uses an unmanned aerial vehicle to take images of the inside of the factory and uploads this image data from their device to a server. The server preprocesses the image data and analyzes it using a machine learning model to generate an optimal layout plan and work flow. The optimized plan is then presented to the user via smart glasses. The user provides feedback on the plan while checking the actual situation on the site, and the server then modifies the plan, achieving optimization.

[1083] Example prompt sentence:

[1084] "Please build a system that can optimize the acquired factory layout images and visually confirm the results of the optimal layout."

[1085] This system enables the factory layout to be optimized in real time, establishing an efficient and sustainable production system.

[1086] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1087] Step 1:

[1088] The user uses an unmanned aerial vehicle to take high-resolution images of the factory and saves the image data on a terminal. The input is the image data taken by the unmanned aerial vehicle, and the output is an image file saved on the terminal.

[1089] Step 2:

[1090] A user uploads image data to a server via a terminal. The input is an image file stored on the terminal, and the output is the image data sent to the server.

[1091] Step 3:

[1092] The server performs preprocessing on the received image data. Specifically, it performs noise removal, normalization, and geometric transformation on the image data. The input is the image data sent to the server, and the output is the preprocessed image data.

[1093] Step 4:

[1094] The server analyzes the preprocessed image data and identifies the equipment layout and work flow within the factory. This analysis uses a machine learning model (generative AI model). The input is the preprocessed image data, and the output is the equipment layout and work flow data as the analysis results.

[1095] Step 5:

[1096] The server then formulates an optimal layout plan based on the generated equipment placement and work flow data. The input is the analysis results data, and the output is an optimized layout plan.

[1097] Step 6:

[1098] The server provides the optimized layout plan to the user through the smart glasses, where the input is the optimized layout plan and the output is the visual plan displayed on the smart glasses.

[1099] Step 7:

[1100] The user can use the smart glasses to check the layout plan in real time and provide feedback as needed. The input is the feedback information from the user, and the output is the feedback data sent to the server via the terminal.

[1101] Step 8:

[1102] The server modifies the layout plan based on the received user feedback and performs re-optimization. At this time, a new layout plan that reflects the user's opinions is generated. The input is the feedback data, and the output is the modified layout plan.

[1103] Step 9:

[1104] The user checks and approves the final layout plan. An example of a prompt is, "Optimize the acquired factory layout image and build a system that allows visual confirmation of the results of the optimal layout." The input is the revised layout plan, and the output is the approved final layout plan.

[1105] Step 10:

[1106] The server manages progress based on the final approved layout plan and provides a system where users can report the execution status of work in real time. The input is real-time execution status data, and the output is a work schedule with appropriately managed progress.

[1107] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1108] This invention is a system for planning and executing efficient and accurate forest felling and tree planting plans, and by combining it with an emotion engine that recognizes the user's emotions, it provides a more user-friendly interface.

[1109] System Overview

[1110] The system is primarily comprised of a server and a terminal, operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal acts as an interface between the user and the server. The emotion engine recognizes user emotions in real time and is used to optimize the user experience.

[1111] Data collection and preprocessing

[1112] Users use drones or unmanned aerial vehicles to take aerial and ground photographs of forests and store the image data on their devices. The users then upload the data to the server via their devices. The server then performs preprocessing on the received image data, including noise reduction, normalization, and geometric transformation.

[1113] Data analysis and feature extraction

[1114] The server applies machine learning models to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources, generating a risk map. The server then stores the analysis results in a database.

[1115] Planning of felling and planting trees

[1116] The server identifies optimal felling and planting locations based on a risk map. The server takes into account factors such as the extent of diseased trees when identifying felling locations. Soil data, rainfall information, and past success stories of plantings are taken into account when selecting planting locations. Additionally, the server uses seasonal information and rainfall forecasts to determine the optimal timing for felling and planting.

[1117] Emotion engine integration

[1118] The emotion engine analyzes the user's facial expressions and tone of voice in real time on the device to recognize the user's emotional state. For example, if the user looks anxious, the system will provide the user with more detailed explanations and advice. If the user looks satisfied, the system will determine that the proposed plan is acceptable.

[1119] Plan visualization and feedback

[1120] The server sends the optimized felling and planting plan along with map data to the device. The device visualizes the plan on a map and displays it to the user. The user provides feedback on the plan, which the device then sends to the server. The emotion engine optimizes the content and presentation of the feedback based on the user's emotions.

[1121] Implementing the plan and managing progress

[1122] Once the user finalizes the plan, the server manages progress based on this plan. The user periodically reports their execution status to the server via their device. The server monitors progress based on this and readjusts the plan as necessary. The emotion engine adjusts notifications and alarms at appropriate times when the user is feeling stressed.

[1123] Specific examples

[1124] For example, when a user creates a tree planting plan, they first use a drone to take aerial photos of the forest and upload the image data from their device to the server. The server receives the data and pre-processes it. Next, it uses a machine learning model to analyze it and identify the soil quality and the health of existing trees. The optimal planting site is then selected and displayed on a map. When the user reviews the plan and enters feedback, the emotion engine analyzes the user's emotional state and optimizes the feedback content. Finally, when the user executes the plan and reports progress to the server, the emotion engine notifies them at the appropriate time and helps manage progress.

[1125] In this way, the system enables efficient and sustainable forest management based on data, while providing an easy-to-use interface that takes into account the user's emotional state.

[1126] The processing flow will be explained below.

[1127] Program processing flow

[1128] Step 1: Collect data

[1129] Users take aerial and ground photographs of forests using drones or unmanned aerial vehicles, save the image data on their devices, and then upload it to a server via their devices.

[1130] Step 2: Receiving and Preprocessing Data

[1131] The server receives image data uploaded from the device. The server performs preprocessing such as noise removal, normalization, and geometric transformation on the received data. This removes distortion and noise from the image, making it easier to analyze.

[1132] Step 3: Image analysis and feature extraction

[1133] The server applies machine learning models (e.g., convolutional neural networks) to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources. The server stores the analysis results in a database.

[1134] Step 4: Generate a risk map

[1135] The server generates a risk map based on the extracted features, visually indicating the extent of diseased trees, areas that need to be cut down, and areas where tree planting is appropriate.

[1136] Step 5: Identifying the harvest site

[1137] The server runs an algorithm based on the risk map to identify optimal logging sites, taking into account the extent of diseased trees and other environmental protection criteria. The server then stores the list of logging sites.

[1138] Step 6: Select a planting site

[1139] The server selects the optimal planting location based on soil data, rainfall information, and past examples of successful planting. The server then displays the latitude and longitude information of the planting location overlaid on map data.

[1140] Step 7: Generate a time schedule

[1141] The server uses rainfall forecasts and seasonal information to determine the optimal timing for tree cutting and planting, and generates a time schedule that is recorded in a database.

[1142] Step 8: Recognizing user emotions with the emotion engine

[1143] The device analyzes the user's facial expressions and tone of voice in real time to recognize their emotional state, and when the user shows a certain emotion, it optimizes the feedback method according to that emotion.

[1144] Step 9: Visualize the plan

[1145] The server sends the optimized tree-cutting and planting plan along with map data to the device. The device visualizes the plan on the map and displays it to the user. The emotion engine analyzes the user's reaction and displays additional explanations or encouraging messages if necessary.

[1146] Step 10: Gather user feedback

[1147] The user inputs feedback about the displayed plan via the device. For example, they can enter specific instructions such as "I would like to postpone the felling of trees in this area." The device then sends the feedback to the server. The emotion engine analyzes the user's emotions and presents the feedback in an easy-to-understand format.

[1148] Step 11: Incorporate feedback and revise your plan

[1149] The server reconstructs the plan based on the received feedback. The revised plan is sent back to the device and displayed to the user. The emotion engine generates a response based on the user's emotions, encouraging the user to provide positive feedback.

[1150] Step 12: Execute the plan

[1151] The user reviews the final plan and approves it for execution. The device sends the execution status to the server, monitoring the progress of the plan in real time. The emotion engine displays encouraging messages at appropriate times to reduce the user's stress level.

[1152] Step 13: Monitor progress and improve

[1153] The server records the progress of the executed tasks in a database and adjusts the plan again if necessary. The user uploads additional data (e.g., information on new disease outbreaks) from their device to the server for new analysis. The emotion engine monitors the user's emotional state and provides appropriate feedback and responses.

[1154] The above is the specific flow of the program processing of this system, which enables users to achieve efficient and sustainable forest management based on data, while also increasing user satisfaction through the emotion engine.

[1155] Example 2

[1156] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1157] Conventional forest management systems lack the mechanisms for formulating and implementing efficient and accurate forest harvesting and planting plans. In particular, they lack the ability to adjust plans to take users' emotions into account, making it difficult to reduce their stress and anxiety. This poses a risk of reducing the sustainability and efficiency of forest management.

[1158] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1159] In this invention, the server includes means for receiving satellite images and images captured by unmanned aerial vehicles, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify the forest health and tree species, means for determining optimal felling and planting sites based on the identified data, means for visualizing the determined felling and planting sites on a map, means for analyzing the user's facial expressions and tone of voice in real time to recognize the user's emotional state, means for optimizing the interface in accordance with the recognized emotional state, receiving feedback from the user, and revising the plan, and means for providing the revised plan to the user and managing its execution and progress. This enables efficient and accurate forest management, and by taking the user's emotional state into consideration, it is possible to provide a user-friendly interface and reduce stress and anxiety.

[1160] "Satellite imagery" is image data taken of the Earth's surface from space.

[1161] An "unmanned aerial vehicle" is an aircraft that is operated by remote control or automatic pilot and has no personnel on board.

[1162] "Preprocessing" refers to the process of converting received data into a format that is easy to analyze and use, and specifically includes noise removal, normalization, geometric transformation, etc.

[1163] "Analysis" is the process of examining and breaking down data in detail to clarify its content and structure.

[1164] "Forest health" is an indicator of how healthy a forest is, including the growth status of trees and the presence of pests and diseases.

[1165] "Tree type" is a classification of the types of trees present in a particular forest.

[1166] "Cutting point" means [a specific location in the forest where trees should be cut].

[1167] "Planting site" means [a suitable location for planting new trees].

[1168] "Visualization" is a technique that makes data and information easier to understand by displaying them visually.

[1169] "Emotional state" refers to the user's current psychological and emotional state.

[1170] "Interface optimization" refers to adjusting the layout and functionality of an interface to improve the user experience.

[1171] "Feedback" is the process of feeding user reactions and opinions back into the system.

[1172] "Progress management" means monitoring the progress of plans and work and making adjustments or corrections as necessary.

[1173] This invention is a system for planning and executing efficient and accurate forest felling and planting plans, and by combining it with an emotion engine that recognizes user emotions, it provides a more user-friendly interface. The system is primarily composed of a server and a terminal, and is operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal functions as an interface connecting the user and the server. The emotion engine recognizes user emotions in real time and is used to optimize the user experience.

[1174] Data collection and preprocessing

[1175] Users use drones or unmanned aerial vehicles to take aerial and ground photographs of forests and store the image data on their devices. The users then upload the data to the server via their devices. The server then performs preprocessing on the received image data, including noise removal, normalization, and geometric transformation. OpenCV is used for noise removal, and Scikit-Image is used for normalization and geometric transformation.

[1176] Data analysis and feature extraction

[1177] The server applies machine learning models to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources, generating a risk map. The analysis is performed using TensorFlow and PyTorch, and the results are stored in a database.

[1178] Planning of felling and planting trees

[1179] The server uses the risk map to identify optimal felling and planting locations. The server takes into account factors such as the extent of diseased trees when identifying felling locations. Soil data, rainfall information, and past success stories of plantings are taken into account when selecting planting locations. The server also uses seasonal information and rainfall forecasts to determine the optimal timing for felling and planting. This is done using geographic information systems such as ArcGIS and QGIS.

[1180] Emotion engine integration

[1181] The emotion engine, powered by Microsoft Azure Cognitive Services and Affectiva, analyzes the user's facial expressions and tone of voice in real time on the device to recognize their emotional state. If the user looks anxious, the system will provide them with more detailed explanations and advice.

[1182] Plan visualization and feedback

[1183] The server sends the optimized felling and planting plans along with map data to the device. The device visualizes the plans on a map and displays them to the user. For example, the device receives the data in GeoJSON format and visualizes it in a dedicated map app. The user provides feedback on the plans, which the device then sends to the server. The emotion engine optimizes the content and presentation of the feedback based on the user's emotions.

[1184] Implementing the plan and managing progress

[1185] Once the user finalizes the plan, the server manages progress based on this plan. The user periodically reports their execution status to the server via their device. The server monitors progress based on this and readjusts the plan as necessary. The emotion engine adjusts notifications and alarms at appropriate times when the user is feeling stressed.

[1186] Specific examples

[1187] For example, when a user creates a tree planting plan, they first use a drone to take aerial photos of the forest and upload the image data from their device to the server. The server receives the data and pre-processes it. Next, it uses a machine learning model to analyze it and identify the soil quality and the health of existing trees. The optimal planting site is then selected and displayed on a map. When the user reviews the plan and enters feedback, the emotion engine analyzes the user's emotional state and optimizes the feedback content. Finally, when the user executes the plan and reports progress to the server, the emotion engine notifies them at the appropriate time and helps manage progress.

[1188] Example prompt sentence:

[1189] "Please analyze aerial photos of the forest taken by a drone, identify the best locations for planting trees, and display the plan on a map. You should also develop a system that analyzes the user's facial expressions and provides appropriate feedback."

[1190] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1191] Step 1: Data collection and upload

[1192] A user takes aerial and ground photographs in a forest area using a drone or unmanned aerial vehicle. The captured image data (input) is saved on the device. The user then uploads the image data to a server via the device. The user uses a dedicated application on the device and presses the "upload" button to send the data (output).

[1193] Step 2: Preprocessing the data

[1194] The image data received by the server (input) is first denoised using OpenCV's Gaussian filter. Next, the image data is normalised to a range of 0 to 1 to maintain data consistency. Furthermore, geometric transformations such as resizing and rotation are performed to make the data suitable for analysis (output). This results in preprocessed image data.

[1195] Step 3: Data analysis and feature extraction

[1196] The server inputs preprocessed image data (input) into a TensorFlow or PyTorch machine learning model. The model then analyzes the data (data processing) to extract features such as forest health, tree species, and water source locations. The analysis results (output) are stored in a database in JSON format.

[1197] Step 4: Planning for felling and planting

[1198] The server generates a risk map and identifies optimal felling and planting locations based on the analysis results (input). The server also considers soil data, rainfall information, and past successful planting cases to create a plan (data calculation). It also determines the optimal timing based on seasonal information and rainfall forecasts (output). This is done using a geographic information system (GIS).

[1199] Step 5: Integrating the Emotion Engine

[1200] The device's camera and microphone are activated, and the user's facial expressions and tone of voice are analyzed in real time (input). For example, Microsoft Azure Cognitive Services or Affectiva are used. Based on the analysis results (output), the device dynamically adjusts the interface and provides detailed explanations and advice to the user.

[1201] Step 6: Visualize and feedback the plan

[1202] The server sends the optimized felling and planting plan (input) along with map data to the device. The device visualizes the plan received in GeoJSON format on a map and displays it to the user (output). The user checks the plan and fills in a feedback form. The device sends the user's feedback to the server, and the emotion engine optimizes the way the feedback is presented based on the user's emotions.

[1203] Step 7: Implement the plan and track progress

[1204] The user finalizes the plan and presses the "Start Execution" button on the device (input). The server starts managing progress based on the plan and monitors it in real time (output). The user periodically reports the execution status via the device, and the server checks and adjusts the progress. The emotion engine analyzes the user's stress state and issues notifications and alarms at appropriate times.

[1205] Specific examples

[1206] For example, if a user were to create a tree planting plan, the process would be as follows: The user would take aerial photos of the forest with a drone, save them on their device, and upload them to the server. The server would use OpenCV to remove noise from the images and a TensorFlow model to identify soil quality and the health of existing trees. Using this information, the optimal planting locations would be identified and displayed on a map. The user would review the plan and provide feedback, and the emotion engine would analyze that feedback and optimize the presentation. Finally, the user would execute the plan, and the server would monitor progress in real time.

[1207] (Application example 2)

[1208] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1209] In conventional forest management systems, it is necessary to consider not only forest health and tree planting optimization, but also the psychological state of workers. In particular, to improve the efficiency of workers who are susceptible to fatigue and stress, a system that can grasp their emotional state and provide appropriate feedback is needed.

[1210] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1211] In this invention, the server includes means for receiving satellite images and images captured by unmanned aerial vehicles, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify the forest health and tree species, means for determining optimal felling and planting sites based on the identified data, means for visualizing the determined felling and planting sites on a map, means for recognizing user emotions in real time and optimizing the content and presentation method of the feedback, means for receiving user feedback and revising the plan, and means for providing the revised plan to the user and managing its execution and progress. This enables a user-friendly system that takes into account the psychological state of workers while improving the efficiency and accuracy of forest management.

[1212] "Satellite imagery" refers to image data taken from an artificial satellite in orbit around the Earth.

[1213] An "unmanned aerial vehicle" is an aircraft that flies remotely or autonomously without a human on board.

[1214] "Preprocessing" refers to processes such as noise removal, normalization, and geometric transformation of image data before analysis.

[1215] "Forest health" refers to the state of the forest, indicating whether the trees and vegetation within it are growing healthily.

[1216] "Tree types" refers to the classification of the various trees present in the forest.

[1217] A "harvesting point" is a location within a forest where selected trees are recommended for felling.

[1218] A "planting site" is a location selected for planting new trees.

[1219] "Visualization" refers to the visual representation of numbers and data in an easy-to-understand way.

[1220] "User emotion" refers to the emotional state felt by the person using the system at that moment.

[1221] "Real-time recognition" means that the system processes data as soon as it receives it and provides analysis results immediately.

[1222] "Feedback" refers to the opinions and requests provided by users and any resulting adjustments to the system.

[1223] "Plan modification" means receiving feedback from users and modifying an existing plan based on this feedback.

[1224] Progress management is the process of monitoring the progress of a plan and making adjustments as needed.

[1225] The system for realizing this invention uses a server, a terminal, and, if necessary, multiple input devices. The server collects data, preprocesses it, analyzes it, creates plans, and manages progress, while the terminal links the user and the server and provides an interface. Specifically, the system is implemented as follows:

[1226] First, the user takes aerial and ground photographs of the forest using an unmanned aerial vehicle or satellite. This image data is stored on the device and the user uploads it to the server via the device. The server performs preprocessing on the received image data, such as noise removal, normalization, and geometric transformation. Libraries such as OpenCV and TensorFlow are used for image processing.

[1227] Next, the server applies machine learning models to the preprocessed image data, extracting features such as forest health, tree species, and the location of water sources. A pre-trained deep learning model (TensorFlow / Keras) is used for feature extraction, generating a risk map.

[1228] The server then uses the risk map to identify optimal felling and planting locations. The server takes into account factors such as diseased trees to identify felling locations, and soil data, rainfall information, and past successful plantings to select planting locations. Seasonal information and rainfall forecasts are also used to determine the optimal timing for felling and planting.

[1229] The server also incorporates an emotion engine that recognizes the user's emotions in real time. This emotion engine analyzes the user's facial expressions and tone of voice on the device to identify the user's emotional state. For example, if the user is feeling tired or stressed, the system will notify them at the appropriate time to take a break. EmotionRecognition is often used as an emotion recognition library.

[1230] The server then receives feedback from the user, analyzes it, and modifies the plan. The modified plan is then reflected on the map and sent to the device. The user then checks the plan on the device and reports the final execution status to the server. The server uses this information to monitor progress and make adjustments as necessary.

[1231] As a concrete example, when a user creates a tree planting plan, they first take aerial photos of the forest using an unmanned aerial vehicle and upload the image data from their device to a server. The server preprocesses this data and analyzes it using machine learning models to identify soil quality and the health of existing trees. The server then selects the optimal planting site and displays it on a map. When the user reviews the plan and enters feedback, the emotion engine analyzes the user's emotional state and optimizes the feedback content. Finally, when the user executes the plan and reports progress to the server, the emotion engine notifies them at the appropriate time and helps manage progress.

[1232] An example prompt might be, "This program is a forest management system installed in an autonomous vehicle. Write Python code that analyzes the following content and generates optimal felling and planting plans. Additionally, add a notification function to recognize the emotional state of the worker and reduce stress."

[1233] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1234] Step 1:

[1235] Users use unmanned aerial vehicles or satellites to take aerial and ground photographs of forests. These image data are stored on the device. The inputs are aerial and ground photographs, and the output is image data stored on the device. The user operates the image capture device to collect forest data over a wide area.

[1236] Step 2:

[1237] A user uploads image data to a server via a terminal. The input is the stored image data, and the output is the image data transferred to the server. The terminal uses an Internet connection to quickly transmit large amounts of image data to the server.

[1238] Step 3:

[1239] The server performs preprocessing on the received image data, such as noise removal, normalization, and geometric transformation. The input is the uploaded image data, and the output is the preprocessed image data. The OpenCV library is used to improve the image quality and make it suitable for analysis.

[1240] Step 4:

[1241] The server applies machine learning models based on the preprocessed image data to identify forest health, tree species, water source locations, etc. The input is the preprocessed image data, and the output is the identified feature data. A detailed analysis of the forest is performed using the TensorFlow / Keras library.

[1242] Step 5:

[1243] The server generates a risk map based on the identified data. The input is the identified feature data, and the output is a risk map. The server integrates the analysis results into a geographic information system (GIS) to visualize defects and dangerous areas.

[1244] Step 6:

[1245] The server identifies optimal felling and planting locations based on the risk map. The input is the risk map, and the output is optimized felling and planting locations. The server uses algorithms to generate an efficient work plan.

[1246] Step 7:

[1247] The server visualizes the optimal felling and planting locations on a map and sends it to the device. The input is the optimized location data, and the output is the visualized data on the map. The server uses a visualization tool to convert the data into a format that is easy to view on the device.

[1248] Step 8:

[1249] The user checks the plan contents and provides feedback using a device. The input is the plan data visualized on a map, and the output is the user's feedback data. The user interactively checks the plan on the device and suggests any necessary modifications.

[1250] Step 9:

[1251] The server receives user feedback and recognizes the user's emotional state in real time using an emotion engine. The inputs are feedback data and real-time emotion data, and the output is emotion recognition results. The server uses the EmotionRecognition library to understand the user's feelings.

[1252] Step 10:

[1253] The server modifies the plan based on the feedback and emotion recognition results. The inputs are user feedback and emotion recognition results, and the output is modified plan data. The server analyzes the feedback information and automatically makes necessary adjustments.

[1254] Step 11:

[1255] The revised plan is resent to the terminal and presented to the user. The input is the revised plan data, and the output is the final plan data presented to the user. The user confirms the revised plan on the terminal and starts implementing it.

[1256] Step 12:

[1257] The user executes the plan and reports the progress to the server. The input is data on the work being performed, and the output is progress report data. The user periodically sends progress information to the server using their terminal, allowing the progress of the work to be monitored in real time.

[1258] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1259] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1260] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1261] [Fourth embodiment]

[1262] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1263] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1264] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1265] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1266] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1267] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1268] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1269] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1270] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1271] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1272] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1273] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1274] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1275] This invention is a system for planning and executing efficient and highly accurate forest felling and tree planting plans. The specific details of this system are described below.

[1276] System Overview

[1277] The system is primarily composed of a server and a terminal, and is operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal functions as an interface connecting the user and the server. The user operates the system via the terminal to collect data, confirm plans, and execute them.

[1278] Data collection and preprocessing

[1279] Users take aerial photographs (including satellite images) of forests using drones or unmanned aerial vehicles and store the image data on their devices. The devices then upload the image data to a server, which then performs preprocessing on the received image data. The preprocessing includes noise removal, normalization, and geometric transformation.

[1280] Data analysis and feature extraction

[1281] The server analyzes the preprocessed image data and extracts features such as forest health, tree species, and the location of water sources. This analysis uses machine learning models (e.g., convolutional neural networks). The server stores these features in a database and generates a risk map.

[1282] Planning of felling and planting trees

[1283] The server uses the risk map to identify optimal felling and planting locations. It considers the extent of diseased trees when identifying felling locations. It also considers soil data, rainfall information, and past success stories when selecting planting locations. It also uses seasonal information and rainfall forecasts to determine the optimal timing for felling and planting.

[1284] Plan visualization and feedback

[1285] The server sends the plan results to the device, which displays them visually on a map. The user provides feedback on the displayed plan, including specific instructions such as "I would like to postpone cutting trees in this area." The device then sends this feedback to the server, which then modifies the plan.

[1286] Implementing the plan and managing progress

[1287] Once the user has finalized the plan, the server will manage the progress based on this plan. The user will periodically report the execution status to the server via their terminal. The server will monitor the progress based on this data and adjust the plan as necessary.

[1288] Specific examples

[1289] For example, consider a user planning a tree planting project. First, the user takes aerial photos of the forest using a drone and uploads the image data from their device to a server. The server receives the data and pre-processes it. Next, the server analyzes it using machine learning models to identify soil quality and the health of existing trees. It then selects optimal planting locations and displays them on a map. The user reviews the plan and provides feedback. Finally, the user executes the plan and periodically reports progress to the server.

[1290] This system enables users to achieve efficient and sustainable forest management.

[1291] The processing flow will be explained below.

[1292] Program processing flow

[1293] Step 1: Collect data

[1294] Users take aerial and ground photographs of forests using drones or unmanned aerial vehicles, save the image data on their devices, and then upload it to a server via their devices.

[1295] Step 2: Receiving and Preprocessing Data

[1296] The server receives image data uploaded from the device. The server performs preprocessing such as noise removal, normalization, and geometric transformation on the received data. In this process, image distortion is corrected, improving the accuracy of analysis.

[1297] Step 3: Image analysis and feature extraction

[1298] The server applies machine learning models (e.g., convolutional neural networks) to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources. The server stores the analysis results in a database.

[1299] Step 4: Generate a risk map

[1300] The server generates a risk map based on the extracted features, visually indicating the extent of diseased trees, areas that need to be cut down, and areas where tree planting is appropriate.

[1301] Step 5: Identifying the harvest site

[1302] The server runs an algorithm based on the risk map to identify optimal felling sites, taking into account necessary environmental protection criteria such as the extent of diseased trees, and stores the list of felling sites.

[1303] Step 6: Select a planting site

[1304] The server selects the optimal planting location based on soil data, rainfall information, and past examples of successful planting. The server then displays the latitude and longitude information of the planting location overlaid on map data.

[1305] Step 7: Generate a time schedule

[1306] The server uses rainfall forecasts and seasonal information to determine the optimal timing for tree cutting and planting, and generates a time schedule that is recorded in a database.

[1307] Step 8: Visualize the plan

[1308] The server sends the optimized felling and planting plan along with map data to the terminal, which visualizes the plan on a map and displays it to the user.

[1309] Step 9: Gather user feedback

[1310] The user inputs feedback about the displayed plan via the terminal. For example, the user inputs an instruction such as "I would like to postpone the felling of this area." The terminal then sends the feedback to the server.

[1311] Step 10: Incorporate feedback and revise your plan

[1312] The server reconstructs the plan based on the received feedback, and the revised plan is sent back to the device and displayed to the user.

[1313] Step 11: Execute the plan

[1314] The user reviews the final plan and approves it for execution. The device sends the execution status to the server, allowing the progress of the plan to be monitored in real time.

[1315] Step 12: Monitor progress and improve

[1316] The server records the progress of the executed tasks in a database and adjusts the plan again if necessary. The user uploads additional data (e.g., information on new disease outbreaks) from their device to the server, and a new analysis is performed.

[1317] This is the flow of the program processing for this system, which enables users to achieve efficient and sustainable forest management based on data.

[1318] Example 1

[1319] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1320] Conventional forest management systems have faced the challenge of making efficient and accurate plans for felling and planting. Specifically, there was little centralized analysis of image data obtained from satellite images and aerial photography, selection of optimal felling and planting locations, or feedback and revision of plans. This resulted in a decrease in the accuracy of plans and the efficiency of their execution, making it difficult to achieve sustainable forest management.

[1321] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1322] In this invention, the server includes means for receiving satellite images and images captured by an aerial imaging device, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify forest health, tree species, and water source locations, means for determining optimal felling and planting sites based on the identified data, means for visualizing the determined felling and planting sites on a map, means for receiving feedback from a user and revising the plan, means for providing the revised plan to the user and managing its execution and progress, means for analyzing the image data using a machine learning model, and means for receiving additional data from the user and readjusting the plan, thereby enabling the efficient and accurate planning and execution of felling and planting plans.

[1323] "Satellite imagery" refers to images taken by satellites orbiting the Earth, and is data that provides information on a wide range of the Earth's surface.

[1324] An "aerial photography device" is a device, such as a drone or unmanned aerial vehicle, used to take photographs or videos of the ground from a high altitude.

[1325] "Preprocessing" refers to processes such as noise removal, normalization, and geometric transformation that are performed on image data to be analyzed, and is a process for improving the quality of the data.

[1326] "Forest health" refers to the overall state of the forest ecosystem, including the state of plant growth and the occurrence of pests and diseases.

[1327] "Tree species" refers to the types of trees that grow in a particular area or forest.

[1328] "Location of water sources" refers to location information of rivers, lakes, and other places where water exists within a forest.

[1329] "Optimal harvesting site" refers to a suitable location chosen to harvest trees efficiently and sustainably.

[1330] "Optimal planting site" refers to the suitable location selected for planting new trees.

[1331] "Visualization" refers to the graphical display of data and planning results on a map.

[1332] "Feedback" refers to information provided by users to modify and improve the plan through their opinions and instructions.

[1333] A "machine learning model" refers to an algorithm or model that learns patterns and features from data and predicts and analyzes future data.

[1334] "Additional data" refers to new information or data provided by the user that serves as the basis for revising or readjusting the plan.

[1335] "Progress" refers to the progress of checking the execution status and degree of achievement of a plan.

[1336] This invention is a system for planning and executing efficient and highly accurate forest felling and tree planting plans. The specific details of this system are described below.

[1337] The system is primarily composed of a server and a terminal, and is operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal functions as an interface connecting the user and the server. The user operates the system via the terminal to collect data, confirm plans, and execute them.

[1338] Data collection and preprocessing

[1339] Users use aerial photography equipment such as drones to take aerial or satellite images of forests and store the image data on their devices. The devices then upload the image data to a server. A high-speed Internet connection is required for uploading, and the HTTPS protocol is used. The server then performs preprocessing on the received image data. This preprocessing includes noise removal, normalization, and geometric transformation. OpenCV can be used for this.

[1340] Data analysis and feature extraction

[1341] The server analyzes the preprocessed image data and extracts features such as forest health, tree species, and the location of water sources. This analysis uses a convolutional neural network (CNN), a machine learning model, using TensorFlow and PyTorch. The server stores these features in a database and generates a risk map. QGIS can be used to visualize the risk map.

[1342] Planning of felling and planting trees

[1343] The server identifies optimal felling and planting locations based on the risk map. The location of felling locations is determined by taking into account the extent of diseased trees and other risk factors. The planting locations are selected based on soil data, rainfall information, and past success stories. Seasonal information and rainfall forecast data are also used to determine the optimal timing for felling and planting. This process utilizes environmental databases and weather data APIs.

[1344] Plan visualization and feedback

[1345] The server sends the proposed tree-cutting and planting plan to the device, which then visually displays it on a map. Open-source JavaScript libraries such as OpenLayers and Leaflet are used to display the map. The user checks the displayed plan and provides feedback, such as "I would like to postpone tree-cutting in this area." The feedback is sent via the device to the server, which then modifies the plan.

[1346] Implementing the plan and managing progress

[1347] Once the user has finalized the plan, the server will manage progress based on this plan. The user periodically reports the execution status to the server via their terminal. The reports include the progress and any issues. The server will monitor the progress based on this data and readjust the plan as necessary. Project management tools such as Asana and JIRA can be used to manage progress.

[1348] Examples of concrete examples and prompts

[1349] For example, consider a case where a user is creating a tree planting plan. The user first uses a drone to take aerial photos of the forest and uploads them from their device to the server. The server preprocesses the received data using OpenCV and then uses TensorFlow to identify soil quality and the health of existing trees. It then generates a risk map using QGIS. The server selects optimal planting sites and sends the plan to the device. The device displays the plan on the map, and the user provides feedback, such as "I would like to postpone tree felling in this area." This feedback is sent to the server through the Django framework, which then modifies the plan. Finally, the user finalizes the plan and periodically reports the progress of the work to the server. The server monitors the progress using Asana.

[1350] Example prompt sentence:

[1351] “You upload aerial photos of your forest. We preprocess the data and analyze it using machine learning models. We identify the best locations for cutting and planting trees and modify the plan based on your feedback.”

[1352] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1353] Step 1:

[1354] Data Acquisition

[1355] Users use drones or unmanned aerial vehicles to take aerial photographs or satellite images of forests, and the captured image data is stored in the drone's internal storage.

[1356] Specific actions

[1357] To take aerial photos of a forest, a user launches the drone and sets a designated flight path. The drone then flies according to the settings and takes photos of the designated area with its high-resolution camera.

[1358] Input and Output

[1359] Input: Flight path, shooting instructions

[1360] Output: Aerial photographs taken

[1361] Step 2:

[1362] Saving image data

[1363] The user transfers the captured image data from the drone to a device and stores it on a high-performance SSD.

[1364] Specific actions

[1365] The user connects a data transfer cable from the drone to the device and downloads the image data to the device, where it is saved in the device's storage.

[1366] Input and Output

[1367] Input: Image data from inside the drone

[1368] Output: Image data in the device

[1369] Step 3:

[1370] Uploading image data

[1371] The device uploads the stored image data to a server over a high-speed internet connection using the HTTPS protocol.

[1372] Specific actions

[1373] The user uploads image data to the server using a dedicated application on the device, and the progress of the data transfer is displayed until it is complete.

[1374] Input and Output

[1375] Input: Image data in the device

[1376] Output: Image data uploaded to the server

[1377] Step 4:

[1378] Image data preprocessing

[1379] The server performs preprocessing such as noise reduction, normalization, and geometric transformation on the received image data using OpenCV.

[1380] Specific actions

[1381] The server first applies a Gaussian filter to remove noise from the image, then scales the image's pixel values ​​to the range 0 to 1, and then performs a geometric transformation to adjust the image's angle and scale.

[1382] Input and Output

[1383] Input: Image data uploaded to the server

[1384] Output: Preprocessed image data

[1385] Step 5:

[1386] Image data analysis and feature extraction

[1387] The server then feeds the preprocessed image data into a machine learning model (e.g., a convolutional neural network) to extract features such as forest health, tree species, and the location of water sources. TensorFlow is used for the analysis.

[1388] Specific actions

[1389] The server runs the preprocessed image data through the TensorFlow environment and applies a trained neural network model that identifies forest health and other important features from the input image and outputs the results in a list format.

[1390] Input and Output

[1391] Input: Preprocessed image data

[1392] Output: Extracted feature data

[1393] Step 6:

[1394] Saving to the database and generating a risk map

[1395] The server stores the extracted feature data in a database and generates a risk map, which is visualized using QGIS.

[1396] Specific actions

[1397] The server inserts the extracted feature data into a database, runs the risk map generation algorithm, and uses QGIS to generate the risk map and save the results in a map format.

[1398] Input and Output

[1399] Input: extracted feature data

[1400] Output: Risk map

[1401] Step 7:

[1402] Deciding on felling and planting locations

[1403] The server determines the optimal locations for cutting and planting trees based on the risk map, taking into account seasonal information and rainfall forecast data.

[1404] Specific actions

[1405] The server takes into account the risk map along with additional environmental data (e.g., rainfall forecasts and seasonal information) and uses algorithms to identify optimal felling and planting locations. The results are stored in a list format and can be displayed later.

[1406] Input and Output

[1407] Input: Risk map, environmental data

[1408] Output: List of optimal cutting and planting locations

[1409] Step 8:

[1410] Sending planning results

[1411] The server sends the proposed tree-cutting and planting plan to the terminal, which includes geographic information and integrates it into the terminal's map system.

[1412] Specific actions

[1413] The server uses a dedicated API to send the list of determined felling and planting locations to the device, and after the sending process is completed, the device sends a message confirming receipt.

[1414] Input and Output

[1415] Input: List of optimal felling and planting sites

[1416] Output: Planning data sent to the terminal

[1417] Step 9:

[1418] Plan visualization

[1419] The device visually displays the received plan on a map, using open source JavaScript libraries such as OpenLayers and Leaflet to achieve detailed map display.

[1420] Specific actions

[1421] The terminal activates the geographic information system, imports the received planning data, and marks the planned felling and planting locations on a map and displays detailed information for the user to easily understand.

[1422] Input and Output

[1423] Input: Planning data

[1424] Output: felling and planting plans displayed on a map

[1425] Step 10:

[1426] User Feedback

[1427] The user checks the displayed plan and provides feedback such as "I would like to postpone the felling of this area." The feedback is sent to the server via the terminal, and the server then modifies the plan.

[1428] Specific actions

[1429] The user checks the plan on the map, selects the part they want to correct, enters the feedback into the terminal, and presses the send button, which sends the correction request to the server.

[1430] Input and Output

[1431] Input: User feedback

[1432] Output: Feedback sent to the server

[1433] Step 11:

[1434] Revise and resubmit the plan

[1435] The server modifies the plan based on feedback from the user and retransmits the newly modified plan to the terminal.

[1436] Specific actions

[1437] The server analyzes the received feedback, applies a plan modification algorithm, and once the modification is complete, sends the plan back to the device.

[1438] Input and Output

[1439] Input: User feedback

[1440] Output: revised planning data

[1441] Step 12:

[1442] Implementing the plan and managing progress

[1443] Once the user has finalized the plan, the server will manage the progress based on this plan. The user will periodically report the execution status to the server via their terminal.

[1444] Specific actions

[1445] The user inputs the progress of the work into the terminal and presses the send button to report it to the server, which analyzes the received progress data and updates the progress status.

[1446] Input and Output

[1447] Input: Work progress data

[1448] Output: Latest progress data

[1449] Step 13:

[1450] Monitor progress and readjust

[1451] The server monitors the progress of the work and readjusts the plan as needed based on progress data reported by the user.

[1452] Specific actions

[1453] The server monitors progress data in real time, immediately issuing warnings if there are any problems with the progress or if new risks arise, and also makes any necessary corrections to the plan and notifies the device again.

[1454] Input and Output

[1455] Input: Work progress data

[1456] Output: Realigned planning data

[1457] The above are the processing steps of the program for this system. The techniques and specific operations used in each step have been explained in detail.

[1458] (Application example 1)

[1459] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1460] Conventional factory layout optimization systems are operated based on fixed layout designs and plans, making it difficult to change or optimize in real time. Furthermore, there was a lack of means to instantly understand and correct the efficiency of ongoing work in the actual environment. This resulted in a decline in work efficiency, making it difficult to improve productivity.

[1461] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1462] In this invention, the server includes means for receiving satellite images and images captured by unmanned aerial vehicles, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify the environmental health status and plant species, means for determining optimal work locations and layout plans based on the identified data, means for visualizing the determined work locations and layout plans on a map, means for receiving feedback from a user and revising the plan, means for providing the revised plan to the user and managing execution and progress, and means for monitoring the situation in real time using smart glasses and visually confirming the optimal layout, thereby enabling the optimal factory layout to be confirmed and revised in real time.

[1463] "Satellite imagery" refers to image data taken from Earth's satellites and is used to obtain information on the terrain and environment over a wide area.

[1464] An "unmanned aerial vehicle" is a remotely controlled or autonomously flown aircraft equipped with a specialized camera used to take high-resolution aerial photographs.

[1465] "Image data preprocessing" is a method of performing initial processing such as noise removal, normalization, and geometric transformation on acquired image data, and converting it into a format suitable for analysis.

[1466] "Environmental health" refers to the overall assessment of factors and elements (e.g., plant health, water quality) in a particular natural or man-made environment.

[1467] A "plant type" refers to a classification of plants present in a particular area, identified based on biological categories such as species, genus, or family.

[1468] "Work location" refers to the geographic location that is best suited to carrying out a particular task (e.g., cutting trees or planting trees).

[1469] A "layout plan" is a blueprint or scheme for a field, factory, etc., for planning and devising the optimal placement of specific tasks or machinery.

[1470] "Visualization" is the technique of visually displaying data and information, and converting it into a format that is easily understood and usable by users.

[1471] "Feedback" refers to the process of receiving opinions and requests from users and reflecting them in systems and plans.

[1472] "Smart glasses" are wearable devices that use augmented reality (AR) technology and are glasses-type devices that can display and operate information in real time.

[1473] This invention is a system for efficiently and accurately optimizing factory layouts. This system can improve work efficiency and productivity in real time. Specifically, optimization is performed through collaboration between servers, terminals, and users.

[1474] System configuration

[1475] The system mainly consists of the following components:

[1476] 1. Hardware

[1477] Server: A central processing unit that performs data analysis and planning.

[1478] Smart glasses: Used for real-time situation monitoring and visualization of optimization plans.

[1479] Unmanned aerial vehicle: Used to photograph the layout of the factory.

[1480] 2. Software

[1481] Python: Used for data analysis and running machine learning models.

[1482] OpenCV: A library for image processing.

[1483] Keras: Used to build and manage machine learning models.

[1484] Django: A web application framework.

[1485] Data collection and preprocessing

[1486] Users use unmanned aerial vehicles to take high-resolution images of their factory interiors and upload them to a server via their terminal. The server then performs preprocessing such as noise removal and normalization on the received image data, converting it into a format suitable for analysis. This removes noise from the image data and improves the accuracy of the analysis.

[1487] Data Analysis and Optimization

[1488] The server analyzes the preprocessed image data to determine the layout of equipment and work flow within the factory. A pre-trained machine learning model (generative AI model) is used for the analysis. This model extracts features from the input image data, such as the health status of the environment, the type of equipment, and its location. Based on these extracted features, the server then generates an optimal layout plan and work flow.

[1489] Visualization and feedback of optimization plans

[1490] The optimization plan generated by the server is displayed on the smart glasses. The user can check it in real time and provide feedback as needed. This feedback is sent from the device to the server, which then modifies the plan based on the received feedback. For example, specific instructions such as "I would like to change the equipment layout in this area" can be sent as feedback.

[1491] Implementing the plan and managing progress

[1492] Once the user has finalized the plan, the server will manage the progress based on this plan. Real-time monitoring and feedback makes it easy to revise and readjust the plan, improving work efficiency.

[1493] Specific examples

[1494] For example, consider the case of optimizing the layout of a factory. A user uses an unmanned aerial vehicle to take images of the inside of the factory and uploads this image data from their device to a server. The server preprocesses the image data and analyzes it using a machine learning model to generate an optimal layout plan and work flow. The optimized plan is then presented to the user via smart glasses. The user provides feedback on the plan while checking the actual situation on the site, and the server then modifies the plan, achieving optimization.

[1495] Example prompt sentence:

[1496] "Please build a system that can optimize the acquired factory layout images and visually confirm the results of the optimal layout."

[1497] This system enables the factory layout to be optimized in real time, establishing an efficient and sustainable production system.

[1498] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1499] Step 1:

[1500] The user uses an unmanned aerial vehicle to take high-resolution images of the factory and saves the image data on a terminal. The input is the image data taken by the unmanned aerial vehicle, and the output is an image file saved on the terminal.

[1501] Step 2:

[1502] A user uploads image data to a server via a terminal. The input is an image file stored on the terminal, and the output is the image data sent to the server.

[1503] Step 3:

[1504] The server performs preprocessing on the received image data. Specifically, it performs noise removal, normalization, and geometric transformation on the image data. The input is the image data sent to the server, and the output is the preprocessed image data.

[1505] Step 4:

[1506] The server analyzes the preprocessed image data and identifies the equipment layout and work flow within the factory. This analysis uses a machine learning model (generative AI model). The input is the preprocessed image data, and the output is the equipment layout and work flow data as the analysis results.

[1507] Step 5:

[1508] The server then formulates an optimal layout plan based on the generated data on equipment placement and work flow. The input is the analysis results data, and the output is an optimized layout plan.

[1509] Step 6:

[1510] The server provides the optimized layout plan to the user through the smart glasses, where the input is the optimized layout plan and the output is the visual plan displayed on the smart glasses.

[1511] Step 7:

[1512] The user can use the smart glasses to check the layout plan in real time and provide feedback as needed. The input is the feedback information from the user, and the output is the feedback data sent to the server via the terminal.

[1513] Step 8:

[1514] The server modifies the layout plan based on the received user feedback and performs re-optimization. At this time, a new layout plan that reflects the user's opinions is generated. The input is the feedback data, and the output is the modified layout plan.

[1515] Step 9:

[1516] The user checks and approves the final layout plan. An example of a prompt is, "Optimize the acquired factory layout image and build a system that allows visual confirmation of the results of the optimal layout." The input is the revised layout plan, and the output is the approved final layout plan.

[1517] Step 10:

[1518] The server manages progress based on the final approved layout plan and provides a system where users can report the execution status of work in real time. The input is real-time execution status data, and the output is a work schedule with appropriately managed progress.

[1519] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1520] This invention is a system for planning and executing efficient and accurate forest felling and tree planting plans, and by combining it with an emotion engine that recognizes the user's emotions, it provides a more user-friendly interface.

[1521] System Overview

[1522] The system is primarily comprised of a server and a terminal, operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal acts as an interface between the user and the server. The emotion engine recognizes user emotions in real time and is used to optimize the user experience.

[1523] Data collection and preprocessing

[1524] Users use drones or unmanned aerial vehicles to take aerial and ground photographs of forests and store the image data on their devices. The users then upload the data to the server via their devices. The server then performs preprocessing on the received image data, including noise reduction, normalization, and geometric transformation.

[1525] Data analysis and feature extraction

[1526] The server applies machine learning models to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources, generating a risk map. The server then stores the analysis results in a database.

[1527] Planning of felling and planting trees

[1528] The server identifies optimal felling and planting locations based on a risk map. The server takes into account factors such as the extent of diseased trees when identifying felling locations. Soil data, rainfall information, and past success stories of plantings are taken into account when selecting planting locations. Additionally, the server uses seasonal information and rainfall forecasts to determine the optimal timing for felling and planting.

[1529] Emotion engine integration

[1530] The emotion engine analyzes the user's facial expressions and tone of voice in real time on the device to recognize the user's emotional state. For example, if the user looks anxious, the system will provide the user with more detailed explanations and advice. If the user looks satisfied, the system will determine that the proposed plan is acceptable.

[1531] Plan visualization and feedback

[1532] The server sends the optimized felling and planting plan along with map data to the device. The device visualizes the plan on a map and displays it to the user. The user provides feedback on the plan, which the device then sends to the server. The emotion engine optimizes the content and presentation of the feedback based on the user's emotions.

[1533] Implementing the plan and managing progress

[1534] Once the user finalizes the plan, the server manages progress based on this plan. The user periodically reports their execution status to the server via their device. The server monitors progress based on this and readjusts the plan as necessary. The emotion engine adjusts notifications and alarms at appropriate times when the user is feeling stressed.

[1535] Specific examples

[1536] For example, when a user creates a tree planting plan, they first use a drone to take aerial photos of the forest and upload the image data from their device to the server. The server receives the data and pre-processes it. Next, it uses a machine learning model to analyze it and identify the soil quality and the health of existing trees. The optimal planting site is then selected and displayed on a map. When the user reviews the plan and enters feedback, the emotion engine analyzes the user's emotional state and optimizes the feedback content. Finally, when the user executes the plan and reports progress to the server, the emotion engine notifies them at the appropriate time and helps manage progress.

[1537] In this way, the system enables efficient and sustainable forest management based on data, while providing an easy-to-use interface that takes into account the user's emotional state.

[1538] The processing flow will be explained below.

[1539] Program processing flow

[1540] Step 1: Collect data

[1541] Users take aerial and ground photographs of forests using drones or unmanned aerial vehicles, save the image data on their devices, and then upload it to a server via their devices.

[1542] Step 2: Receiving and Preprocessing Data

[1543] The server receives the image data uploaded from the device. The server performs preprocessing such as noise removal, normalization, and geometric transformation on the received data. This removes distortion and noise from the image, making it easier to analyze.

[1544] Step 3: Image analysis and feature extraction

[1545] The server applies machine learning models (e.g., convolutional neural networks) to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources. The server stores the analysis results in a database.

[1546] Step 4: Generate a risk map

[1547] The server generates a risk map based on the extracted features, visually indicating the extent of diseased trees, areas that need to be cut down, and areas where tree planting is appropriate.

[1548] Step 5: Identifying the harvest site

[1549] The server runs an algorithm based on the risk map to identify optimal logging sites, taking into account the extent of diseased trees and other environmental protection criteria. The server then stores the list of logging sites.

[1550] Step 6: Select a planting site

[1551] The server selects the optimal planting location based on soil data, rainfall information, and past examples of successful planting. The server then displays the latitude and longitude information of the planting location overlaid on map data.

[1552] Step 7: Generate a time schedule

[1553] The server uses rainfall forecasts and seasonal information to determine the optimal timing for tree cutting and planting, and generates a time schedule that is recorded in a database.

[1554] Step 8: Recognizing user emotions with the emotion engine

[1555] The device analyzes the user's facial expressions and tone of voice in real time to recognize their emotional state, and when the user shows a certain emotion, it optimizes the feedback method according to that emotion.

[1556] Step 9: Visualize the plan

[1557] The server sends the optimized tree-cutting and planting plan along with map data to the device. The device visualizes the plan on the map and displays it to the user. The emotion engine analyzes the user's reaction and displays additional explanations or encouraging messages if necessary.

[1558] Step 10: Gather user feedback

[1559] The user inputs feedback about the displayed plan via the device. For example, they can enter specific instructions such as "I would like to postpone the felling of trees in this area." The device then sends the feedback to the server. The emotion engine analyzes the user's emotions and presents the feedback in an easy-to-understand format.

[1560] Step 11: Incorporate feedback and revise your plan

[1561] The server reconstructs the plan based on the received feedback. The revised plan is sent back to the device and displayed to the user. The emotion engine generates a response based on the user's emotions, encouraging the user to provide positive feedback.

[1562] Step 12: Execute the plan

[1563] The user reviews the final plan and approves it for execution. The device sends the execution status to the server, monitoring the progress of the plan in real time. The emotion engine displays encouraging messages at appropriate times to reduce the user's stress level.

[1564] Step 13: Monitor progress and improve

[1565] The server records the progress of the executed tasks in a database and adjusts the plan again if necessary. The user uploads additional data (e.g., information on new disease outbreaks) from their device to the server for new analysis. The emotion engine monitors the user's emotional state and provides appropriate feedback and responses.

[1566] The above is the specific flow of the program processing of this system, which enables users to achieve efficient and sustainable forest management based on data, while also increasing user satisfaction through the emotion engine.

[1567] Example 2

[1568] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1569] Conventional forest management systems lack the mechanisms for formulating and implementing efficient and accurate forest harvesting and planting plans. In particular, they lack the ability to adjust plans to take users' emotions into account, making it difficult to reduce their stress and anxiety. This poses a risk of reducing the sustainability and efficiency of forest management.

[1570] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1571] In this invention, the server includes means for receiving satellite images and images captured by unmanned aerial vehicles, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify the forest health and tree species, means for determining optimal felling and planting sites based on the identified data, means for visualizing the determined felling and planting sites on a map, means for analyzing the user's facial expressions and tone of voice in real time to recognize the user's emotional state, means for optimizing the interface in accordance with the recognized emotional state, receiving feedback from the user, and revising the plan, and means for providing the revised plan to the user and managing its execution and progress. This enables efficient and accurate forest management, and by taking the user's emotional state into consideration, it is possible to provide a user-friendly interface and reduce stress and anxiety.

[1572] "Satellite imagery" is image data taken of the Earth's surface from space.

[1573] An "unmanned aerial vehicle" is an aircraft that is operated by remote control or automatic pilot and has no personnel on board.

[1574] "Preprocessing" refers to the process of converting received data into a format that is easy to analyze and use, and specifically includes noise removal, normalization, geometric transformation, etc.

[1575] "Analysis" is the process of examining and breaking down data in detail to clarify its content and structure.

[1576] "Forest health" is an indicator of how healthy a forest is, including the growth status of trees and the presence of pests and diseases.

[1577] "Tree type" is a classification of the types of trees present in a particular forest.

[1578] "Cutting point" means [a specific location in the forest where trees should be cut].

[1579] "Planting site" means [a suitable location for planting new trees].

[1580] "Visualization" is a technique that makes data and information easier to understand by displaying them visually.

[1581] "Emotional state" refers to the user's current psychological and emotional state.

[1582] "Interface optimization" refers to adjusting the layout and functionality of an interface to improve the user experience.

[1583] "Feedback" is the process of feeding user reactions and opinions back into the system.

[1584] "Progress management" means monitoring the progress of plans and work and making adjustments or corrections as necessary.

[1585] This invention is a system for planning and executing efficient and accurate forest felling and planting plans, and by combining it with an emotion engine that recognizes user emotions, it provides a more user-friendly interface. The system is primarily composed of a server and a terminal, and is operated by the user. The server is responsible for analyzing data and formulating plans, while the terminal functions as an interface connecting the user and the server. The emotion engine recognizes user emotions in real time and is used to optimize the user experience.

[1586] Data collection and preprocessing

[1587] Users use drones or unmanned aerial vehicles to take aerial and ground photographs of forests and store the image data on their devices. The users then upload the data to the server via their devices. The server then performs preprocessing on the received image data, including noise removal, normalization, and geometric transformation. OpenCV is used for noise removal, and Scikit-Image is used for normalization and geometric transformation.

[1588] Data analysis and feature extraction

[1589] The server applies machine learning models to the preprocessed image data to extract features such as forest health, tree species, and the location of water sources, generating a risk map. The analysis is performed using TensorFlow and PyTorch, and the results are stored in a database.

[1590] Planning of felling and planting trees

[1591] The server uses the risk map to identify optimal felling and planting locations. The server takes into account factors such as the extent of diseased trees when identifying felling locations. Soil data, rainfall information, and past success stories of plantings are taken into account when selecting planting locations. The server also uses seasonal information and rainfall forecasts to determine the optimal timing for felling and planting. This is done using geographic information systems such as ArcGIS and QGIS.

[1592] Emotion engine integration

[1593] The emotion engine, powered by Microsoft Azure Cognitive Services and Affectiva, analyzes the user's facial expressions and tone of voice in real time on the device to recognize their emotional state. If the user looks anxious, the system will provide them with more detailed explanations and advice.

[1594] Plan visualization and feedback

[1595] The server sends the optimized felling and planting plans along with map data to the device. The device visualizes the plans on a map and displays them to the user. For example, the device receives the data in GeoJSON format and visualizes it in a dedicated map app. The user provides feedback on the plans, which the device then sends to the server. The emotion engine optimizes the content and presentation of the feedback based on the user's emotions.

[1596] Implementing the plan and managing progress

[1597] Once the user finalizes the plan, the server manages progress based on this plan. The user periodically reports their execution status to the server via their device. The server monitors progress based on this and readjusts the plan as necessary. The emotion engine adjusts notifications and alarms at appropriate times when the user is feeling stressed.

[1598] Specific examples

[1599] For example, when a user creates a tree planting plan, they first use a drone to take aerial photos of the forest and upload the image data from their device to the server. The server receives the data and pre-processes it. Next, it uses a machine learning model to analyze it and identify the soil quality and the health of existing trees. The optimal planting site is then selected and displayed on a map. When the user reviews the plan and enters feedback, the emotion engine analyzes the user's emotional state and optimizes the feedback content. Finally, when the user executes the plan and reports progress to the server, the emotion engine notifies them at the appropriate time and helps manage progress.

[1600] Example prompt sentence:

[1601] "Please analyze aerial photos of the forest taken by a drone, identify the best locations for planting trees, and display the plan on a map. Please also develop a system that analyzes the user's facial expressions and provides appropriate feedback."

[1602] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1603] Step 1: Data collection and upload

[1604] A user takes aerial and ground photographs in a forest area using a drone or unmanned aerial vehicle. The captured image data (input) is saved on the device. The user then uploads the image data to a server via the device. The user uses a dedicated application on the device and presses the "upload" button to send the data (output).

[1605] Step 2: Preprocessing the data

[1606] The image data received by the server (input) is first denoised using OpenCV's Gaussian filter. Next, the image data is normalised to a range of 0 to 1 to maintain data consistency. Furthermore, geometric transformations such as resizing and rotation are performed to make the data suitable for analysis (output). This results in preprocessed image data.

[1607] Step 3: Data analysis and feature extraction

[1608] The server inputs preprocessed image data (input) into a TensorFlow or PyTorch machine learning model. The model then analyzes the data (data processing) to extract features such as forest health, tree species, and water source locations. The analysis results (output) are stored in a database in JSON format.

[1609] Step 4: Planning for felling and planting

[1610] The server generates a risk map and identifies optimal felling and planting locations based on the analysis results (input). The server also considers soil data, rainfall information, and past successful planting cases to create a plan (data calculation). It also determines the optimal timing based on seasonal information and rainfall forecasts (output). This is done using a geographic information system (GIS).

[1611] Step 5: Integrating the Emotion Engine

[1612] The device's camera and microphone are activated, and the user's facial expressions and tone of voice are analyzed in real time (input). For example, Microsoft Azure Cognitive Services or Affectiva are used. Based on the analysis results (output), the device dynamically adjusts the interface and provides detailed explanations and advice to the user.

[1613] Step 6: Visualize and feedback the plan

[1614] The server sends the optimized felling and planting plan (input) along with map data to the device. The device visualizes the plan received in GeoJSON format on a map and displays it to the user (output). The user checks the plan and fills in a feedback form. The device sends the user's feedback to the server, and the emotion engine optimizes the way the feedback is presented based on the user's emotions.

[1615] Step 7: Implement the plan and track progress

[1616] The user finalizes the plan and presses the "Start Execution" button on the device (input). The server starts managing progress based on the plan and monitors it in real time (output). The user periodically reports the execution status via the device, and the server checks and adjusts the progress. The emotion engine analyzes the user's stress state and issues notifications and alarms at appropriate times.

[1617] Specific examples

[1618] For example, if a user were to create a tree planting plan, the process would be as follows: The user would take aerial photos of the forest with a drone, save them on their device, and upload them to the server. The server would use OpenCV to remove noise from the images and a TensorFlow model to identify soil quality and the health of existing trees. Using this information, the optimal planting locations would be identified and displayed on a map. The user would review the plan and provide feedback, and the emotion engine would analyze that feedback and optimize the presentation. Finally, the user would execute the plan, and the server would monitor progress in real time.

[1619] (Application example 2)

[1620] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1621] In conventional forest management systems, it is necessary to consider not only forest health and tree planting optimization, but also the psychological state of workers. In particular, to improve the efficiency of workers who are susceptible to fatigue and stress, a system that can grasp their emotional state and provide appropriate feedback is needed.

[1622] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1623] In this invention, the server includes means for receiving satellite images and images captured by unmanned aerial vehicles, means for preprocessing the received image data, means for analyzing the preprocessed image data to identify the forest health and tree species, means for determining optimal felling and planting sites based on the identified data, means for visualizing the determined felling and planting sites on a map, means for recognizing user emotions in real time and optimizing the content and presentation method of the feedback, means for receiving user feedback and revising the plan, and means for providing the revised plan to the user and managing its execution and progress. This enables a user-friendly system that takes into account the psychological state of workers while improving the efficiency and accuracy of forest management.

[1624] "Satellite imagery" refers to image data taken from an artificial satellite in orbit around the Earth.

[1625] An "unmanned aerial vehicle" is an aircraft that flies remotely or autonomously without a human on board.

[1626] "Preprocessing" refers to processes such as noise removal, normalization, and geometric transformation of image data before analysis.

[1627] "Forest health" refers to the state of the forest, indicating whether the trees and vegetation within it are growing healthily.

[1628] "Tree types" refers to the classification of the various trees present in the forest.

[1629] A "harvesting point" is a location within a forest where selected trees are recommended for felling.

[1630] A "planting site" is a location selected for planting new trees.

[1631] "Visualization" refers to the visual representation of numbers and data in an easy-to-understand way.

[1632] "User emotion" refers to the emotional state felt by the person using the system at that moment.

[1633] "Real-time recognition" means that the system processes data as soon as it receives it and provides analysis results immediately.

[1634] "Feedback" refers to the opinions and requests provided by users and any resulting adjustments to the system.

[1635] "Plan modification" means receiving feedback from users and modifying an existing plan based on this feedback.

[1636] Progress management is the process of monitoring the progress of a plan and making adjustments as needed.

[1637] The system for realizing this invention uses a server, a terminal, and, if necessary, multiple input devices. The server collects data, preprocesses it, analyzes it, creates plans, and manages progress, while the terminal links the user and the server and provides an interface. Specifically, the system is implemented as follows:

[1638] First, the user takes aerial and ground photographs of the forest using an unmanned aerial vehicle or satellite. This image data is stored on the device and the user uploads it to the server via the device. The server performs preprocessing on the received image data, such as noise removal, normalization, and geometric transformation. Libraries such as OpenCV and TensorFlow are used for image processing.

[1639] Next, the server applies machine learning models to the preprocessed image data, extracting features such as forest health, tree species, and the location of water sources. A pre-trained deep learning model (TensorFlow / Keras) is used for feature extraction, generating a risk map.

[1640] The server then uses the risk map to identify optimal felling and planting locations. The server takes into account factors such as diseased trees to identify felling locations, and soil data, rainfall information, and past successful plantings to select planting locations. Seasonal information and rainfall forecasts are also used to determine the optimal timing for felling and planting.

[1641] The server also incorporates an emotion engine that recognizes the user's emotions in real time. This emotion engine analyzes the user's facial expressions and tone of voice on the device to identify the user's emotional state. For example, if the user is feeling tired or stressed, the system will notify them at the appropriate time to take a break. EmotionRecognition is often used as an emotion recognition library.

[1642] The server then receives feedback from the user, analyzes it, and modifies the plan. The modified plan is then reflected on the map and sent to the device. The user then checks the plan on the device and reports the final execution status to the server. The server uses this information to monitor progress and make adjustments as necessary.

[1643] As a concrete example, when a user creates a tree planting plan, they first take aerial photos of the forest using an unmanned aerial vehicle and upload the image data from their device to a server. The server preprocesses this data and analyzes it using machine learning models to identify soil quality and the health of existing trees. The server then selects the optimal planting site and displays it on a map. When the user reviews the plan and enters feedback, the emotion engine analyzes the user's emotional state and optimizes the feedback content. Finally, when the user executes the plan and reports progress to the server, the emotion engine notifies them at the appropriate time and helps manage progress.

[1644] An example prompt might be, "This program is a forest management system installed in an autonomous vehicle. Write Python code that analyzes the following content and generates optimal felling and planting plans. Additionally, add a notification function to recognize the emotional state of the worker and reduce stress."

[1645] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1646] Step 1:

[1647] Users use unmanned aerial vehicles or satellites to take aerial and ground photographs of forests. These image data are stored on the device. The inputs are aerial and ground photographs, and the output is image data stored on the device. The user operates the image capture device to collect forest data over a wide area.

[1648] Step 2:

[1649] A user uploads image data to a server via a terminal. The input is the stored image data, and the output is the image data transferred to the server. The terminal uses an Internet connection to quickly transmit large amounts of image data to the server.

[1650] Step 3:

[1651] The server performs preprocessing on the received image data, such as noise removal, normalization, and geometric transformation. The input is the uploaded image data, and the output is the preprocessed image data. The OpenCV library is used to improve the image quality and make it suitable for analysis.

[1652] Step 4:

[1653] The server applies machine learning models based on the preprocessed image data to identify forest health, tree species, water source locations, etc. The input is the preprocessed image data, and the output is the identified feature data. A detailed analysis of the forest is performed using the TensorFlow / Keras library.

[1654] Step 5:

[1655] The server generates a risk map based on the identified data. The input is the identified feature data, and the output is a risk map. The server integrates the analysis results into a geographic information system (GIS) to visualize defects and dangerous areas.

[1656] Step 6:

[1657] The server identifies optimal felling and planting locations based on the risk map. The input is the risk map, and the output is optimized felling and planting locations. The server uses algorithms to generate an efficient work plan.

[1658] Step 7:

[1659] The server visualizes the optimal felling and planting locations on a map and sends it to the device. The input is the optimized location data, and the output is the visualized data on the map. The server uses a visualization tool to convert the data into a format that is easy to view on the device.

[1660] Step 8:

[1661] The user checks the plan contents and provides feedback using a device. The input is the plan data visualized on a map, and the output is the user's feedback data. The user interactively checks the plan on the device and suggests any necessary modifications.

[1662] Step 9:

[1663] The server receives user feedback and recognizes the user's emotional state in real time using an emotion engine. The inputs are feedback data and real-time emotion data, and the output is emotion recognition results. The server uses the EmotionRecognition library to understand the user's feelings.

[1664] Step 10:

[1665] The server modifies the plan based on the feedback and emotion recognition results. The inputs are user feedback and emotion recognition results, and the output is modified plan data. The server analyzes the feedback information and automatically makes necessary adjustments.

[1666] Step 11:

[1667] The revised plan is resent to the terminal and presented to the user. The input is the revised plan data, and the output is the final plan data presented to the user. The user confirms the revised plan on the terminal and starts implementing it.

[1668] Step 12:

[1669] The user executes the plan and reports the progress to the server. The input is data on the work being performed, and the output is progress report data. The user periodically sends progress information to the server using their terminal, allowing the progress of the work to be monitored in real time.

[1670] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1671] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1672] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1673] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1674] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1675] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1676] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1677] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1678] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1679] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1680] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1681] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1682] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1683] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1684] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1685] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1686] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1687] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1688] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1689] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1690] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1691] The following is further disclosed regarding the above embodiment.

[1692] (Claim 1)

[1693] means for receiving satellite images and images captured by unmanned aerial vehicles;

[1694] means for preprocessing the received image data;

[1695] means for analyzing the preprocessed image data to identify forest health and tree species;

[1696] A means for determining optimal felling and planting locations based on the identified...

Claims

1. means for receiving satellite images and images captured by unmanned aerial vehicles; means for preprocessing the received image data; means for analyzing the preprocessed image data to identify forest health and tree species; A means for determining optimal felling and planting locations based on the identified data; A means for visualizing the determined felling and planting locations on a map; A means of receiving user feedback and revising the plan; A means of providing revised plans to users and managing their execution and progress; A system including:

2. Further including measures to determine the optimal timing for tree felling and planting, taking into account environmental protection and sustainability; The system of claim 1 .

3. further including means for receiving additional data from the user and readjusting the plan; The system of claim 1 .

Images