System

Abstract

A system is provided.SOLUTION: Means for acquiring an artificial satellite image or a drone image, means for analyzing the acquired image and specifying a buried place of a mine, means for transmitting the analysis result to a terminal and visually displaying the analysis result, means for storing past buried mine information and accident information in a database, means for predicting a buried place of a mine on the basis of the stored information, means for providing a prediction result to the terminal so that a user can confirm the prediction result, means for creating a removal work plan on the basis of the buried place of the mine and the prediction result, and an automatic robot for executing the created work plan. Means for monitoring in real time the working status of the automated robot.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] There are currently approximately 70 to 100 million landmines buried around the world, and their removal is extremely dangerous, labor-intensive, and costly. Furthermore, the removal process results in thousands of casualties each year, making it a serious humanitarian issue. The problem this invention aims to solve is to use artificial intelligence (AI) technology to safely and efficiently remove landmines, thereby saving lives and improving the efficiency of the work. [Means for solving the problem]

[0005] The present invention solves the aforementioned problems with a system that includes the following means: First, satellite and drone images are acquired and analyzed to identify buried mine locations. The analysis results are sent to a device and visually displayed, allowing users to easily grasp the location of mines. Furthermore, information on past mine placements and accidents is stored in a database, and mine locations are predicted based on this information. The prediction results are also provided to the device so that users can check them. Next, a mine removal plan is created based on the mine placement locations and the prediction results. Then, an automated robot is used to execute the plan, performing the actual removal work. The automated robot uses GPS information to move automatically and remove the mines. Furthermore, the automated robot's work status is monitored in real time, allowing accurate understanding of the work progress. This series of means significantly improves the safety and efficiency of mine removal work.

[0006] "Satellite imagery" refers to images of the Earth's surface taken from a satellite.

[0007] "Drone imagery" refers to images of the earth's surface taken from an unmanned aerial vehicle (drone).

[0008] "Image analysis" refers to the process of analyzing image data using computer algorithms to extract specific information.

[0009] "Mine site" refers to the specific location where a mine is planted and buried.

[0010] "Terminal" refers to a computer or digital device used by a user.

[0011] "Visually displaying" refers to visually showing analysis results and information on images, graphs, or maps.

[0012] A "database" refers to a system that systematically manages and stores large amounts of data.

[0013] "Past mine placement information" refers to information on historically recorded mine placements.

[0014] "Accident information" refers to records of accidents caused by landmines and information on damage.

[0015] "Prediction" refers to estimating future events or situations based on past data and current information.

[0016] "Mine clearance plan" refers to detailed plans and procedures for efficiently carrying out mine clearance operations.

[0017] "Autonomous robot" refers to a mechanical device that operates automatically according to programmed instructions to remove landmines.

[0018] "GPS Information" refers to location information obtained using the Global Positioning System.

[0019] "Monitoring" refers to the continuous observation of a particular process or condition, as well as the collection and analysis of data.

[0020] "Real-time" refers to data processing and information being provided at the exact moment an event occurs. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This invention relates to a system for safely and efficiently removing landmines using artificial intelligence technology. Specific embodiments for carrying out the invention will be described below.

[0043] Image Recognition

[0044] First, the server receives satellite or drone images, which are then sent to the server via a network.

[0045] The server then uses an image analysis algorithm to analyze the received image data. The algorithm identifies specific patterns and identifies the location of buried mines. The analysis results are output as coordinate data indicating the location of buried mines.

[0046] The coordinate data from the analysis is sent to the device, which receives it and visually displays it. Specifically, it marks specific coordinates on a map to show the user where the mines are buried.

[0047] Information gathering and prediction

[0048] The server collects and stores in a database information on past mine placements and accidents, which is obtained from historical databases and local reports.

[0049] Based on the stored information, the server makes predictions about mine locations. An AI model analyzes this data and predicts unknown mine locations. The predictions are then provided to the device and displayed as a dashboard for the user to access.

[0050] Removal work planning and execution

[0051] Based on the identified mine locations and prediction results, the server creates a removal operation plan, which includes efficient work procedures and time schedules.

[0052] The work plan is presented to the terminal and reviewed and approved by the user. Once approved, the plan is sent to the automated robot.

[0053] The automated robot will then begin clearing operations in the designated area based on the received work plan, and will use GPS information to navigate automatically and safely clear specific mines.

[0054] Finally, the server monitors the work of the automated robot in real time, tracking its progress and updating the data to ensure that the work is going smoothly.

[0055] Specific examples

[0056] For example, when carrying out mine clearance work in a certain area, the following specific steps are taken:

[0057] 1. The server receives the image data sent from the drone.

[0058] 2. The server performs image analysis and finds a mine at a specific coordinate, for example, (10.1234, 20.5678).

[0059] 3. The device displays the analysis results and provides coordinate information to the user.

[0060] 4. The server collects and organizes past mine placement data and accident information, and predicts where mines will be placed.

[0061] 5. The forecast results are displayed as a dashboard, providing information, for example, about the newly predicted coordinates (10.2345, 20.6789).

[0062] 6. The server will create a removal plan based on the identified mine locations and the prediction results.

[0063] 7. The terminal presents the work plan to the user and obtains confirmation.

[0064] 8. Autonomous robots will follow a work plan and head to the site to remove the mines.

[0065] 9. The server monitors the work in real time and tracks the progress.

[0066] This series of processes ensures that mine clearance work is carried out safely and efficiently, and the system protects many people from the dangers of landmines and minimizes damage.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] The server receives image data from a satellite or drone. The image data is sent over the network, and the server saves the data in storage. Specifically, it calls the receive_image_data() function to retrieve the image. This data is saved in a variable called "image_data," for example.

[0070] Step 2:

[0071] The server analyzes the received image data. The acquired image data is input into the AI ​​model to identify the location of buried landmines. The AI ​​model contains a specific image recognition algorithm and executes the analyze_image(image_data) function, which identifies the coordinates of the buried landmines.

[0072] Step 3:

[0073] The server sends the analysis results to the device. It generates coordinate data for the identified mines and sends it to the device. It sends the analysis results using the send_results(analysis results, device ID) function.

[0074] Step 4:

[0075] The device receives the analysis results. The device visually displays the received coordinate data. For example, it marks specific coordinates on a map to show the user where the mines are buried. This is done using the display_results(analysis results) function.

[0076] Step 5:

[0077] The server collects information on past mine placements and accidents and stores it in a database. It retrieves information from historical databases and various reports and stores it in the database using the collect_data(data source) function. For example, it stores it in a variable called "historical_data."

[0078] Step 6:

[0079] Organize and consolidate the data collected by the server. Normalize and clean the data to make it analyzable by the AI ​​model. Use the normalize_data(collected_data) function.

[0080] Step 7:

[0081] The server uses an AI model to predict mine locations. The organized data is input into the AI ​​model to predict unknown mine locations. The predict_mine_locations(model, organized data) function is executed to obtain the predicted results.

[0082] Step 8:

[0083] The server provides the prediction results to the terminal. The prediction results are generated so that they can be displayed in dashboard format and sent to the terminal. The dashboard is generated using the generate_dashboard(prediction result) function.

[0084] Step 9:

[0085] The device displays the forecast results to the user, allowing the user to see the results in a visual format (e.g., a graph or map). This is done using the display_dashboard(forecastresult) function.

[0086] Step 10:

[0087] The server creates a mine removal plan. It creates an efficient removal plan based on the prediction results and the identified mine locations. It uses the create_removal_plan(identified location, prediction result) function.

[0088] Step 11:

[0089] The terminal presents the work plan to the user. The details of the work plan are displayed and the user confirms and approves it. The display_plan(plan) function is used.

[0090] Step 12:

[0091] The user checks and approves the work plan. The plan presented on the terminal is checked and determined to be feasible. After approval, the server is notified. The approve_plan() function is used.

[0092] Step 13:

[0093] The automated robot begins work based on the work plan received from the server. It follows GPS information to move to the designated mine site and carry out the removal work. The execute_removal_task(robot, plan) function is used.

[0094] Step 14:

[0095] The server monitors the work status of the automated robot in real time. It receives feedback data from the robot and monitors its progress. It uses the monitor_progress(robot) function.

[0096] These steps allow the system to safely and efficiently clear mines.

[0097] Example 1

[0098] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0099] Landmines are buried widely around the world as remnants of war and conflict, and their removal is extremely dangerous and time-consuming, making it a major social problem in many regions. Conventional mine removal methods rely on manual labor, which entails significant risks and costs. Furthermore, it is often difficult to identify or predict where mines will be buried, making the work inefficient. This invention aims to solve these problems by using artificial intelligence technology to remove landmines safely and efficiently.

[0100] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0101] In this invention, the server includes means for acquiring satellite images or unmanned aerial vehicle images, means for analyzing the acquired images to identify buried mine locations, means for transmitting the analysis results to a terminal and visually displaying them, means for storing information on past buried mines and accident information in a database, means for predicting buried mine locations based on the stored information, means for providing the prediction results to the terminal so that the user can check them, means for formulating a removal work plan based on the buried mine locations and the prediction results, an automated work device for executing the formulated work plan, and means for monitoring the work status of the automated work device in real time. This makes it possible to automate the process from identifying and predicting the location of mines to the removal work as a series of steps, thereby significantly improving the safety and efficiency of work.

[0102] "Satellite imagery" refers to image data of the Earth's surface taken from an artificial satellite in Earth's orbit.

[0103] "Unmanned aerial vehicle imagery" refers to image data of the earth's surface taken from an unmanned aerial vehicle such as a drone.

[0104] "Means of acquisition" refers to the equipment, software, and protocols necessary to receive image data transmitted from satellites and unmanned aerial vehicles.

[0105] "Means of analysis" refers to the algorithms, software, and hardware used to identify buried mine locations using acquired image data.

[0106] "Visual display means" refers to a device or application that provides the analysis results to the user in a visual format such as a map or graph.

[0107] "Means of storing information in a database" refers to the system or structure for managing and storing information on past mine burials and accidents.

[0108] "Predictive tools" refers to software and computational models that use artificial intelligence algorithms to predict new mine locations based on stored data.

[0109] "Means for providing and enabling users to check" refers to an interface or application that provides prediction results in a form that users can easily access and visually check.

[0110] "Planning tools for clearance operations" refers to algorithms and software that plan efficient and safe clearance procedures based on identified mine locations and predicted outcomes.

[0111] "Automated work equipment" refers to robots and mechanical devices that automatically remove mines according to a work plan.

[0112] "Monitoring means" means systems or software that monitor the progress or status of automated work equipment in real time and intervene or adjust as necessary.

[0113] This invention relates to a system for safely and efficiently removing landmines by utilizing artificial intelligence technology. Specific embodiments for carrying out this invention will be described below.

[0114] Image Recognition

[0115] First, the server acquires satellite and drone imagery, which includes, for example, downloading images uploaded to Amazon S3 using the AWS SDK.

[0116] The server then analyzes the acquired image data. Using Python's OpenCV and TensorFlow, it uses an object detection algorithm (such as YOLO) to extract the characteristics of the mines and identify their locations. Once the analysis is complete, it generates the results as coordinate data.

[0117] The server sends the analysis results to the device in JSON format, using WebSocket or REST API for communication. Specifically, the JSON data contains the coordinate information of the analyzed mines.

[0118] The device visually displays the received analysis results and uses the JavaScript Google Maps API to mark the coordinates of detected mines on a map and notify the user, allowing the user to intuitively confirm the location of specific mines.

[0119] Information gathering and prediction

[0120] The server then collects information on past mine placements and accidents and stores it in a database, which is managed using an RDBMS such as MySQL or PostgreSQL, and retrieves information from historical databases and local reports.

[0121] The server predicts mine locations based on the stored information. To do this, it uses machine learning libraries (such as Scikit-learn or TensorFlow) to build a predictive model and analyze past data to estimate unknown mine locations.

[0122] The predicted results are sent from the server to the device and displayed on a dashboard for the user to review. The dashboard, built using React and Vue.js, provides coordinate information for newly predicted mines, allowing users to easily view them.

[0123] Removal work planning and execution

[0124] Based on the prediction results and the identified mine locations, the server creates a removal plan, which includes efficient and safe work procedures, the equipment to be used, and a time schedule.

[0125] The terminal presents the work plan to the user. The user checks the presented plan and presses the approval button if there are no problems. The user's approval is recorded as a log on the server.

[0126] The approved work plan is then sent from the server to the automated work device, using MQTT or other real-time messaging protocols.

[0127] Based on the received plan, the automated device will begin mine clearance work in the designated area. It will use GPS to locate specific mines and move automatically to carry out the clearance work. The work status will be reported to the server in real time.

[0128] The server monitors the operation status of the automated work equipment and tracks its progress. The acquired data is updated to a database and displayed on a dashboard, allowing users to see the progress of the work in real time.

[0129] Specific examples

[0130] For example, if a mine clearance operation is envisaged in a certain area, the following steps would be taken:

[0131] 1. The server receives the image data sent from the drone.

[0132] 2. The server performs image analysis and detects a mine at a specific coordinate, for example, (10.1234, 20.5678).

[0133] 3. The device displays the analysis results and provides coordinate information to the user.

[0134] 4. The server collects and organizes past mine placement data and accident information, and predicts where mines will be placed.

[0135] 5. The forecast results are displayed as a dashboard, providing information about the newly predicted coordinates (10.2345, 20.6789).

[0136] 6. The server creates a removal plan based on the identified mine locations and the prediction results.

[0137] 7. The terminal presents the work plan to the user and obtains confirmation.

[0138] 8. Automated work equipment will head to the site according to the work plan and remove the mines.

[0139] 9. The server monitors the work in real time and tracks the progress.

[0140] Prompt Sentence Examples

[0141] Example prompts to input to a generative AI model:

[0142] "Please analyze satellite images to identify buried mine sites."

[0143] "Build a predictive model for mine placement using historical data."

[0144] "Please draw up a mine clearance operation plan for the specified coordinates."

[0145] This series of processes ensures that mine clearance work is carried out safely and efficiently, and that many people are protected from the dangers of landmines.

[0146] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0147] Program processing flow

[0148] Step 1:

[0149] The server receives image data from satellites and unmanned aerial vehicles. Specifically, it downloads the image data using HTTP or FTP protocols. The input is the image file, which becomes the data for processing in the next step.

[0150] Step 2:

[0151] The server analyzes the received image data and applies an object detection algorithm (such as YOLO) using Python's OpenCV or TensorFlow. The input for this analysis is the acquired image file, and the output is coordinate data indicating the location of buried mines. Specifically, the server extracts image features and identifies the parts that can be identified as mines using a trained model.

[0152] Step 3:

[0153] The server sends the analysis results, which identify the buried mine locations, to the device. Coordinate data is sent in JSON format using WebSocket or REST API. The input is the coordinate data from the analysis results, and the output is the JSON data transferred to the device. For example, coordinates are sent in the format {"latitude": 10.1234, "longitude": 20.5678}.

[0154] Step 4:

[0155] The device receives the analysis results sent from the server and displays them visually. It uses the Google Maps API in JavaScript to mark the identified coordinates on a map. The input is coordinate data in JSON format, and the output is a map that is displayed to the user. The device draws icons on the map to indicate the location of mines, allowing the user to intuitively identify them.

[0156] Step 5:

[0157] The server collects information on past mine placements and accidents and stores it in a database. Sources of information include historical databases and local reports, and the data is managed using MySQL or PostgreSQL. The input is data from each source, and the output is information stored in the database. Specifically, the server retrieves information through an API, standardizes it, and inserts it into the database.

[0158] Step 6:

[0159] The server predicts mine locations based on the stored information. It uses a machine learning library (such as Scikit-learn or TensorFlow) to learn from past data and build a predictive model. The input is past data stored in the database, and the output is the coordinates of predicted mine locations. The server uses the trained model to analyze the input data and predict unknown mine locations.

[0160] Step 7:

[0161] The server sends the prediction results to the device, where the user can view them on a dashboard. The prediction results are sent to the device in JSON format. The input is coordinate data from the prediction model, and the output is coordinate information displayed on the dashboard. The device uses front-end libraries such as React and Vue.js to display the information in a visually easy-to-understand format.

[0162] Step 8:

[0163] The server creates a removal operation plan based on the identified mine locations and prediction results. The input is mine coordinate data and prediction data, and the output is a detailed operation plan. The server calculates efficient and safe operation procedures and generates a timeline for each step and a list of required equipment.

[0164] Step 9:

[0165] The terminal presents the work plan to the user, who then reviews and approves the plan. The input is the work plan sent from the server, and the output is the user's approval data. The terminal displays the work plan to the user as an interactive guide and provides an approval button.

[0166] Step 10:

[0167] The server sends the approved work plan to the automated work device. It transmits data in real time using the MQTT protocol. The input is the work plan approved by the user, and the output is the data sent to the automated work device. The server starts work according to the instructions.

[0168] Step 11:

[0169] The automated work device carries out mine removal work based on the received work plan. It uses GPS to move to the designated area and remove the mines. The input is the work plan data, and the output is a report data on the completion of the removal work. The automated work device periodically sends its progress to the server.

[0170] Step 12:

[0171] The server monitors the work status of the automated work equipment in real time. The server receives the transmitted progress data and tracks the work progress. The input is feedback data from the automated work equipment, and the output is updated progress data. The server visually displays the progress to the user and issues warnings if any problems occur.

[0172] (Application example 1)

[0173] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0174] Conventional autonomous vehicle systems lack sufficient means to detect obstacles and accidents on the road in real time and provide safe detour routes. This has resulted in increased operational interruptions and dangers due to accidents and obstacles, resulting in problems with operational efficiency and safety. In addition, existing mine clearance systems are limited to mine detection and clearance and lack the ability to respond to dynamically changing situations.

[0175] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0176] In this invention, the server includes means for acquiring satellite or drone images, means for analyzing the acquired images to identify buried mine locations, means for transmitting the analysis results to a terminal and visually displaying them, means for storing information on past mine placements and accident information in a database, means for predicting mine locations based on the stored information, means for providing the prediction results to the terminal so that a user can check them, means for formulating a mine removal operation plan based on the buried mine locations and the prediction results, an automated machine for executing the operation plan, means for monitoring the operation status of the automated machine in real time, and means for detecting hazardous objects while the automated machine is operating and providing an alternative route. This enables automated vehicles to detect hazardous objects on roads in real time and provide safe detour routes, thereby improving safety and operating efficiency.

[0177] "Satellite imagery" refers to image data obtained by satellites photographing the Earth's surface.

[0178] "Drone images" are image data taken from the air by a drone, a small unmanned aerial vehicle.

[0179] "Analysis" is the process of applying specific algorithms to acquired image data to extract information and identify objects of interest.

[0180] "Mine site" refers to the location where a landmine is buried underground.

[0181] A "terminal" is an electronic device that a user uses to receive and visually view information.

[0182] A "database" is a computer system for organizing and storing data, allowing for efficient searching and updating of data.

[0183] "Prediction" refers to predicting future situations or events based on past data and current information.

[0184] A "clearance operation plan" is a plan that outlines the methods and procedures for safely and efficiently removing buried mine sites.

[0185] An "automated machine" refers to a robot or mechanical device that performs work automatically according to programmed instructions.

[0186] "Monitoring" refers to watching the progress of a system or task in real time and making adjustments as needed.

[0187] "Hazardous material" means any substance or condition that may cause injury or danger during operation or work.

[0188] "Alternate Route" means a proposed alternative route to avoid an accident or hazard.

[0189] MODE FOR CARRYING OUT THE INVENTION

[0190] The present invention provides a system for an autonomous vehicle to detect dangerous objects on the road and provide a safe detour route. The system comprises the following means.

[0191] First, the server acquires satellite or drone images, which are then transmitted in real time and aggregated on the server, where they are analyzed using image analysis algorithms such as OpenCV and Keras, utilizing high-performance computing resources.

[0192] As a result of the analysis, the locations of buried mines and other hazards are identified. This identified information is sent to the device and visually displayed to the user. For example, the coordinate information indicated by the analysis results is marked on a map to inform the user.

[0193] The server also stores information on past mine placements and accidents in a database. This information is used to predict mines and other hazards. An AI model analyzes this data and generates predictions. These predictions are also displayed on the device as a dashboard for users to review.

[0194] Based on the location of buried mines and the prediction results, the server creates a removal work plan. This work plan includes efficient work procedures and time schedules. The created work plan is displayed on the terminal and the user confirms and approves it. The approved plan is then sent to the automated machine.

[0195] The automated machines begin clearance operations in designated areas based on a work plan. They use GPS information to navigate automatically and safely remove specific mines and hazardous materials. Additionally, the automated machines have the ability to detect new hazards while in operation and can provide alternative routes based on detected hazards.

[0196] The server also monitors the work status of the automated machines in real time, allowing it to track the progress of the work and update the data to ensure that the work is progressing smoothly.

[0197] Specific examples

[0198] For example, when clearing landmines in an area, drones can take pictures of accidents on the road and send the images to a server, which analyzes the images and detects that an accident has occurred. This information is then sent to the autonomous vehicle's system, which then provides a safe detour route.

[0199] Prompt Sentence Examples

[0200] "As an autonomous vehicle travels from Tokyo to Los Angeles, a drone detects an accident on the road. Use the image analysis system to confirm the existence of the accident, use the Google Maps API to obtain a safe detour route, and provide that information to the autonomous vehicle."

[0201] With the introduction of this system, autonomous vehicles will be able to monitor road conditions in real time and quickly detect hazards and accidents, improving safety and operational efficiency.

[0202] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0203] Step 1:

[0204] The server receives image data transmitted from a satellite or drone. As input, the image data from the satellite or drone is passed to the server. As output, the raw data is obtained and stored in the server.

[0205] Step 2:

[0206] The server analyzes the image data it receives. It uses OpenCV and Keras as its analysis algorithms to identify landmines and hazardous objects in the image. The image data stored on the server is used as input. The output is the coordinate data of the identified landmines and hazardous objects.

[0207] Step 3:

[0208] The analysis results are sent to the terminal and displayed for the user to visually confirm. As input, the identified coordinate data is sent from the server to the terminal. As output, the coordinate information is displayed marked on a map on the terminal screen.

[0209] Step 4:

[0210] The server stores the past mine laying information and accident information in the database. As input, the past mine laying data and accident information are taken into the database. As output, the information storage is completed.

[0211] Step 5:

[0212] The server analyzes the stored information and predicts mine locations and hazardous materials. It uses a generative AI model to make predictions based on past data. The input is the stored data in the database. The output is the predicted coordinate data.

[0213] Step 6:

[0214] The prediction results are provided to the terminal and displayed for the user to check. As input, the predicted coordinate data is sent to the terminal. As output, the prediction information is displayed on the dashboard.

[0215] Step 7:

[0216] The server creates a removal operation plan based on the location of the mines and the prediction results. The input is the identified and predicted coordinate data. The output is a removal operation plan that includes the work procedure and time schedule.

[0217] Step 8:

[0218] The work plan is presented to the terminal and the user confirms and approves it. The work plan is sent to the terminal as input. The user's confirmation and approval is sent from the terminal to the server as output.

[0219] Step 9:

[0220] The automated machine starts the removal work in the designated area based on the received work plan. It uses GPS information to move automatically and safely remove the mines. The work plan and GPS information are used as inputs. The progress of the removal work is sent from the work site to the server as output.

[0221] Step 10:

[0222] The server monitors the work status of the automated machine in real time and tracks the progress. As input, work status data sent from the work site is taken into the server. As output, the progress status is updated in real time and displayed on the monitoring screen.

[0223] Step 11:

[0224] The automated machine detects new hazards while in operation and provides an alternative route. As input, sensor data acquired during operation is used. As output, an alternative route to avoid the hazard is provided to the automated machine.

[0225] This allows autonomous vehicle systems to monitor road conditions in real time, clear mines and provide safe detour routes.

[0226] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0227] This invention is a system for safely and efficiently removing landmines by using artificial intelligence technology and an emotion engine. Specific embodiments for carrying out the invention will be described below.

[0228] Image Recognition

[0229] First, the server receives image data from satellites or drones, and these images are sent to the server via a network and stored appropriately.

[0230] The server then uses the received image data to apply image analysis algorithms, using AI models to identify specific patterns and pinpoint the location of buried mines, generating coordinate data for the mines.

[0231] The analysis results are sent to the device, which receives this data and visually displays it, specifically marking specific coordinates on a map to show the user the location of the mines.

[0232] Information gathering and prediction

[0233] The server collects and stores information about past mine placements and accidents in a database, which is obtained from historical databases and local reports.

[0234] The server then organizes and consolidates the collected data, normalizing and cleaning it to make it suitable for analysis by AI models.

[0235] The server uses an AI model to predict where landmines are buried. The organized data is passed to the AI ​​model, which then predicts unknown landmine locations. The prediction results are generated in a dashboard format and provided to the device.

[0236] The device visually displays the prediction results to the user, specifically by showing the predicted locations using graphs or maps so that the user can easily check them.

[0237] Removal work planning and execution

[0238] The server then creates a removal plan based on the identified mine locations and the prediction results, generating a plan that includes efficient work procedures and a time schedule.

[0239] The work plan is presented on the terminal and an interface is provided for the user to review and approve. Once the user reviews and approves the plan, it is sent to the automated robot.

[0240] The automated robot will begin work in the designated area based on the received work plan, and will use GPS information to navigate automatically and remove mines.

[0241] The server monitors the work status of the automated robot in real time, constantly updating progress information to ensure that the work is progressing smoothly.

[0242] Emotion engine integration

[0243] Furthermore, the system is integrated with an emotion engine that recognizes the user's emotions. The server receives and analyzes the user's facial expression and voice data.

[0244] The server uses an emotion analysis algorithm to determine the user's emotional state, for example, if the user is feeling anxious, that emotional state is detected.

[0245] The device dynamically adjusts the interface based on the user's emotional state, for example simplifying the interface to present information more clearly when the user is feeling anxious.

[0246] Specific examples

[0247] As a concrete example, the procedure for carrying out mine clearance work in a certain area will be explained.

[0248] 1. The server receives the image data sent from the drone.

[0249] 2. The server performs image analysis and detects a mine at coordinates (10.1234, 20.5678).

[0250] 3. The device displays the analysis results on a map for the user.

[0251] 4. The server collects and organizes past mine data and predicts where mines are buried.

[0252] 5. The server generates the prediction results and provides them to the device as a dashboard.

[0253] 6. The device displays the prediction results for the user to confirm.

[0254] 7. The server creates a removal plan and displays it on the terminal.

[0255] 8. The user reviews and approves the work plan.

[0256] 9. The automated robot moves to the designated location and removes the mines.

[0257] 10. The server monitors the robot's work status in real time.

[0258] 11. The server analyzes the user's emotions and dynamically adjusts the interface.

[0259] This series of processes not only ensures safe and efficient mine clearance work, but also reduces the psychological burden on the user. This system protects many people from the dangers of landmines while also improving the efficiency of work.

[0260] The processing flow will be explained below.

[0261] Step 1:

[0262] The server receives image data from a satellite or drone. The image data is sent over the network, and the server saves the data in storage. Specifically, it calls the receive_image_data() function to retrieve the image. This data is saved in a variable called "image_data," for example.

[0263] Step 2:

[0264] The server analyzes the received image data. The acquired image data is input into the AI ​​model to identify the location of buried landmines. The AI ​​model contains a specific image recognition algorithm and executes the analyze_image(image_data) function, which identifies the coordinates of the buried landmines.

[0265] Step 3:

[0266] The server sends the analysis results to the device. It generates coordinate data for the identified mines and sends it to the device. It sends the analysis results using the send_results(analysis results, device ID) function.

[0267] Step 4:

[0268] The device receives the analysis results. The device visually displays the received coordinate data. For example, the display_results(analysis results) function is used to mark specific coordinates on a map to show the user where the mines are located.

[0269] Step 5:

[0270] The server collects information on past mine placements and accidents and stores it in a database. It retrieves information from historical databases and various reports and stores it in the database using the collect_data(data source) function. For example, it is stored in a variable called "historical_data."

[0271] Step 6:

[0272] Organize and consolidate the data collected by the server. Normalize and clean the data to make it analyzable by the AI ​​model. Use the normalize_data(collected_data) function.

[0273] Step 7:

[0274] The server uses an AI model to predict mine locations. The organized data is input into the AI ​​model to predict unknown mine locations. The predict_mine_locations(model, organized data) function is executed to obtain the predicted results.

[0275] Step 8:

[0276] The server provides the prediction results to the terminal. The prediction results are generated so that they can be displayed in dashboard format and sent to the terminal. The dashboard is generated using the generate_dashboard(prediction result) function.

[0277] Step 9:

[0278] The device displays the forecast results to the user, allowing the user to see the results in a visual format (e.g., a graph or map). This is done using the display_dashboard(forecastresult) function.

[0279] Step 10:

[0280] The server creates a mine removal plan. It creates an efficient removal plan based on the prediction results and the identified mine locations. It uses the create_removal_plan(identified location, prediction result) function.

[0281] Step 11:

[0282] The terminal presents the work plan to the user. The details of the work plan are displayed and the user confirms and approves it. The display_plan(plan) function is used.

[0283] Step 12:

[0284] The user checks and approves the work plan. The plan presented on the terminal is checked and determined to be feasible. After approval, the server is notified. The approve_plan() function is used.

[0285] Step 13:

[0286] The automated robot begins work based on the work plan received from the server. It follows GPS information to move to the designated mine site and carry out the removal work. The execute_removal_task(robot, plan) function is used.

[0287] Step 14:

[0288] The server monitors the work status of the automated robot in real time. It receives feedback data from the robot and monitors its progress. It uses the monitor_progress(robot) function.

[0289] Step 15:

[0290] The server receives the user's facial expression and voice data and performs emotion analysis. It analyzes the user's emotional state in real time and identifies the state using the emotion analysis engine. It uses the analyze_emotion(facial expression data, voice data) function.

[0291] Step 16:

[0292] The device dynamically adjusts the interface based on the user's emotional state. For example, if the user is feeling anxious, the interface can be simplified to make the information easier for the user to understand. This is done using the adjust_interface(emotional_state) function.

[0293] These steps allow the system to safely and efficiently remove mines and further optimize the user experience based on the user's emotional state. For example, in step 7 above, the system predicts that a new mine has been buried at coordinates (10.1234, 20.5678), and the removal process proceeds based on that prediction. The interface is simplified when the user feels anxious. In this way, the overall system further enhances safety and efficiency, and also provides psychological support.

[0294] Example 2

[0295] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0296] Conventional mine clearance systems have low analytical accuracy for accurately identifying the location of mines, limiting the efficiency of clearance work. Furthermore, they lack a method for reducing the psychological burden on users. Therefore, there is a need for a system that can clear mines quickly and safely while also reducing the psychological burden on users.

[0297] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for acquiring satellite images or drone images; means for analyzing the acquired images to identify buried mine locations; means for transmitting the analysis results to the terminal and visually displaying them; means for storing past mine placement information and accident information in a database; means for predicting mine locations based on the stored information; means for providing the prediction results to the terminal so that the user can confirm them; means for formulating a removal work plan based on the buried mine locations and the prediction results; an automatic robot for executing the formulated work plan; means for monitoring the work status of the automatic robot in real time; emotion engine means for receiving and analyzing facial expression data and voice data of the user; and means for dynamically adjusting the terminal interface based on the user's emotional state. This enables mine identification, prediction, and removal work to be performed quickly and safely while reducing the psychological burden on the user.

[0298] "Satellite imagery" refers to image data taken of a specific area on Earth by an artificial satellite.

[0299] "Drone images" are aerial image data taken by a camera mounted on a drone.

[0300] "Image analysis" is the technique of processing acquired image data to identify specific objects.

[0301] "Mine burial location" is information about the location where a mine is buried in the ground.

[0302] A "terminal" is a computing device through which a user can view information.

[0303] A "database" is a system that systematically stores and manages data.

[0304] "Prediction results" are estimated information derived using artificial intelligence algorithms.

[0305] A "clearance operation plan" is a specific procedure and schedule for efficiently clearing landmines.

[0306] An "automatic robot" is a mechanical device that operates autonomously based on set instructions to remove landmines.

[0307] "Real-time monitoring" is a technology that instantly monitors current progress.

[0308] An "emotion engine" is a technology that analyzes a user's facial expression data and voice data to identify their emotional state.

[0309] An "interface" is a screen or operating method that serves as a point of contact for the user and the system to exchange information with each other.

[0310] An "artificial intelligence algorithm" is a computational method that analyzes large amounts of data to derive patterns and predictions.

[0311] "Visually displaying" means displaying information on a screen in a format that is easy for the user to understand.

[0312] "Facial expression data" is information that captures the user's facial movements and expressions.

[0313] "Voice data" is information that records the user's utterances and voice characteristics.

[0314] "Dynamic adjustment" means changing the settings flexibly according to the situation at hand.

[0315] MODE FOR CARRYING OUT THE INVENTION

[0316] This invention is a system that combines artificial intelligence technology and an emotion engine to safely and efficiently remove landmines. Specific embodiments for carrying out this invention will be described below.

[0317] Image Recognition

[0318] First, the server receives image data sent from satellites or drones. This image data is then sent to the server via a network and stored in an appropriate storage. This process can use cloud storage such as Amazon S3.

[0319] The server then uses the received image data to apply image analysis algorithms, specifically using machine learning libraries such as TensorFlow to identify the locations of buried mines using a Convolutional Neural Network (CNN) model, and generates coordinate data for the areas where the mines are located.

[0320] The analysis results are sent to the device and displayed visually, and the device uses the Google Maps API or similar to mark specific coordinates on a map to show the user the location of the mines.

[0321] Information gathering and prediction

[0322] The server collects and stores information on past mine placements and accidents in a database. This information is obtained from historical databases and local reports. The database can be a database system such as PostgreSQL.

[0323] The server then normalizes and cleans the collected and stored data, converting it into a format suitable for AI analysis. This process uses data processing libraries such as Pandas.

[0324] The server inputs the organized data into an AI model to predict the location of buried mines, using a time-series prediction model such as a Long Short-Term Memory (LSTM) network. The prediction results are generated in the form of a dashboard and sent to the device.

[0325] The terminal visualizes the prediction results provided by the server, allowing users to easily check them. By using concrete diagrams and maps, users can intuitively understand the prediction results.

[0326] Removal work planning and execution

[0327] The server then creates a removal plan based on the identified mine locations and the predicted results. This plan includes efficient work procedures and time schedules. An optimization algorithm can be used to generate the most efficient removal plan.

[0328] The work plan is presented on the terminal and an interface is provided for the user to review and approve, and once the user approves the plan, it is sent to the automated robot.

[0329] The autonomous robot uses a GPS device to automatically navigate to the designated area and begin the clearance process. Specifically, the robot detects landmines and uses explosive ordnance disposal equipment to safely remove them.

[0330] The server monitors the work status of the automated robot in real time, and progress information is updated and displayed on the terminal.

[0331] Emotion engine integration

[0332] Furthermore, this system is integrated with an emotion engine that recognizes the user's emotions. The server receives and analyzes the user's facial expression and voice data. The analysis uses a voice recognition library and an expression analysis library.

[0333] The server uses an emotion analysis algorithm to identify the user's emotional state, for example, if the user is feeling anxious, that emotion is detected.

[0334] The device dynamically adjusts its interface based on the user's emotional state: when the user is feeling anxious, it simplifies the interface and provides more understandable information.

[0335] Examples of concrete examples and prompts

[0336] As a concrete example, the procedure for carrying out mine clearance work in a certain area is shown below.

[0337] 1. The server receives high-resolution image data sent from the drone and begins processing.

[0338] 2. The server uses a Convolutional Neural Network (CNN) to analyze the image and detect a landmine at coordinates (10.1234, 20.5678).

[0339] 3. The device uses the Google Maps API to mark the analysis results on a map in real time and display them to the user.

[0340] 4. The server collects and organizes landmine data from the past 10 years from the PostgreSQL database and inputs it into the AI ​​model.

[0341] 5. The server uses the LSTM model to generate a dashboard of predicted mine locations and send it to the device.

[0342] 6. The device displays the dashboard provided by the server, and the user checks the prediction results.

[0343] 7. The server uses an algorithm to create an optimal removal plan and displays it on the device.

[0344] 8. The user reviews the plan and approves it through the device interface.

[0345] 9. An automated robot uses a GPS device to navigate to a designated location and uses a robotic arm to dig up the mines.

[0346] 10. The server monitors the robot's operation in real time and displays the progress on the terminal.

[0347] 11. The server receives the user's facial expression data from the camera and detects feelings of anxiety.

[0348] 12. The device simplifies the interface and displays a reassuring message to the user.

[0349] These systems allow mine clearance work to be carried out quickly and safely, and also reduce the psychological burden on users.

[0350] Example prompt sentence:

[0351] "Please generate explanatory text for a mine removal system that detects buried mine locations from image data, makes predictions based on past mine data, and allows users to proceed with confirmation and removal work with confidence."

[0352] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0353] Step 1: Receiving image data

[0354] The server receives high-resolution image data transmitted from a satellite or drone. The input is the image data transmitted from the satellite or drone, and the output is the image data stored in the server's storage. Specifically, this can be done using cloud storage such as Amazon S3.

[0355] Step 2: Image analysis

[0356] The server uses the image data received by the server to identify buried landmine locations by applying a Convolutional Neural Network (CNN) model using machine learning libraries such as TensorFlow. The input is the image data stored in storage, and the output is the coordinate data of the area where the landmines are located. Specifically, the CNN model extracts image features and identifies the patterns of landmines.

[0357] Step 3: Viewing the analysis results

[0358] The server sends the results of the image analysis to the device. The device visually displays the analysis results using Google Maps API or similar. The input is the coordinate data sent from the server, and the output is the location of the mine marked on the map. Specifically, it marks specific coordinates on the map to show the user the location of the mine.

[0359] Step 4: Gather information

[0360] The server collects information on past mine placements and accidents from historical databases and local reports, and stores this information in a database. The input is mine information obtained from external data sources, and the output is information stored in the database. Specific operations use a database system such as PostgreSQL.

[0361] Step 5: Organize your data

[0362] The server normalizes and cleans the collected and stored data. The input is the stored minefield information, and the output is data formatted for AI analysis. Specifically, it uses data processing libraries such as Pandas to fill in missing values ​​and remove outliers.

[0363] Step 6: Predicting mine locations

[0364] The server inputs the organized data into an AI model to predict where landmines will be buried. The input is normalized landmine information, and the output is coordinate data of the predicted landmine locations. Specifically, a time series prediction model such as a Long Short-Term Memory (LSTM) network is used.

[0365] Step 7: View the prediction results

[0366] The server generates the prediction results in a dashboard format and sends them to the terminal. The terminal visualizes them so that the user can easily check them. The input is the prediction result data sent from the server, and the output is the prediction results displayed in graphs, maps, etc.

[0367] Step 8: Develop a removal plan

[0368] The server then creates a removal work plan based on the identified mine locations and the prediction results. The input is information on mine locations and the prediction results, and the output is a work plan. Specifically, an optimization algorithm is used to generate a plan that includes efficient work procedures and time schedules.

[0369] Step 9: Workplan Approval

[0370] The terminal presents the work plan to the user, who then reviews and approves the plan through the interface. The input is the work plan sent from the server, and the output is the plan approved by the user.

[0371] Step 10: Perform the removal work

[0372] The automated robot begins work in the designated area based on the work plan it receives. The input is the approved work plan, and the output is the number of mines removed and the progress of the work. Specifically, it uses GPS information to move autonomously and physically remove the mines.

[0373] Step 11: Monitoring the work

[0374] The server monitors the work status of the automated robot in real time. The input is progress information from the automated robot, and the output is progress status data updated in real time. Specifically, the data is displayed on the terminal as it is processed.

[0375] Step 12: Analyze emotional state

[0376] The server receives the user's facial expression and voice data and uses an emotion analysis algorithm to identify the user's emotional state. The input is data obtained from a camera or microphone, and the output is the analyzed emotional state. Specific operations use a voice recognition library and an emotion analysis library.

[0377] Step 13: Adjusting the Interface

[0378] The device dynamically adjusts the interface based on the user's emotional state. The input is the analyzed emotional state, and the output is an interface that corresponds to the user's emotional state. Specifically, if the user feels anxious, the interface is simplified to present information in a more understandable manner.

[0379] (Application example 2)

[0380] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0381] Conventional mine clearance systems are required to identify buried mines and remove them safely and efficiently, but there is a problem in that it is difficult for the entire system to perform the task quickly and accurately. Furthermore, they lack support functions that take into account the driver's emotional state, and do not improve safety or comfort while driving. Therefore, a system that can simultaneously remove mines and provide driver support was needed.

[0382] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring satellite images or drone images, means for analyzing the acquired images to identify buried mine locations, means for transmitting the analysis results to the terminal and visually displaying them, means for storing past mine placement information and accident information in a database, means for predicting mine locations based on the stored information, means for providing the prediction results to the terminal so that the user can check them, means for formulating a removal operation plan based on the buried mine locations and the prediction results, an automated robot for executing the formulated operation plan, means for monitoring the operation status of the automated robot in real time, means for analyzing the driver's emotions while driving and dynamically adjusting the interface, and means for providing hazard prediction information in accordance with the driver's emotional state. This not only enables efficient mine removal operations but also provides safe and comfortable assistance to the driver while driving.

[0383] A "server" is a computer system that sends, receives, and processes data over a network.

[0384] "Satellite imagery" refers to high-resolution images of the Earth's surface or objects taken by satellites.

[0385] "Drone imagery" refers to images of the earth's surface or objects taken by unmanned aerial vehicles (drones).

[0386] "Image analysis" is a technology that identifies and identifies objects or specific patterns based on acquired image data.

[0387] "Means for identifying buried landmine locations" refers to technology that uses image analysis to identify locations where landmines are buried.

[0388] A "terminal" is a device such as a computer or smartphone that displays data sent from a server and can be operated by a user.

[0389] "Past mine placement information" refers to data that indicates previously recorded mine placement locations and related information.

[0390] "Accident information" is data that records the location and circumstances of mine-related accidents.

[0391] A "database" is a data collection system that allows information to be efficiently stored, retrieved, and updated.

[0392] The "prediction results" are estimates of future mine locations based on past data.

[0393] A "clearance operation plan" is a plan that outlines specific procedures and schedules for safely clearing landmines based on the identified locations of the mines.

[0394] An "automatic robot" is a machine that autonomously carries out tasks as instructed.

[0395] "Real-time monitoring means" refers to a system for instantly monitoring and checking ongoing situations.

[0396] "Means for analyzing emotions" refers to technology for analyzing a user's facial expressions, voice, etc. to identify their emotional state.

[0397] "Means for dynamically adjusting the interface" refers to technology that automatically changes the display content and operation method according to the user's emotional state.

[0398] "Means for providing risk prediction information" refers to technology that predicts future risks based on past data and current conditions and notifies the user.

[0399] The embodiment of the present invention is based on a series of systems including a server, a terminal, an automatic robot, and an emotion analysis system. This system can analyze images in real time, identify the location of landmines, and efficiently remove them, while also grasping the emotional state of the driver and providing appropriate feedback.

[0400] The server receives image data from satellites and drones and performs image analysis based on this data. Specifically, the server uses a high-performance GPU and deep learning frameworks such as TensorFlow and Keras. This image analysis identifies specific patterns and locates buried mines. The coordinate data of identified mines is sent to the device and displayed visually.

[0401] Past mine placement and accident information is stored in a database. The server integrates this information and uses an AI model to predict where mines will be placed. The results of this prediction are also provided to the device, where users can check them. Predictions require cleansing and normalization of past data.

[0402] The server then creates a removal work plan based on the location of buried mines and the prediction results. This plan, which includes efficient work procedures and time schedules, is displayed on the terminal. Once the user confirms and approves the plan, work instructions are sent to the automated robot. The automated robot uses GPS information to automatically move to the designated area and remove the mines. The server also monitors the automated robot's work status in real time and immediately addresses any problems that arise.

[0403] The integration of an emotion engine is also a key element of this system. The server receives facial expression and voice data from the driver or operator via the device and performs emotion analysis. This analysis uses an emotion recognition algorithm, and if the user feels anxious or tired, the interface is dynamically adjusted. This allows the user to use information more easily and with peace of mind.

[0404] For example, a system can analyze live video feeds of a highway, detect obstacles ahead, and warn the driver. At the same time, if anxiety is detected from the driver's facial expression, the system simplifies the interface and activates reassuring voice guidance. In this way, both mine clearance and driver assistance can be carried out efficiently.

[0405] Example prompt sentence:

[0406] "Creating an application that detects obstacles ahead from live video footage on the highway, analyzes the driver's emotions, and provides appropriate feedback."

[0407] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0408] Step 1:

[0409] The server receives satellite or drone images. These images are sent to the server through a network and stored in a database. The input data is high-resolution image data, and the output is stored image data.

[0410] Step 2:

[0411] The server applies image analysis algorithms to the stored image data. Specifically, it uses deep learning frameworks such as TensorFlow and Keras to identify specific patterns and identify buried mine locations. The input data is the stored image data, and the output is the coordinate data of the mines.

[0412] Step 3:

[0413] The server sends coordinate data of the identified landmines to the terminal. The terminal visually displays the received data on a map and provides the user with information on the location of the landmines. The input data is the coordinate data of the landmines, and the output is the location information of the landmines marked on the map.

[0414] Step 4:

[0415] The server collects and organizes past mine laying and accident information into a database. The input data is past mine and accident information, and the output is an organized database.

[0416] Step 5:

[0417] The server uses an AI model to predict future mine locations based on the collected data. The input data is an organized database, and the output is predicted mine location data.

[0418] Step 6:

[0419] The server sends the prediction results to the terminal, which then visually displays the prediction results to the user. The input data is the predicted mine location data, and the output is the visually displayed prediction results.

[0420] Step 7:

[0421] The server then creates a removal plan based on the identified mine locations and prediction results. The plan includes efficient work procedures and time schedules. The input data is mine location information and prediction results, and the output is a detailed work plan.

[0422] Step 8:

[0423] The server sends the prepared work plan to the automated robot, which then uses GPS information to automatically move to the designated location and remove the mines based on the plan. The input data is the work plan, and the output is the progress of the mine removal work.

[0424] Step 9:

[0425] The server monitors the work status of the automated robot in real time and makes adjustments and instructions as necessary. The input data is progress data, and the output is the monitoring results and instructions.

[0426] Step 10:

[0427] The server receives the driver's facial expression and voice data through the terminal and performs emotion analysis. The input data is facial expression data and voice data, and the output is analyzed emotional state data.

[0428] Step 11:

[0429] The server dynamically adjusts the device interface based on the analyzed emotional state and provides risk prediction information according to the driver's emotional state. The input data are emotional state data and past prediction data, and the output is the dynamically adjusted interface and the provided risk prediction information.

[0430] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0431] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0432] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0433] [Second embodiment]

[0434] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0435] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0436] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0437] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0438] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0439] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0440] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0441] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0442] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0443] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0444] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0445] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0446] This invention relates to a system for safely and efficiently removing landmines using artificial intelligence technology. Specific embodiments for carrying out the invention will be described below.

[0447] Image Recognition

[0448] First, the server receives satellite or drone images, which are then sent to the server via a network.

[0449] The server then uses an image analysis algorithm to analyze the received image data. The algorithm identifies specific patterns and identifies the location of buried mines. The analysis results are output as coordinate data indicating the location of buried mines.

[0450] The coordinate data from the analysis is sent to the device, which receives it and visually displays it. Specifically, it marks specific coordinates on a map to show the user where the mines are buried.

[0451] Information gathering and prediction

[0452] The server collects and stores in a database information on past mine placements and accidents, which is obtained from historical databases and local reports.

[0453] Based on the stored information, the server makes predictions about mine locations. An AI model analyzes this data and predicts unknown mine locations. The predictions are then provided to the device and displayed as a dashboard for the user to access.

[0454] Removal work planning and execution

[0455] Based on the identified mine locations and prediction results, the server creates a removal operation plan, which includes efficient work procedures and time schedules.

[0456] The work plan is presented to the terminal and reviewed and approved by the user. Once approved, the plan is sent to the automated robot.

[0457] The automated robot will then begin clearing operations in the designated area based on the received work plan, and will use GPS information to navigate automatically and safely clear specific mines.

[0458] Finally, the server monitors the work of the automated robot in real time, tracking its progress and updating the data to ensure that the work is going smoothly.

[0459] Specific examples

[0460] For example, when carrying out mine clearance work in a certain area, the following specific steps are taken:

[0461] 1. The server receives the image data sent from the drone.

[0462] 2. The server performs image analysis and finds a mine at a specific coordinate, for example, (10.1234, 20.5678).

[0463] 3. The device displays the analysis results and provides coordinate information to the user.

[0464] 4. The server collects and organizes past mine placement data and accident information, and predicts where mines will be placed.

[0465] 5. The forecast results are displayed as a dashboard, providing information, for example, about the newly predicted coordinates (10.2345, 20.6789).

[0466] 6. The server will create a removal plan based on the identified mine locations and the prediction results.

[0467] 7. The terminal presents the work plan to the user and obtains confirmation.

[0468] 8. Autonomous robots will follow a work plan and head to the site to remove the mines.

[0469] 9. The server monitors the work in real time and tracks the progress.

[0470] This series of processes ensures that mine clearance work is carried out safely and efficiently, and the system protects many people from the dangers of landmines and minimizes damage.

[0471] The processing flow will be explained below.

[0472] Step 1:

[0473] The server receives image data from a satellite or drone. The image data is sent over the network, and the server saves the data in storage. Specifically, it calls the receive_image_data() function to retrieve the image. This data is saved in a variable called "image_data," for example.

[0474] Step 2:

[0475] The server analyzes the received image data. The acquired image data is input into the AI ​​model to identify the location of buried landmines. The AI ​​model contains a specific image recognition algorithm and executes the analyze_image(image_data) function, which identifies the coordinates of the buried landmines.

[0476] Step 3:

[0477] The server sends the analysis results to the device. It generates coordinate data for the identified mines and sends it to the device. It sends the analysis results using the send_results(analysis results, device ID) function.

[0478] Step 4:

[0479] The device receives the analysis results. The device visually displays the received coordinate data. For example, it marks specific coordinates on a map to show the user where the mines are buried. This is done using the display_results(analysis results) function.

[0480] Step 5:

[0481] The server collects information on past mine placements and accidents and stores it in a database. It retrieves information from historical databases and various reports and stores it in the database using the collect_data(data source) function. For example, it stores it in a variable called "historical_data."

[0482] Step 6:

[0483] Organize and consolidate the data collected by the server. Normalize and clean the data to make it analyzable by the AI ​​model. Use the normalize_data(collected_data) function.

[0484] Step 7:

[0485] The server uses an AI model to predict mine locations. The organized data is input into the AI ​​model to predict unknown mine locations. The predict_mine_locations(model, organized data) function is executed to obtain the predicted results.

[0486] Step 8:

[0487] The server provides the prediction results to the terminal. The prediction results are generated so that they can be displayed in dashboard format and sent to the terminal. The dashboard is generated using the generate_dashboard(prediction result) function.

[0488] Step 9:

[0489] The device displays the forecast results to the user, allowing the user to see the results in a visual format (e.g., a graph or map). This is done using the display_dashboard(forecastresult) function.

[0490] Step 10:

[0491] The server creates a mine removal plan. It creates an efficient removal plan based on the prediction results and the identified mine locations. It uses the create_removal_plan(identified location, prediction result) function.

[0492] Step 11:

[0493] The terminal presents the work plan to the user. The details of the work plan are displayed and the user confirms and approves it. The display_plan(plan) function is used.

[0494] Step 12:

[0495] The user checks and approves the work plan. The plan presented on the terminal is checked and determined to be feasible. After approval, the server is notified. The approve_plan() function is used.

[0496] Step 13:

[0497] The automated robot begins work based on the work plan received from the server. It follows GPS information to move to the designated mine site and carry out the removal work. The execute_removal_task(robot, plan) function is used.

[0498] Step 14:

[0499] The server monitors the work status of the automated robot in real time. It receives feedback data from the robot and monitors its progress. It uses the monitor_progress(robot) function.

[0500] These steps allow the system to safely and efficiently clear mines.

[0501] Example 1

[0502] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0503] Landmines are buried widely around the world as remnants of war and conflict, and their removal is extremely dangerous and time-consuming, making it a major social problem in many regions. Conventional mine removal methods rely on manual labor, which entails significant risks and costs. Furthermore, it is often difficult to identify or predict where mines will be buried, making the work inefficient. This invention aims to solve these problems by using artificial intelligence technology to remove landmines safely and efficiently.

[0504] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0505] In this invention, the server includes means for acquiring satellite images or unmanned aerial vehicle images, means for analyzing the acquired images to identify buried mine locations, means for transmitting the analysis results to a terminal and visually displaying them, means for storing information on past buried mines and accident information in a database, means for predicting buried mine locations based on the stored information, means for providing the prediction results to the terminal so that the user can check them, means for formulating a removal work plan based on the buried mine locations and the prediction results, an automated work device for executing the formulated work plan, and means for monitoring the work status of the automated work device in real time. This makes it possible to automate the process from identifying and predicting the location of mines to the removal work as a series of steps, thereby significantly improving the safety and efficiency of work.

[0506] "Satellite imagery" refers to image data of the Earth's surface taken from an artificial satellite in Earth's orbit.

[0507] "Unmanned aerial vehicle imagery" refers to image data of the earth's surface taken from an unmanned aerial vehicle such as a drone.

[0508] "Means of acquisition" refers to the equipment, software, and protocols necessary to receive image data transmitted from satellites and unmanned aerial vehicles.

[0509] "Means of analysis" refers to the algorithms, software, and hardware used to identify buried mine locations using acquired image data.

[0510] "Visual display means" refers to a device or application that provides the analysis results to the user in a visual format such as a map or graph.

[0511] "Means of storing information in a database" refers to the system or structure for managing and storing information on past mine burials and accidents.

[0512] "Predictive tools" refers to software and computational models that use artificial intelligence algorithms to predict new mine locations based on stored data.

[0513] "Means for providing and enabling users to check" refers to an interface or application that provides prediction results in a form that users can easily access and visually check.

[0514] "Planning tools for clearance operations" refers to algorithms and software that plan efficient and safe clearance procedures based on identified mine locations and predicted outcomes.

[0515] "Automated work equipment" refers to robots and mechanical devices that automatically remove mines according to a work plan.

[0516] "Monitoring means" means systems or software that monitor the progress or status of automated work equipment in real time and intervene or adjust as necessary.

[0517] This invention relates to a system for safely and efficiently removing landmines by utilizing artificial intelligence technology. Specific embodiments for carrying out this invention will be described below.

[0518] Image Recognition

[0519] First, the server acquires satellite and drone imagery, which includes, for example, downloading images uploaded to Amazon S3 using the AWS SDK.

[0520] The server then analyzes the acquired image data. Using Python's OpenCV and TensorFlow, it uses an object detection algorithm (such as YOLO) to extract the characteristics of the mines and identify their locations. Once the analysis is complete, it generates the results as coordinate data.

[0521] The server sends the analysis results to the device in JSON format, using WebSocket or REST API for communication. Specifically, the JSON data contains the coordinate information of the analyzed mines.

[0522] The device visually displays the received analysis results and uses the JavaScript Google Maps API to mark the coordinates of detected mines on a map and notify the user, allowing the user to intuitively confirm the location of specific mines.

[0523] Information gathering and prediction

[0524] The server then collects information on past mine placements and accidents and stores it in a database, which is managed using an RDBMS such as MySQL or PostgreSQL, and retrieves information from historical databases and local reports.

[0525] The server predicts mine locations based on the stored information. To do this, it uses machine learning libraries (such as Scikit-learn or TensorFlow) to build a predictive model and analyze past data to estimate unknown mine locations.

[0526] The predicted results are sent from the server to the device and displayed on a dashboard for the user to review. The dashboard, built using React and Vue.js, provides coordinate information for newly predicted mines, allowing users to easily view them.

[0527] Removal work planning and execution

[0528] Based on the prediction results and the identified mine locations, the server creates a removal plan, which includes efficient and safe work procedures, the equipment to be used, and a time schedule.

[0529] The terminal presents the work plan to the user. The user checks the presented plan and presses the approval button if there are no problems. The user's approval is recorded as a log on the server.

[0530] The approved work plan is then sent from the server to the automated work device, using MQTT or other real-time messaging protocols.

[0531] Based on the received plan, the automated device will begin mine clearance work in the designated area. It will use GPS to locate specific mines and move automatically to carry out the clearance work. The work status will be reported to the server in real time.

[0532] The server monitors the operation status of the automated work equipment and tracks its progress. The acquired data is updated to a database and displayed on a dashboard, allowing users to see the progress of the work in real time.

[0533] Specific examples

[0534] For example, if a mine clearance operation is envisaged in a certain area, the following steps would be taken:

[0535] 1. The server receives the image data sent from the drone.

[0536] 2. The server performs image analysis and detects a mine at a specific coordinate, for example, (10.1234, 20.5678).

[0537] 3. The device displays the analysis results and provides coordinate information to the user.

[0538] 4. The server collects and organizes past mine placement data and accident information, and predicts where mines will be placed.

[0539] 5. The forecast results are displayed as a dashboard, providing information about the newly predicted coordinates (10.2345, 20.6789).

[0540] 6. The server creates a removal plan based on the identified mine locations and the prediction results.

[0541] 7. The terminal presents the work plan to the user and obtains confirmation.

[0542] 8. Automated work equipment will head to the site according to the work plan and remove the mines.

[0543] 9. The server monitors the work in real time and tracks the progress.

[0544] Prompt Sentence Examples

[0545] Example prompts to input to a generative AI model:

[0546] "Please analyze satellite images to identify buried mine sites."

[0547] "Build a predictive model for mine placement using historical data."

[0548] "Please draw up a mine clearance operation plan for the specified coordinates."

[0549] This series of processes ensures that mine clearance work is carried out safely and efficiently, and that many people are protected from the dangers of landmines.

[0550] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0551] Program processing flow

[0552] Step 1:

[0553] The server receives image data from satellites and unmanned aerial vehicles. Specifically, it downloads the image data using HTTP or FTP protocols. The input is the image file, which becomes the data for processing in the next step.

[0554] Step 2:

[0555] The server analyzes the received image data and applies an object detection algorithm (such as YOLO) using Python's OpenCV or TensorFlow. The input for this analysis is the acquired image file, and the output is coordinate data indicating the location of buried mines. Specifically, the server extracts image features and identifies the parts that can be identified as mines using a trained model.

[0556] Step 3:

[0557] The server sends the analysis results, which identify the buried mine locations, to the device. Coordinate data is sent in JSON format using WebSocket or REST API. The input is the coordinate data from the analysis results, and the output is the JSON data transferred to the device. For example, coordinates are sent in the format {"latitude": 10.1234, "longitude": 20.5678}.

[0558] Step 4:

[0559] The device receives the analysis results sent from the server and displays them visually. It uses the Google Maps API in JavaScript to mark the identified coordinates on a map. The input is coordinate data in JSON format, and the output is a map that is displayed to the user. The device draws icons on the map to indicate the location of mines, allowing the user to intuitively identify them.

[0560] Step 5:

[0561] The server collects information on past mine placements and accidents and stores it in a database. Sources of information include historical databases and local reports, and the data is managed using MySQL or PostgreSQL. The input is data from each source, and the output is information stored in the database. Specifically, the server retrieves information through an API, standardizes it, and inserts it into the database.

[0562] Step 6:

[0563] The server predicts mine locations based on the stored information. It uses a machine learning library (such as Scikit-learn or TensorFlow) to learn from past data and build a predictive model. The input is past data stored in the database, and the output is the coordinates of predicted mine locations. The server uses the trained model to analyze the input data and predict unknown mine locations.

[0564] Step 7:

[0565] The server sends the prediction results to the device, where the user can view them on a dashboard. The prediction results are sent to the device in JSON format. The input is coordinate data from the prediction model, and the output is coordinate information displayed on the dashboard. The device uses front-end libraries such as React and Vue.js to display the information in a visually easy-to-understand format.

[0566] Step 8:

[0567] The server creates a removal operation plan based on the identified mine locations and prediction results. The input is mine coordinate data and prediction data, and the output is a detailed operation plan. The server calculates efficient and safe operation procedures and generates a timeline for each step and a list of required equipment.

[0568] Step 9:

[0569] The terminal presents the work plan to the user, who then reviews and approves the plan. The input is the work plan sent from the server, and the output is the user's approval data. The terminal displays the work plan to the user as an interactive guide and provides an approval button.

[0570] Step 10:

[0571] The server sends the approved work plan to the automated work device. It transmits data in real time using the MQTT protocol. The input is the work plan approved by the user, and the output is the data sent to the automated work device. The server starts work according to the instructions.

[0572] Step 11:

[0573] The automated work device carries out mine removal work based on the received work plan. It uses GPS to move to the designated area and remove the mines. The input is the work plan data, and the output is a report data on the completion of the removal work. The automated work device periodically sends its progress to the server.

[0574] Step 12:

[0575] The server monitors the work status of the automated work equipment in real time. The server receives the transmitted progress data and tracks the work progress. The input is feedback data from the automated work equipment, and the output is updated progress data. The server visually displays the progress to the user and issues warnings if any problems occur.

[0576] (Application example 1)

[0577] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0578] Conventional autonomous vehicle systems lack sufficient means to detect obstacles and accidents on the road in real time and provide safe detour routes. This has resulted in increased operational interruptions and dangers due to accidents and obstacles, resulting in problems with operational efficiency and safety. In addition, existing mine clearance systems are limited to mine detection and clearance and lack the ability to respond to dynamically changing situations.

[0579] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0580] In this invention, the server includes means for acquiring satellite or drone images, means for analyzing the acquired images to identify buried mine locations, means for transmitting the analysis results to a terminal and visually displaying them, means for storing information on past mine placements and accident information in a database, means for predicting mine locations based on the stored information, means for providing the prediction results to the terminal so that a user can check them, means for formulating a mine removal operation plan based on the buried mine locations and the prediction results, an automated machine for executing the operation plan, means for monitoring the operation status of the automated machine in real time, and means for detecting hazardous objects while the automated machine is operating and providing an alternative route. This enables automated vehicles to detect hazardous objects on roads in real time and provide safe detour routes, thereby improving safety and operating efficiency.

[0581] "Satellite imagery" refers to image data obtained by satellites photographing the Earth's surface.

[0582] "Drone images" are image data taken from the air by a drone, a small unmanned aerial vehicle.

[0583] "Analysis" is the process of applying specific algorithms to acquired image data to extract information and identify objects of interest.

[0584] "Mine site" refers to the location where a landmine is buried underground.

[0585] A "terminal" is an electronic device that a user uses to receive and visually view information.

[0586] A "database" is a computer system for organizing and storing data, allowing for efficient searching and updating of data.

[0587] "Prediction" refers to predicting future situations or events based on past data and current information.

[0588] A "clearance operation plan" is a plan that outlines the methods and procedures for safely and efficiently removing buried mine sites.

[0589] An "automated machine" refers to a robot or mechanical device that performs work automatically according to programmed instructions.

[0590] "Monitoring" refers to watching the progress of a system or task in real time and making adjustments as needed.

[0591] "Hazardous material" means any substance or condition that may cause injury or danger during operation or work.

[0592] "Alternate Route" means a proposed alternative route to avoid an accident or hazard.

[0593] MODE FOR CARRYING OUT THE INVENTION

[0594] The present invention provides a system for an autonomous vehicle to detect dangerous objects on the road and provide a safe detour route. The system comprises the following means.

[0595] First, the server acquires satellite or drone images, which are then transmitted in real time and aggregated on the server, where they are analyzed using image analysis algorithms such as OpenCV and Keras, utilizing high-performance computing resources.

[0596] As a result of the analysis, the locations of buried mines and other hazards are identified. This identified information is sent to the device and visually displayed to the user. For example, the coordinate information indicated by the analysis results is marked on a map to inform the user.

[0597] The server also stores information on past mine placements and accidents in a database. This information is used to predict mines and other hazards. An AI model analyzes this data and generates predictions. These predictions are also displayed on the device as a dashboard for users to review.

[0598] Based on the location of buried mines and the prediction results, the server creates a removal work plan. This work plan includes efficient work procedures and time schedules. The created work plan is displayed on the terminal and the user confirms and approves it. The approved plan is then sent to the automated machine.

[0599] The automated machines begin clearance operations in designated areas based on a work plan. They use GPS information to navigate automatically and safely remove specific mines and hazardous materials. Additionally, the automated machines have the ability to detect new hazards while in operation and can provide alternative routes based on detected hazards.

[0600] The server also monitors the work status of the automated machines in real time, allowing it to track the progress of the work and update the data to ensure that the work is progressing smoothly.

[0601] Specific examples

[0602] For example, when clearing landmines in an area, drones can take pictures of accidents on the road and send the images to a server, which analyzes the images and detects that an accident has occurred. This information is then sent to the autonomous vehicle's system, which then provides a safe detour route.

[0603] Prompt Sentence Examples

[0604] "As an autonomous vehicle travels from Tokyo to Los Angeles, a drone detects an accident on the road. Use the image analysis system to confirm the existence of the accident, use the Google Maps API to obtain a safe detour route, and provide that information to the autonomous vehicle."

[0605] With the introduction of this system, autonomous vehicles will be able to monitor road conditions in real time and quickly detect hazards and accidents, improving safety and operational efficiency.

[0606] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0607] Step 1:

[0608] The server receives image data transmitted from a satellite or drone. As input, the image data from the satellite or drone is passed to the server. As output, the raw data is obtained and stored in the server.

[0609] Step 2:

[0610] The server analyzes the image data it receives. It uses OpenCV and Keras as its analysis algorithms to identify landmines and hazardous objects in the image. The image data stored on the server is used as input. The output is the coordinate data of the identified landmines and hazardous objects.

[0611] Step 3:

[0612] The analysis results are sent to the terminal and displayed for the user to visually confirm. As input, the identified coordinate data is sent from the server to the terminal. As output, the coordinate information is displayed marked on a map on the terminal screen.

[0613] Step 4:

[0614] The server stores the past mine laying information and accident information in the database. As input, the past mine laying data and accident information are taken into the database. As output, the information storage is completed.

[0615] Step 5:

[0616] The server analyzes the stored information and predicts mine locations and hazardous materials. It uses a generative AI model to make predictions based on past data. The input is the stored data in the database. The output is the predicted coordinate data.

[0617] Step 6:

[0618] The prediction results are provided to the terminal and displayed for the user to check. As input, the predicted coordinate data is sent to the terminal. As output, the prediction information is displayed on the dashboard.

[0619] Step 7:

[0620] The server creates a removal operation plan based on the location of the mines and the prediction results. The input is the identified and predicted coordinate data. The output is a removal operation plan that includes the work procedure and time schedule.

[0621] Step 8:

[0622] The work plan is presented to the terminal and the user confirms and approves it. The work plan is sent to the terminal as input. The user's confirmation and approval is sent from the terminal to the server as output.

[0623] Step 9:

[0624] The automated machine starts the removal work in the designated area based on the received work plan. It uses GPS information to move automatically and safely remove the mines. The work plan and GPS information are used as inputs. The progress of the removal work is sent from the work site to the server as output.

[0625] Step 10:

[0626] The server monitors the work status of the automated machine in real time and tracks the progress. As input, work status data sent from the work site is taken into the server. As output, the progress status is updated in real time and displayed on the monitoring screen.

[0627] Step 11:

[0628] The automated machine detects new hazards while in operation and provides an alternative route. As input, sensor data acquired during operation is used. As output, an alternative route to avoid the hazard is provided to the automated machine.

[0629] This allows autonomous vehicle systems to monitor road conditions in real time, clear mines and provide safe detour routes.

[0630] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0631] This invention is a system for safely and efficiently removing landmines by using artificial intelligence technology and an emotion engine. Specific embodiments for carrying out the invention will be described below.

[0632] Image Recognition

[0633] First, the server receives image data from satellites or drones, and these images are sent to the server via a network and stored appropriately.

[0634] The server then uses the received image data to apply image analysis algorithms, using AI models to identify specific patterns and pinpoint the location of buried mines, generating coordinate data for the mines.

[0635] The analysis results are sent to the device, which receives this data and visually displays it, specifically marking specific coordinates on a map to show the user the location of the mines.

[0636] Information gathering and prediction

[0637] The server collects and stores information about past mine placements and accidents in a database, which is obtained from historical databases and local reports.

[0638] The server then organizes and consolidates the collected data, normalizing and cleaning it to make it suitable for analysis by AI models.

[0639] The server uses an AI model to predict where landmines are buried. The organized data is passed to the AI ​​model, which then predicts unknown landmine locations. The prediction results are generated in a dashboard format and provided to the device.

[0640] The device visually displays the prediction results to the user, specifically by showing the predicted locations using graphs or maps so that the user can easily check them.

[0641] Removal work planning and execution

[0642] The server then creates a removal plan based on the identified mine locations and the prediction results, generating a plan that includes efficient work procedures and a time schedule.

[0643] The work plan is presented on the terminal and an interface is provided for the user to review and approve. Once the user reviews and approves the plan, it is sent to the automated robot.

[0644] The automated robot will begin work in the designated area based on the received work plan, and will use GPS information to navigate automatically and remove mines.

[0645] The server monitors the work status of the automated robot in real time, constantly updating progress information to ensure that the work is progressing smoothly.

[0646] Emotion engine integration

[0647] Furthermore, the system is integrated with an emotion engine that recognizes the user's emotions. The server receives and analyzes the user's facial expression and voice data.

[0648] The server uses an emotion analysis algorithm to determine the user's emotional state, for example, if the user is feeling anxious, that emotional state is detected.

[0649] The device dynamically adjusts the interface based on the user's emotional state, for example simplifying the interface to present information more clearly when the user is feeling anxious.

[0650] Specific examples

[0651] As a concrete example, the procedure for carrying out mine clearance work in a certain area will be explained.

[0652] 1. The server receives the image data sent from the drone.

[0653] 2. The server performs image analysis and detects a mine at coordinates (10.1234, 20.5678).

[0654] 3. The device displays the analysis results on a map for the user.

[0655] 4. The server collects and organizes past mine data and predicts where mines are buried.

[0656] 5. The server generates the prediction results and provides them to the device as a dashboard.

[0657] 6. The device displays the prediction results for the user to confirm.

[0658] 7. The server creates a removal plan and displays it on the terminal.

[0659] 8. The user reviews and approves the work plan.

[0660] 9. The automated robot moves to the designated location and removes the mines.

[0661] 10. The server monitors the robot's work status in real time.

[0662] 11. The server analyzes the user's emotions and dynamically adjusts the interface.

[0663] This series of processes not only ensures safe and efficient mine clearance work, but also reduces the psychological burden on the user. This system protects many people from the dangers of landmines while also improving the efficiency of work.

[0664] The processing flow will be explained below.

[0665] Step 1:

[0666] The server receives image data from a satellite or drone. The image data is sent over the network, and the server saves the data in storage. Specifically, it calls the receive_image_data() function to retrieve the image. This data is saved in a variable called "image_data," for example.

[0667] Step 2:

[0668] The server analyzes the received image data. The acquired image data is input into the AI ​​model to identify the location of buried landmines. The AI ​​model contains a specific image recognition algorithm and executes the analyze_image(image_data) function, which identifies the coordinates of the buried landmines.

[0669] Step 3:

[0670] The server sends the analysis results to the device. It generates coordinate data for the identified mines and sends it to the device. It sends the analysis results using the send_results(analysis results, device ID) function.

[0671] Step 4:

[0672] The device receives the analysis results. The device visually displays the received coordinate data. For example, the display_results(analysis results) function is used to mark specific coordinates on a map to show the user where the mines are located.

[0673] Step 5:

[0674] The server collects information on past mine placements and accidents and stores it in a database. It retrieves information from historical databases and various reports and stores it in the database using the collect_data(data source) function. For example, it is stored in a variable called "historical_data."

[0675] Step 6:

[0676] Organize and consolidate the data collected by the server. Normalize and clean the data to make it analyzable by the AI ​​model. Use the normalize_data(collected_data) function.

[0677] Step 7:

[0678] The server uses an AI model to predict mine locations. The organized data is input into the AI ​​model to predict unknown mine locations. The predict_mine_locations(model, organized data) function is executed to obtain the predicted results.

[0679] Step 8:

[0680] The server provides the prediction results to the terminal. The prediction results are generated so that they can be displayed in dashboard format and sent to the terminal. The dashboard is generated using the generate_dashboard(prediction result) function.

[0681] Step 9:

[0682] The device displays the forecast results to the user, allowing the user to see the results in a visual format (e.g., a graph or map). This is done using the display_dashboard(forecastresult) function.

[0683] Step 10:

[0684] The server creates a mine removal plan. It creates an efficient removal plan based on the prediction results and the identified mine locations. It uses the create_removal_plan(identified location, prediction result) function.

[0685] Step 11:

[0686] The terminal presents the work plan to the user. The details of the work plan are displayed and the user confirms and approves it. The display_plan(plan) function is used.

[0687] Step 12:

[0688] The user checks and approves the work plan. The plan presented on the terminal is checked and determined to be feasible. After approval, the server is notified. The approve_plan() function is used.

[0689] Step 13:

[0690] The automated robot begins work based on the work plan received from the server. It follows GPS information to move to the designated mine site and carry out the removal work. The execute_removal_task(robot, plan) function is used.

[0691] Step 14:

[0692] The server monitors the work status of the automated robot in real time. It receives feedback data from the robot and monitors its progress. It uses the monitor_progress(robot) function.

[0693] Step 15:

[0694] The server receives the user's facial expression and voice data and performs emotion analysis. It analyzes the user's emotional state in real time and identifies the state using the emotion analysis engine. It uses the analyze_emotion(facial expression data, voice data) function.

[0695] Step 16:

[0696] The device dynamically adjusts the interface based on the user's emotional state. For example, if the user is feeling anxious, the interface can be simplified to make the information easier for the user to understand. This is done using the adjust_interface(emotional_state) function.

[0697] These steps allow the system to safely and efficiently remove mines and further optimize the user experience based on the user's emotional state. For example, in step 7 above, the system predicts that a new mine has been buried at coordinates (10.1234, 20.5678), and the removal process proceeds based on that prediction. The interface is simplified when the user feels anxious. In this way, the overall system further enhances safety and efficiency, and also provides psychological support.

[0698] Example 2

[0699] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0700] Conventional mine clearance systems have low analytical accuracy for accurately identifying the location of mines, limiting the efficiency of clearance work. Furthermore, they lack a method for reducing the psychological burden on users. Therefore, there is a need for a system that can clear mines quickly and safely while also reducing the psychological burden on users.

[0701] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for acquiring satellite images or drone images; means for analyzing the acquired images to identify buried mine locations; means for transmitting the analysis results to the terminal and visually displaying them; means for storing past mine placement information and accident information in a database; means for predicting mine locations based on the stored information; means for providing the prediction results to the terminal so that the user can confirm them; means for formulating a removal work plan based on the buried mine locations and the prediction results; an automatic robot for executing the formulated work plan; means for monitoring the work status of the automatic robot in real time; emotion engine means for receiving and analyzing facial expression data and voice data of the user; and means for dynamically adjusting the terminal interface based on the user's emotional state. This enables mine identification, prediction, and removal work to be performed quickly and safely while reducing the psychological burden on the user.

[0702] "Satellite imagery" refers to image data taken of a specific area on Earth by an artificial satellite.

[0703] "Drone images" are aerial image data taken by a camera mounted on a drone.

[0704] "Image analysis" is the technique of processing acquired image data to identify specific objects.

[0705] "Mine burial location" is information about the location where a mine is buried in the ground.

[0706] A "terminal" is a computing device through which a user can view information.

[0707] A "database" is a system that systematically stores and manages data.

[0708] "Prediction results" are estimated information derived using artificial intelligence algorithms.

[0709] A "clearance operation plan" is a specific procedure and schedule for efficiently clearing landmines.

[0710] An "automatic robot" is a mechanical device that operates autonomously based on set instructions to remove landmines.

[0711] "Real-time monitoring" is a technology that instantly monitors current progress.

[0712] An "emotion engine" is a technology that analyzes a user's facial expression data and voice data to identify their emotional state.

[0713] An "interface" is a screen or operating method that serves as a point of contact for the user and the system to exchange information with each other.

[0714] An "artificial intelligence algorithm" is a computational method that analyzes large amounts of data to derive patterns and predictions.

[0715] "Visually displaying" means displaying information on a screen in a format that is easy for the user to understand.

[0716] "Facial expression data" is information that captures the user's facial movements and expressions.

[0717] "Voice data" is information that records the user's utterances and voice characteristics.

[0718] "Dynamic adjustment" means changing the settings flexibly according to the situation at hand.

[0719] MODE FOR CARRYING OUT THE INVENTION

[0720] This invention is a system that combines artificial intelligence technology and an emotion engine to safely and efficiently remove landmines. Specific embodiments for carrying out this invention will be described below.

[0721] Image Recognition

[0722] First, the server receives image data sent from satellites or drones. This image data is then sent to the server via a network and stored in an appropriate storage. This process can use cloud storage such as Amazon S3.

[0723] The server then uses the received image data to apply image analysis algorithms, specifically using machine learning libraries such as TensorFlow to identify the locations of buried mines using a Convolutional Neural Network (CNN) model, and generates coordinate data for the areas where the mines are located.

[0724] The analysis results are sent to the device and displayed visually, and the device uses the Google Maps API or similar to mark specific coordinates on a map to show the user the location of the mines.

[0725] Information gathering and prediction

[0726] The server collects and stores information on past mine placements and accidents in a database. This information is obtained from historical databases and local reports. The database can be a database system such as PostgreSQL.

[0727] The server then normalizes and cleans the collected and stored data, converting it into a format suitable for AI analysis. This process uses data processing libraries such as Pandas.

[0728] The server inputs the organized data into an AI model to predict the location of buried mines, using a time-series prediction model such as a Long Short-Term Memory (LSTM) network. The prediction results are generated in the form of a dashboard and sent to the device.

[0729] The terminal visualizes the prediction results provided by the server, allowing users to easily check them. By using concrete diagrams and maps, users can intuitively understand the prediction results.

[0730] Removal work planning and execution

[0731] The server then creates a removal plan based on the identified mine locations and the predicted results. This plan includes efficient work procedures and time schedules. An optimization algorithm can be used to generate the most efficient removal plan.

[0732] The work plan is presented on the terminal and an interface is provided for the user to review and approve, and once the user approves the plan, it is sent to the automated robot.

[0733] The autonomous robot uses a GPS device to automatically navigate to the designated area and begin the clearance process. Specifically, the robot detects landmines and uses explosive ordnance disposal equipment to safely remove them.

[0734] The server monitors the work status of the automated robot in real time, and progress information is updated and displayed on the terminal.

[0735] Emotion engine integration

[0736] Furthermore, this system is integrated with an emotion engine that recognizes the user's emotions. The server receives and analyzes the user's facial expression and voice data. The analysis uses a voice recognition library and an expression analysis library.

[0737] The server uses an emotion analysis algorithm to identify the user's emotional state, for example, if the user is feeling anxious, that emotion is detected.

[0738] The device dynamically adjusts its interface based on the user's emotional state: when the user is feeling anxious, it simplifies the interface and provides more understandable information.

[0739] Examples of concrete examples and prompts

[0740] As a concrete example, the procedure for carrying out mine clearance work in a certain area is shown below.

[0741] 1. The server receives high-resolution image data sent from the drone and begins processing.

[0742] 2. The server uses a Convolutional Neural Network (CNN) to analyze the image and detect a landmine at coordinates (10.1234, 20.5678).

[0743] 3. The device uses the Google Maps API to mark the analysis results on a map in real time and display them to the user.

[0744] 4. The server collects and organizes landmine data from the past 10 years from the PostgreSQL database and inputs it into the AI ​​model.

[0745] 5. The server uses the LSTM model to generate a dashboard of predicted mine locations and send it to the device.

[0746] 6. The device displays the dashboard provided by the server, and the user checks the prediction results.

[0747] 7. The server uses an algorithm to create an optimal removal plan and displays it on the device.

[0748] 8. The user reviews the plan and approves it through the device interface.

[0749] 9. An automated robot uses a GPS device to navigate to a designated location and uses a robotic arm to dig up the mines.

[0750] 10. The server monitors the robot's operation in real time and displays the progress on the terminal.

[0751] 11. The server receives the user's facial expression data from the camera and detects feelings of anxiety.

[0752] 12. The device simplifies the interface and displays a reassuring message to the user.

[0753] These systems allow mine clearance work to be carried out quickly and safely, and also reduce the psychological burden on users.

[0754] Example prompt sentence:

[0755] "Please generate explanatory text for a mine removal system that detects buried mine locations from image data, makes predictions based on past mine data, and allows users to proceed with confirmation and removal work with confidence."

[0756] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0757] Step 1: Receiving image data

[0758] The server receives high-resolution image data transmitted from a satellite or drone. The input is the image data transmitted from the satellite or drone, and the output is the image data stored in the server's storage. Specifically, this can be done using cloud storage such as Amazon S3.

[0759] Step 2: Image analysis

[0760] The server uses the image data received by the server to identify buried landmine locations by applying a Convolutional Neural Network (CNN) model using machine learning libraries such as TensorFlow. The input is the image data stored in storage, and the output is the coordinate data of the area where the landmines are located. Specifically, the CNN model extracts image features and identifies the patterns of landmines.

[0761] Step 3: Viewing the analysis results

[0762] The server sends the results of the image analysis to the device. The device visually displays the analysis results using Google Maps API or similar. The input is the coordinate data sent from the server, and the output is the location of the mine marked on the map. Specifically, it marks specific coordinates on the map to show the user the location of the mine.

[0763] Step 4: Gather information

[0764] The server collects information on past mine placements and accidents from historical databases and local reports, and stores this information in a database. The input is mine information obtained from external data sources, and the output is information stored in the database. Specific operations use a database system such as PostgreSQL.

[0765] Step 5: Organize your data

[0766] The server normalizes and cleans the collected and stored data. The input is the stored minefield information, and the output is data formatted for AI analysis. Specifically, it uses data processing libraries such as Pandas to fill in missing values ​​and remove outliers.

[0767] Step 6: Predicting mine locations

[0768] The server inputs the organized data into an AI model to predict where landmines will be buried. The input is normalized landmine information, and the output is coordinate data of the predicted landmine locations. Specifically, a time series prediction model such as a Long Short-Term Memory (LSTM) network is used.

[0769] Step 7: View the prediction results

[0770] The server generates the prediction results in a dashboard format and sends them to the terminal. The terminal visualizes them so that the user can easily check them. The input is the prediction result data sent from the server, and the output is the prediction results displayed in graphs, maps, etc.

[0771] Step 8: Develop a removal plan

[0772] The server then creates a removal work plan based on the identified mine locations and the prediction results. The input is information on mine locations and the prediction results, and the output is a work plan. Specifically, an optimization algorithm is used to generate a plan that includes efficient work procedures and time schedules.

[0773] Step 9: Workplan Approval

[0774] The terminal presents the work plan to the user, who then reviews and approves the plan through the interface. The input is the work plan sent from the server, and the output is the plan approved by the user.

[0775] Step 10: Perform the removal work

[0776] The automated robot begins work in the designated area based on the work plan it receives. The input is the approved work plan, and the output is the number of mines removed and the progress of the work. Specifically, it uses GPS information to move autonomously and physically remove the mines.

[0777] Step 11: Monitoring the work

[0778] The server monitors the work status of the automated robot in real time. The input is progress information from the automated robot, and the output is progress status data updated in real time. Specifically, the data is displayed on the terminal as it is processed.

[0779] Step 12: Analyze emotional state

[0780] The server receives the user's facial expression and voice data and uses an emotion analysis algorithm to identify the user's emotional state. The input is data obtained from a camera or microphone, and the output is the analyzed emotional state. Specific operations use a voice recognition library and an emotion analysis library.

[0781] Step 13: Adjusting the Interface

[0782] The device dynamically adjusts the interface based on the user's emotional state. The input is the analyzed emotional state, and the output is an interface that corresponds to the user's emotional state. Specifically, if the user feels anxious, the interface is simplified to present information in a more understandable manner.

[0783] (Application example 2)

[0784] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0785] Conventional mine clearance systems are required to identify buried mines and remove them safely and efficiently, but there is a problem in that it is difficult for the entire system to perform the task quickly and accurately. Furthermore, they lack support functions that take into account the driver's emotional state, and do not improve safety or comfort while driving. Therefore, a system that can simultaneously remove mines and provide driver support was needed.

[0786] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring satellite images or drone images, means for analyzing the acquired images to identify buried mine locations, means for transmitting the analysis results to the terminal and visually displaying them, means for storing past mine placement information and accident information in a database, means for predicting mine locations based on the stored information, means for providing the prediction results to the terminal so that the user can check them, means for formulating a removal operation plan based on the buried mine locations and the prediction results, an automated robot for executing the formulated operation plan, means for monitoring the operation status of the automated robot in real time, means for analyzing the driver's emotions while driving and dynamically adjusting the interface, and means for providing hazard prediction information in accordance with the driver's emotional state. This not only enables efficient mine removal operations but also provides safe and comfortable assistance to the driver while driving.

[0787] A "server" is a computer system that sends, receives, and processes data over a network.

[0788] "Satellite imagery" refers to high-resolution images of the Earth's surface or objects taken by satellites.

[0789] "Drone imagery" refers to images of the earth's surface or objects taken by unmanned aerial vehicles (drones).

[0790] "Image analysis" is a technology that identifies and identifies objects or specific patterns based on acquired image data.

[0791] "Means for identifying buried landmine locations" refers to technology that uses image analysis to identify locations where landmines are buried.

[0792] A "terminal" is a device such as a computer or smartphone that displays data sent from a server and can be operated by a user.

[0793] "Past mine placement information" refers to data that indicates previously recorded mine placement locations and related information.

[0794] "Accident information" is data that records the location and circumstances of mine-related accidents.

[0795] A "database" is a data collection system that allows information to be efficiently stored, retrieved, and updated.

[0796] The "prediction results" are estimates of future mine locations based on past data.

[0797] A "clearance operation plan" is a plan that outlines specific procedures and schedules for safely clearing landmines based on the identified locations of the mines.

[0798] An "automatic robot" is a machine that autonomously carries out tasks as instructed.

[0799] "Real-time monitoring means" refers to a system for instantly monitoring and checking ongoing situations.

[0800] "Means for analyzing emotions" refers to technology for analyzing a user's facial expressions, voice, etc. to identify their emotional state.

[0801] "Means for dynamically adjusting the interface" refers to technology that automatically changes the display content and operation method according to the user's emotional state.

[0802] "Means for providing risk prediction information" refers to technology that predicts future risks based on past data and current conditions and notifies the user.

[0803] The embodiment of the present invention is based on a series of systems including a server, a terminal, an automatic robot, and an emotion analysis system. This system can analyze images in real time, identify the location of landmines, and efficiently remove them, while also grasping the emotional state of the driver and providing appropriate feedback.

[0804] The server receives image data from satellites and drones and performs image analysis based on this data. Specifically, the server uses a high-performance GPU and deep learning frameworks such as TensorFlow and Keras. This image analysis identifies specific patterns and locates buried mines. The coordinate data of identified mines is sent to the device and displayed visually.

[0805] Past mine placement and accident information is stored in a database. The server integrates this information and uses an AI model to predict where mines will be placed. The results of this prediction are also provided to the device, where users can check them. Predictions require cleansing and normalization of past data.

[0806] The server then creates a removal work plan based on the location of buried mines and the prediction results. This plan, which includes efficient work procedures and time schedules, is displayed on the terminal. Once the user confirms and approves the plan, work instructions are sent to the automated robot. The automated robot uses GPS information to automatically move to the designated area and remove the mines. The server also monitors the automated robot's work status in real time and immediately addresses any problems that arise.

[0807] The integration of an emotion engine is also a key element of this system. The server receives facial expression and voice data from the driver or operator via the device and performs emotion analysis. This analysis uses an emotion recognition algorithm, and if the user feels anxious or tired, the interface is dynamically adjusted. This allows the user to use information more easily and with peace of mind.

[0808] For example, a system can analyze live video feeds of a highway, detect obstacles ahead, and warn the driver. At the same time, if anxiety is detected from the driver's facial expression, the system simplifies the interface and activates reassuring voice guidance. In this way, both mine clearance and driver assistance can be carried out efficiently.

[0809] Example prompt sentence:

[0810] "Creating an application that detects obstacles ahead from live video footage on the highway, analyzes the driver's emotions, and provides appropriate feedback."

[0811] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0812] Step 1:

[0813] The server receives satellite or drone images. These images are sent to the server through a network and stored in a database. The input data is high-resolution image data, and the output is stored image data.

[0814] Step 2:

[0815] The server applies image analysis algorithms to the stored image data. Specifically, it uses deep learning frameworks such as TensorFlow and Keras to identify specific patterns and identify buried mine locations. The input data is the stored image data, and the output is the coordinate data of the mines.

[0816] Step 3:

[0817] The server sends coordinate data of the identified landmines to the terminal. The terminal visually displays the received data on a map and provides the user with information on the location of the landmines. The input data is the coordinate data of the landmines, and the output is the location information of the landmines marked on the map.

[0818] Step 4:

[0819] The server collects and organizes past mine laying and accident information into a database. The input data is past mine and accident information, and the output is an organized database.

[0820] Step 5:

[0821] The server uses an AI model to predict future mine locations based on the collected data. The input data is an organized database, and the output is predicted mine location data.

[0822] Step 6:

[0823] The server sends the prediction results to the terminal, which then visually displays the prediction results to the user. The input data is the predicted mine location data, and the output is the visually displayed prediction results.

[0824] Step 7:

[0825] The server then creates a removal plan based on the identified mine locations and prediction results. The plan includes efficient work procedures and time schedules. The input data is mine location information and prediction results, and the output is a detailed work plan.

[0826] Step 8:

[0827] The server sends the prepared work plan to the automated robot, which then uses GPS information to automatically move to the designated location and remove the mines based on the plan. The input data is the work plan, and the output is the progress of the mine removal work.

[0828] Step 9:

[0829] The server monitors the work status of the automated robot in real time and makes adjustments and instructions as necessary. The input data is progress data, and the output is the monitoring results and instructions.

[0830] Step 10:

[0831] The server receives the driver's facial expression and voice data through the terminal and performs emotion analysis. The input data is facial expression data and voice data, and the output is analyzed emotional state data.

[0832] Step 11:

[0833] The server dynamically adjusts the device interface based on the analyzed emotional state and provides risk prediction information according to the driver's emotional state. The input data are emotional state data and past prediction data, and the output is the dynamically adjusted interface and the provided risk prediction information.

[0834] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0835] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0836] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0837] [Third embodiment]

[0838] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0839] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0840] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0841] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0842] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0843] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0844] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0845] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0846] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0847] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0848] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0849] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0850] This invention relates to a system for safely and efficiently removing landmines using artificial intelligence technology. Specific embodiments for carrying out the invention will be described below.

[0851] Image Recognition

[0852] First, the server receives satellite or drone images, which are then sent to the server via a network.

[0853] The server then uses an image analysis algorithm to analyze the received image data. The algorithm identifies specific patterns and identifies the location of buried mines. The analysis results are output as coordinate data indicating the location of buried mines.

[0854] The coordinate data from the analysis is sent to the device, which receives it and visually displays it. Specifically, it marks specific coordinates on a map to show the user where the mines are buried.

[0855] Information gathering and prediction

[0856] The server collects and stores in a database information on past mine placements and accidents, which is obtained from historical databases and local reports.

[0857] Based on the stored information, the server makes predictions about mine locations. An AI model analyzes this data and predicts unknown mine locations. The predictions are then provided to the device and displayed as a dashboard for the user to access.

[0858] Removal work planning and execution

[0859] Based on the identified mine locations and prediction results, the server creates a removal operation plan, which includes efficient work procedures and time schedules.

[0860] The work plan is presented to the terminal and reviewed and approved by the user. Once approved, the plan is sent to the automated robot.

[0861] The automated robot will then begin clearing operations in the designated area based on the received work plan, and will use GPS information to navigate automatically and safely clear specific mines.

[0862] Finally, the server monitors the work of the automated robot in real time, tracking its progress and updating the data to ensure that the work is going smoothly.

[0863] Specific examples

[0864] For example, when carrying out mine clearance work in a certain area, the following specific steps are taken:

[0865] 1. The server receives the image data sent from the drone.

[0866] 2. The server performs image analysis and finds a mine at a specific coordinate, for example, (10.1234, 20.5678).

[0867] 3. The device displays the analysis results and provides coordinate information to the user.

[0868] 4. The server collects and organizes past mine placement data and accident information, and predicts where mines will be placed.

[0869] 5. The forecast results are displayed as a dashboard, providing information, for example, about the newly predicted coordinates (10.2345, 20.6789).

[0870] 6. The server will create a removal plan based on the identified mine locations and the prediction results.

[0871] 7. The terminal presents the work plan to the user and obtains confirmation.

[0872] 8. Autonomous robots will follow a work plan and head to the site to remove the mines.

[0873] 9. The server monitors the work in real time and tracks the progress.

[0874] This series of processes ensures that mine clearance work is carried out safely and efficiently, and the system protects many people from the dangers of landmines and minimizes damage.

[0875] The processing flow will be explained below.

[0876] Step 1:

[0877] The server receives image data from a satellite or drone. The image data is sent over the network, and the server saves the data in storage. Specifically, it calls the receive_image_data() function to retrieve the image. This data is saved in a variable called "image_data," for example.

[0878] Step 2:

[0879] The server analyzes the received image data. The acquired image data is input into the AI ​​model to identify the location of buried landmines. The AI ​​model contains a specific image recognition algorithm and executes the analyze_image(image_data) function, which identifies the coordinates of the buried landmines.

[0880] Step 3:

[0881] The server sends the analysis results to the device. It generates coordinate data for the identified mines and sends it to the device. It sends the analysis results using the send_results(analysis results, device ID) function.

[0882] Step 4:

[0883] The device receives the analysis results. The device visually displays the received coordinate data. For example, it marks specific coordinates on a map to show the user where the mines are buried. This is done using the display_results(analysis results) function.

[0884] Step 5:

[0885] The server collects information on past mine placements and accidents and stores it in a database. It retrieves information from historical databases and various reports and stores it in the database using the collect_data(data source) function. For example, it stores it in a variable called "historical_data."

[0886] Step 6:

[0887] Organize and consolidate the data collected by the server. Normalize and clean the data to make it analyzable by the AI ​​model. Use the normalize_data(collected_data) function.

[0888] Step 7:

[0889] The server uses an AI model to predict mine locations. The organized data is input into the AI ​​model to predict unknown mine locations. The predict_mine_locations(model, organized data) function is executed to obtain the predicted results.

[0890] Step 8:

[0891] The server provides the prediction results to the terminal. The prediction results are generated so that they can be displayed in dashboard format and sent to the terminal. The dashboard is generated using the generate_dashboard(prediction result) function.

[0892] Step 9:

[0893] The device displays the forecast results to the user, allowing the user to see the results in a visual format (e.g., a graph or map). This is done using the display_dashboard(forecastresult) function.

[0894] Step 10:

[0895] The server creates a mine removal plan. It creates an efficient removal plan based on the prediction results and the identified mine locations. It uses the create_removal_plan(identified location, prediction result) function.

[0896] Step 11:

[0897] The terminal presents the work plan to the user. The details of the work plan are displayed and the user confirms and approves it. The display_plan(plan) function is used.

[0898] Step 12:

[0899] The user checks and approves the work plan. The plan presented on the terminal is checked and determined to be feasible. After approval, the server is notified. The approve_plan() function is used.

[0900] Step 13:

[0901] The automated robot begins work based on the work plan received from the server. It follows GPS information to move to the designated mine site and carry out the removal work. The execute_removal_task(robot, plan) function is used.

[0902] Step 14:

[0903] The server monitors the work status of the automated robot in real time. It receives feedback data from the robot and monitors its progress. It uses the monitor_progress(robot) function.

[0904] These steps allow the system to safely and efficiently clear mines.

[0905] Example 1

[0906] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0907] Landmines are buried widely around the world as remnants of war and conflict, and their removal is extremely dangerous and time-consuming, making it a major social problem in many regions. Conventional mine removal methods rely on manual labor, which entails significant risks and costs. Furthermore, it is often difficult to identify or predict where mines will be buried, making the work inefficient. This invention aims to solve these problems by using artificial intelligence technology to remove landmines safely and efficiently.

[0908] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0909] In this invention, the server includes means for acquiring satellite images or unmanned aerial vehicle images, means for analyzing the acquired images to identify buried mine locations, means for transmitting the analysis results to a terminal and visually displaying them, means for storing information on past buried mines and accident information in a database, means for predicting buried mine locations based on the stored information, means for providing the prediction results to the terminal so that the user can check them, means for formulating a removal work plan based on the buried mine locations and the prediction results, an automated work device for executing the formulated work plan, and means for monitoring the work status of the automated work device in real time. This makes it possible to automate the process from identifying and predicting the location of mines to the removal work as a series of steps, thereby significantly improving the safety and efficiency of work.

[0910] "Satellite imagery" refers to image data of the Earth's surface taken from an artificial satellite in Earth's orbit.

[0911] "Unmanned aerial vehicle imagery" refers to image data of the earth's surface taken from an unmanned aerial vehicle such as a drone.

[0912] "Means of acquisition" refers to the equipment, software, and protocols necessary to receive image data transmitted from satellites and unmanned aerial vehicles.

[0913] "Means of analysis" refers to the algorithms, software, and hardware used to identify buried mine locations using acquired image data.

[0914] "Visual display means" refers to a device or application that provides the analysis results to the user in a visual format such as a map or graph.

[0915] "Means of storing information in a database" refers to the system or structure for managing and storing information on past mine burials and accidents.

[0916] "Predictive tools" refers to software and computational models that use artificial intelligence algorithms to predict new mine locations based on stored data.

[0917] "Means for providing and enabling users to check" refers to an interface or application that provides prediction results in a form that users can easily access and visually check.

[0918] "Planning tools for clearance operations" refers to algorithms and software that plan efficient and safe clearance procedures based on identified mine locations and predicted outcomes.

[0919] "Automated work equipment" refers to robots and mechanical devices that automatically remove mines according to a work plan.

[0920] "Monitoring means" means systems or software that monitor the progress or status of automated work equipment in real time and intervene or adjust as necessary.

[0921] This invention relates to a system for safely and efficiently removing landmines by utilizing artificial intelligence technology. Specific embodiments for carrying out this invention will be described below.

[0922] Image Recognition

[0923] First, the server acquires satellite and drone imagery, which includes, for example, downloading images uploaded to Amazon S3 using the AWS SDK.

[0924] The server then analyzes the acquired image data. Using Python's OpenCV and TensorFlow, it uses an object detection algorithm (such as YOLO) to extract the characteristics of the mines and identify their locations. Once the analysis is complete, it generates the results as coordinate data.

[0925] The server sends the analysis results to the device in JSON format, using WebSocket or REST API for communication. Specifically, the JSON data contains the coordinate information of the analyzed mines.

[0926] The device visually displays the received analysis results and uses the JavaScript Google Maps API to mark the coordinates of detected mines on a map and notify the user, allowing the user to intuitively confirm the location of specific mines.

[0927] Information gathering and prediction

[0928] The server then collects information on past mine placements and accidents and stores it in a database, which is managed using an RDBMS such as MySQL or PostgreSQL, and retrieves information from historical databases and local reports.

[0929] The server predicts mine locations based on the stored information. To do this, it uses machine learning libraries (such as Scikit-learn or TensorFlow) to build a predictive model and analyze past data to estimate unknown mine locations.

[0930] The predicted results are sent from the server to the device and displayed on a dashboard for the user to review. The dashboard, built using React and Vue.js, provides coordinate information for newly predicted mines, allowing users to easily view them.

[0931] Removal work planning and execution

[0932] Based on the prediction results and the identified mine locations, the server creates a removal plan, which includes efficient and safe work procedures, the equipment to be used, and a time schedule.

[0933] The terminal presents the work plan to the user. The user checks the presented plan and presses the approval button if there are no problems. The user's approval is recorded as a log on the server.

[0934] The approved work plan is then sent from the server to the automated work device, using MQTT or other real-time messaging protocols.

[0935] Based on the received plan, the automated device will begin mine clearance work in the designated area. It will use GPS to locate specific mines and move automatically to carry out the clearance work. The work status will be reported to the server in real time.

[0936] The server monitors the operation status of the automated work equipment and tracks its progress. The acquired data is updated to a database and displayed on a dashboard, allowing users to see the progress of the work in real time.

[0937] Specific examples

[0938] For example, if a mine clearance operation is envisaged in a certain area, the following steps would be taken:

[0939] 1. The server receives the image data sent from the drone.

[0940] 2. The server performs image analysis and detects a mine at a specific coordinate, for example, (10.1234, 20.5678).

[0941] 3. The device displays the analysis results and provides coordinate information to the user.

[0942] 4. The server collects and organizes past mine placement data and accident information, and predicts where mines will be placed.

[0943] 5. The forecast results are displayed as a dashboard, providing information about the newly predicted coordinates (10.2345, 20.6789).

[0944] 6. The server creates a removal plan based on the identified mine locations and the prediction results.

[0945] 7. The terminal presents the work plan to the user and obtains confirmation.

[0946] 8. Automated work equipment will head to the site according to the work plan and remove the mines.

[0947] 9. The server monitors the work in real time and tracks the progress.

[0948] Prompt Sentence Examples

[0949] Example prompts to input to a generative AI model:

[0950] "Please analyze satellite images to identify buried mine sites."

[0951] "Build a predictive model for mine placement using historical data."

[0952] "Please draw up a mine clearance operation plan for the specified coordinates."

[0953] This series of processes ensures that mine clearance work is carried out safely and efficiently, and that many people are protected from the dangers of landmines.

[0954] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0955] Program processing flow

[0956] Step 1:

[0957] The server receives image data from satellites and unmanned aerial vehicles. Specifically, it downloads the image data using HTTP or FTP protocols. The input is the image file, which becomes the data for processing in the next step.

[0958] Step 2:

[0959] The server analyzes the received image data and applies an object detection algorithm (such as YOLO) using Python's OpenCV or TensorFlow. The input for this analysis is the acquired image file, and the output is coordinate data indicating the location of buried mines. Specifically, the server extracts image features and identifies the parts that can be identified as mines using a trained model.

[0960] Step 3:

[0961] The server sends the analysis results, which identify the buried mine locations, to the device. Coordinate data is sent in JSON format using WebSocket or REST API. The input is the coordinate data from the analysis results, and the output is the JSON data transferred to the device. For example, coordinates are sent in the format {"latitude": 10.1234, "longitude": 20.5678}.

[0962] Step 4:

[0963] The device receives the analysis results sent from the server and displays them visually. It uses the Google Maps API in JavaScript to mark the identified coordinates on a map. The input is coordinate data in JSON format, and the output is a map that is displayed to the user. The device draws icons on the map to indicate the location of mines, allowing the user to intuitively identify them.

[0964] Step 5:

[0965] The server collects information on past mine placements and accidents and stores it in a database. Sources of information include historical databases and local reports, and the data is managed using MySQL or PostgreSQL. The input is data from each source, and the output is information stored in the database. Specifically, the server retrieves information through an API, standardizes it, and inserts it into the database.

[0966] Step 6:

[0967] The server predicts mine locations based on the stored information. It uses a machine learning library (such as Scikit-learn or TensorFlow) to learn from past data and build a predictive model. The input is past data stored in the database, and the output is the coordinates of predicted mine locations. The server uses the trained model to analyze the input data and predict unknown mine locations.

[0968] Step 7:

[0969] The server sends the prediction results to the device, where the user can view them on a dashboard. The prediction results are sent to the device in JSON format. The input is coordinate data from the prediction model, and the output is coordinate information displayed on the dashboard. The device uses front-end libraries such as React and Vue.js to display the information in a visually easy-to-understand format.

[0970] Step 8:

[0971] The server creates a removal operation plan based on the identified mine locations and prediction results. The input is mine coordinate data and prediction data, and the output is a detailed operation plan. The server calculates efficient and safe operation procedures and generates a timeline for each step and a list of required equipment.

[0972] Step 9:

[0973] The terminal presents the work plan to the user, who then reviews and approves the plan. The input is the work plan sent from the server, and the output is the user's approval data. The terminal displays the work plan to the user as an interactive guide and provides an approval button.

[0974] Step 10:

[0975] The server sends the approved work plan to the automated work device. It transmits data in real time using the MQTT protocol. The input is the work plan approved by the user, and the output is the data sent to the automated work device. The server starts work according to the instructions.

[0976] Step 11:

[0977] The automated work device carries out mine removal work based on the received work plan. It uses GPS to move to the designated area and remove the mines. The input is the work plan data, and the output is a report data on the completion of the removal work. The automated work device periodically sends its progress to the server.

[0978] Step 12:

[0979] The server monitors the work status of the automated work equipment in real time. The server receives the transmitted progress data and tracks the work progress. The input is feedback data from the automated work equipment, and the output is updated progress data. The server visually displays the progress to the user and issues warnings if any problems occur.

[0980] (Application example 1)

[0981] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0982] Conventional autonomous vehicle systems lack sufficient means to detect obstacles and accidents on the road in real time and provide safe detour routes. This has resulted in increased operational interruptions and dangers due to accidents and obstacles, resulting in problems with operational efficiency and safety. In addition, existing mine clearance systems are limited to mine detection and clearance and lack the ability to respond to dynamically changing situations.

[0983] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0984] In this invention, the server includes means for acquiring satellite or drone images, means for analyzing the acquired images to identify buried mine locations, means for transmitting the analysis results to a terminal and visually displaying them, means for storing information on past mine placements and accident information in a database, means for predicting mine locations based on the stored information, means for providing the prediction results to the terminal so that a user can check them, means for formulating a mine removal operation plan based on the buried mine locations and the prediction results, an automated machine for executing the operation plan, means for monitoring the operation status of the automated machine in real time, and means for detecting hazardous objects while the automated machine is operating and providing an alternative route. This enables automated vehicles to detect hazardous objects on roads in real time and provide safe detour routes, thereby improving safety and operating efficiency.

[0985] "Satellite imagery" refers to image data obtained by satellites photographing the Earth's surface.

[0986] "Drone images" are image data taken from the air by a drone, a small unmanned aerial vehicle.

[0987] "Analysis" is the process of applying specific algorithms to acquired image data to extract information and identify objects of interest.

[0988] "Mine site" refers to the location where a landmine is buried underground.

[0989] A "terminal" is an electronic device that a user uses to receive and visually view information.

[0990] A "database" is a computer system for organizing and storing data, allowing for efficient searching and updating of data.

[0991] "Prediction" refers to predicting future situations or events based on past data and current information.

[0992] A "clearance operation plan" is a plan that outlines the methods and procedures for safely and efficiently removing buried mine sites.

[0993] An "automated machine" refers to a robot or mechanical device that performs work automatically according to programmed instructions.

[0994] "Monitoring" refers to watching the progress of a system or task in real time and making adjustments as needed.

[0995] "Hazardous material" means any substance or condition that may cause injury or danger during operation or work.

[0996] "Alternate Route" means a proposed alternative route to avoid an accident or hazard.

[0997] MODE FOR CARRYING OUT THE INVENTION

[0998] The present invention provides a system for an autonomous vehicle to detect dangerous objects on the road and provide a safe detour route. The system comprises the following means.

[0999] First, the server acquires satellite or drone images, which are then transmitted in real time and aggregated on the server, where they are analyzed using image analysis algorithms such as OpenCV and Keras, utilizing high-performance computing resources.

[1000] As a result of the analysis, the locations of buried mines and other hazards are identified. This identified information is sent to the device and visually displayed to the user. For example, the coordinate information indicated by the analysis results is marked on a map to inform the user.

[1001] The server also stores information on past mine placements and accidents in a database. This information is used to predict mines and other hazards. An AI model analyzes this data and generates predictions. These predictions are also displayed on the device as a dashboard for users to review.

[1002] Based on the location of buried mines and the prediction results, the server creates a removal work plan. This work plan includes efficient work procedures and time schedules. The created work plan is displayed on the terminal and the user confirms and approves it. The approved plan is then sent to the automated machine.

[1003] The automated machines begin clearance operations in designated areas based on a work plan. They use GPS information to navigate automatically and safely remove specific mines and hazardous materials. Additionally, the automated machines have the ability to detect new hazards while in operation and can provide alternative routes based on detected hazards.

[1004] The server also monitors the work status of the automated machines in real time, allowing it to track the progress of the work and update the data to ensure that the work is progressing smoothly.

[1005] Specific examples

[1006] For example, when clearing landmines in an area, drones can take pictures of accidents on the road and send the images to a server, which analyzes the images and detects that an accident has occurred. This information is then sent to the autonomous vehicle's system, which then provides a safe detour route.

[1007] Prompt Sentence Examples

[1008] "As an autonomous vehicle travels from Tokyo to Los Angeles, a drone detects an accident on the road. Use the image analysis system to confirm the existence of the accident, use the Google Maps API to obtain a safe detour route, and provide that information to the autonomous vehicle."

[1009] With the introduction of this system, autonomous vehicles will be able to monitor road conditions in real time and quickly detect hazards and accidents, improving safety and operational efficiency.

[1010] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1011] Step 1:

[1012] The server receives image data transmitted from a satellite or drone. As input, the image data from the satellite or drone is passed to the server. As output, the raw data is obtained and stored in the server.

[1013] Step 2:

[1014] The server analyzes the image data it receives. It uses OpenCV and Keras as its analysis algorithms to identify landmines and hazardous objects in the image. The image data stored on the server is used as input. The output is the coordinate data of the identified landmines and hazardous objects.

[1015] Step 3:

[1016] The analysis results are sent to the terminal and displayed for the user to visually confirm. As input, the identified coordinate data is sent from the server to the terminal. As output, the coordinate information is displayed marked on a map on the terminal screen.

[1017] Step 4:

[1018] The server stores the past mine laying information and accident information in the database. As input, the past mine laying data and accident information are taken into the database. As output, the information storage is completed.

[1019] Step 5:

[1020] The server analyzes the stored information and predicts mine locations and hazardous materials. It uses a generative AI model to make predictions based on past data. The input is the stored data in the database. The output is the predicted coordinate data.

[1021] Step 6:

[1022] The prediction results are provided to the terminal and displayed for the user to check. As input, the predicted coordinate data is sent to the terminal. As output, the prediction information is displayed on the dashboard.

[1023] Step 7:

[1024] The server creates a removal operation plan based on the location of the mines and the prediction results. The input is the identified and predicted coordinate data. The output is a removal operation plan that includes the work procedure and time schedule.

[1025] Step 8:

[1026] The work plan is presented to the terminal and the user confirms and approves it. The work plan is sent to the terminal as input. The user's confirmation and approval is sent from the terminal to the server as output.

[1027] Step 9:

[1028] The automated machine starts the removal work in the designated area based on the received work plan. It uses GPS information to move automatically and safely remove the mines. The work plan and GPS information are used as inputs. The progress of the removal work is sent from the work site to the server as output.

[1029] Step 10:

[1030] The server monitors the work status of the automated machine in real time and tracks the progress. As input, work status data sent from the work site is taken into the server. As output, the progress status is updated in real time and displayed on the monitoring screen.

[1031] Step 11:

[1032] The automated machine detects new hazards while in operation and provides an alternative route. As input, sensor data acquired during operation is used. As output, an alternative route to avoid the hazard is provided to the automated machine.

[1033] This allows autonomous vehicle systems to monitor road conditions in real time, clear mines and provide safe detour routes.

[1034] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1035] This invention is a system for safely and efficiently removing landmines by using artificial intelligence technology and an emotion engine. Specific embodiments for carrying out the invention will be described below.

[1036] Image Recognition

[1037] First, the server receives image data from satellites or drones, and these images are sent to the server via a network and stored appropriately.

[1038] The server then uses the received image data to apply image analysis algorithms, using AI models to identify specific patterns and pinpoint the location of buried mines, generating coordinate data for the mines.

[1039] The analysis results are sent to the device, which receives this data and visually displays it, specifically marking specific coordinates on a map to show the user the location of the mines.

[1040] Information gathering and prediction

[1041] The server collects and stores information about past mine placements and accidents in a database, which is obtained from historical databases and local reports.

[1042] The server then organizes and consolidates the collected data, normalizing and cleaning it to make it suitable for analysis by AI models.

[1043] The server uses an AI model to predict where landmines are buried. The organized data is passed to the AI ​​model, which then predicts unknown landmine locations. The prediction results are generated in a dashboard format and provided to the device.

[1044] The device visually displays the prediction results to the user, specifically by showing the predicted locations using graphs or maps so that the user can easily check them.

[1045] Removal work planning and execution

[1046] The server then creates a removal plan based on the identified mine locations and the prediction results, generating a plan that includes efficient work procedures and a time schedule.

[1047] The work plan is presented on the terminal and an interface is provided for the user to review and approve. Once the user reviews and approves the plan, it is sent to the automated robot.

[1048] The automated robot will begin work in the designated area based on the received work plan, and will use GPS information to navigate automatically and remove mines.

[1049] The server monitors the work status of the automated robot in real time, constantly updating progress information to ensure that the work is progressing smoothly.

[1050] Emotion engine integration

[1051] Furthermore, the system is integrated with an emotion engine that recognizes the user's emotions. The server receives and analyzes the user's facial expression and voice data.

[1052] The server uses an emotion analysis algorithm to determine the user's emotional state, for example, if the user is feeling anxious, that emotional state is detected.

[1053] The device dynamically adjusts the interface based on the user's emotional state, for example simplifying the interface to present information more clearly when the user is feeling anxious.

[1054] Specific examples

[1055] As a concrete example, the procedure for carrying out mine clearance work in a certain area will be explained.

[1056] 1. The server receives the image data sent from the drone.

[1057] 2. The server performs image analysis and detects a mine at coordinates (10.1234, 20.5678).

[1058] 3. The device displays the analysis results on a map for the user.

[1059] 4. The server collects and organizes past mine data and predicts where mines are buried.

[1060] 5. The server generates the prediction results and provides them to the device as a dashboard.

[1061] 6. The device displays the prediction results for the user to confirm.

[1062] 7. The server creates a removal plan and displays it on the terminal.

[1063] 8. The user reviews and approves the work plan.

[1064] 9. The automated robot moves to the designated location and removes the mines.

[1065] 10. The server monitors the robot's work status in real time.

[1066] 11. The server analyzes the user's emotions and dynamically adjusts the interface.

[1067] This series of processes not only ensures safe and efficient mine clearance work, but also reduces the psychological burden on the user. This system protects many people from the dangers of landmines while also improving the efficiency of work.

[1068] The processing flow will be explained below.

[1069] Step 1:

[1070] The server receives image data from a satellite or drone. The image data is sent over the network, and the server saves the data in storage. Specifically, it calls the receive_image_data() function to retrieve the image. This data is saved in a variable called "image_data," for example.

[1071] Step 2:

[1072] The server analyzes the received image data. The acquired image data is input into the AI ​​model to identify the location of buried landmines. The AI ​​model contains a specific image recognition algorithm and executes the analyze_image(image_data) function, which identifies the coordinates of the buried landmines.

[1073] Step 3:

[1074] The server sends the analysis results to the device. It generates coordinate data for the identified mines and sends it to the device. It sends the analysis results using the send_results(analysis results, device ID) function.

[1075] Step 4:

[1076] The device receives the analysis results. The device visually displays the received coordinate data. For example, the display_results(analysis results) function is used to mark specific coordinates on a map to show the user where the mines are located.

[1077] Step 5:

[1078] The server collects information on past mine placements and accidents and stores it in a database. It retrieves information from historical databases and various reports and stores it in the database using the collect_data(data source) function. For example, it is stored in a variable called "historical_data."

[1079] Step 6:

[1080] Organize and consolidate the data collected by the server. Normalize and clean the data to make it analyzable by the AI ​​model. Use the normalize_data(collected_data) function.

[1081] Step 7:

[1082] The server uses an AI model to predict mine locations. The organized data is input into the AI ​​model to predict unknown mine locations. The predict_mine_locations(model, organized data) function is executed to obtain the predicted results.

[1083] Step 8:

[1084] The server provides the prediction results to the terminal. The prediction results are generated so that they can be displayed in dashboard format and sent to the terminal. The dashboard is generated using the generate_dashboard(prediction result) function.

[1085] Step 9:

[1086] The device displays the forecast results to the user, allowing the user to see the results in a visual format (e.g., a graph or map). This is done using the display_dashboard(forecastresult) function.

[1087] Step 10:

[1088] The server creates a mine removal plan. It creates an efficient removal plan based on the prediction results and the identified mine locations. It uses the create_removal_plan(identified location, prediction result) function.

[1089] Step 11:

[1090] The terminal presents the work plan to the user. The details of the work plan are displayed and the user confirms and approves it. The display_plan(plan) function is used.

[1091] Step 12:

[1092] The user checks and approves the work plan. The plan presented on the terminal is checked and determined to be feasible. After approval, the server is notified. The approve_plan() function is used.

[1093] Step 13:

[1094] The automated robot begins work based on the work plan received from the server. It follows GPS information to move to the designated mine site and carry out the removal work. The execute_removal_task(robot, plan) function is used.

[1095] Step 14:

[1096] The server monitors the work status of the automated robot in real time. It receives feedback data from the robot and monitors its progress. It uses the monitor_progress(robot) function.

[1097] Step 15:

[1098] The server receives the user's facial expression and voice data and performs emotion analysis. It analyzes the user's emotional state in real time and identifies the state using the emotion analysis engine. It uses the analyze_emotion(facial expression data, voice data) function.

[1099] Step 16:

[1100] The device dynamically adjusts the interface based on the user's emotional state. For example, if the user is feeling anxious, the interface can be simplified to make the information easier for the user to understand. This is done using the adjust_interface(emotional_state) function.

[1101] These steps allow the system to safely and efficiently remove mines and further optimize the user experience based on the user's emotional state. For example, in step 7 above, the system predicts that a new mine has been buried at coordinates (10.1234, 20.5678), and the removal process proceeds based on that prediction. The interface is simplified when the user feels anxious. In this way, the overall system further enhances safety and efficiency, and also provides psychological support.

[1102] Example 2

[1103] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1104] Conventional mine clearance systems have low analytical accuracy for accurately identifying the location of mines, limiting the efficiency of clearance work. Furthermore, they lack a method for reducing the psychological burden on users. Therefore, there is a need for a system that can clear mines quickly and safely while also reducing the psychological burden on users.

[1105] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for acquiring satellite images or drone images; means for analyzing the acquired images to identify buried mine locations; means for transmitting the analysis results to the terminal and visually displaying them; means for storing past mine placement information and accident information in a database; means for predicting mine locations based on the stored information; means for providing the prediction results to the terminal so that the user can confirm them; means for formulating a removal work plan based on the buried mine locations and the prediction results; an automatic robot for executing the formulated work plan; means for monitoring the work status of the automatic robot in real time; emotion engine means for receiving and analyzing facial expression data and voice data of the user; and means for dynamically adjusting the terminal interface based on the user's emotional state. This enables mine identification, prediction, and removal work to be performed quickly and safely while reducing the psychological burden on the user.

[1106] "Satellite imagery" refers to image data taken of a specific area on Earth by an artificial satellite.

[1107] "Drone images" are aerial image data taken by a camera mounted on a drone.

[1108] "Image analysis" is the technique of processing acquired image data to identify specific objects.

[1109] "Mine burial location" is information about the location where a mine is buried in the ground.

[1110] A "terminal" is a computing device through which a user can view information.

[1111] A "database" is a system that systematically stores and manages data.

[1112] "Prediction results" are estimated information derived using artificial intelligence algorithms.

[1113] A "clearance operation plan" is a specific procedure and schedule for efficiently clearing landmines.

[1114] An "automatic robot" is a mechanical device that operates autonomously based on set instructions to remove landmines.

[1115] "Real-time monitoring" is a technology that instantly monitors current progress.

[1116] An "emotion engine" is a technology that analyzes a user's facial expression data and voice data to identify their emotional state.

[1117] An "interface" is a screen or operating method that serves as a point of contact for the user and the system to exchange information with each other.

[1118] An "artificial intelligence algorithm" is a computational method that analyzes large amounts of data to derive patterns and predictions.

[1119] "Visually displaying" means displaying information on a screen in a format that is easy for the user to understand.

[1120] "Facial expression data" is information that captures the user's facial movements and expressions.

[1121] "Voice data" is information that records the user's utterances and voice characteristics.

[1122] "Dynamic adjustment" means changing the settings flexibly according to the situation at hand.

[1123] MODE FOR CARRYING OUT THE INVENTION

[1124] This invention is a system that combines artificial intelligence technology and an emotion engine to safely and efficiently remove landmines. Specific embodiments for carrying out this invention will be described below.

[1125] Image Recognition

[1126] First, the server receives image data sent from satellites or drones. This image data is then sent to the server via a network and stored in an appropriate storage. This process can use cloud storage such as Amazon S3.

[1127] The server then uses the received image data to apply image analysis algorithms, specifically using machine learning libraries such as TensorFlow to identify the locations of buried mines using a Convolutional Neural Network (CNN) model, and generates coordinate data for the areas where the mines are located.

[1128] The analysis results are sent to the device and displayed visually, and the device uses the Google Maps API or similar to mark specific coordinates on a map to show the user the location of the mines.

[1129] Information gathering and prediction

[1130] The server collects and stores information on past mine placements and accidents in a database. This information is obtained from historical databases and local reports. The database can be a database system such as PostgreSQL.

[1131] The server then normalizes and cleans the collected and stored data, converting it into a format suitable for AI analysis. This process uses data processing libraries such as Pandas.

[1132] The server inputs the organized data into an AI model to predict the location of buried mines, using a time-series prediction model such as a Long Short-Term Memory (LSTM) network. The prediction results are generated in the form of a dashboard and sent to the device.

[1133] The terminal visualizes the prediction results provided by the server, allowing users to easily check them. By using concrete diagrams and maps, users can intuitively understand the prediction results.

[1134] Removal work planning and execution

[1135] The server then creates a removal plan based on the identified mine locations and the predicted results. This plan includes efficient work procedures and time schedules. An optimization algorithm can be used to generate the most efficient removal plan.

[1136] The work plan is presented on the terminal and an interface is provided for the user to review and approve, and once the user approves the plan, it is sent to the automated robot.

[1137] The autonomous robot uses a GPS device to automatically navigate to the designated area and begin the clearance process. Specifically, the robot detects landmines and uses explosive ordnance disposal equipment to safely remove them.

[1138] The server monitors the work status of the automated robot in real time, and progress information is updated and displayed on the terminal.

[1139] Emotion engine integration

[1140] Furthermore, this system is integrated with an emotion engine that recognizes the user's emotions. The server receives and analyzes the user's facial expression and voice data. The analysis uses a voice recognition library and an expression analysis library.

[1141] The server uses an emotion analysis algorithm to identify the user's emotional state, for example, if the user is feeling anxious, that emotion is detected.

[1142] The device dynamically adjusts its interface based on the user's emotional state: when the user is feeling anxious, it simplifies the interface and provides more understandable information.

[1143] Examples of concrete examples and prompts

[1144] As a concrete example, the procedure for carrying out mine clearance work in a certain area is shown below.

[1145] 1. The server receives high-resolution image data sent from the drone and begins processing.

[1146] 2. The server uses a Convolutional Neural Network (CNN) to analyze the image and detect a landmine at coordinates (10.1234, 20.5678).

[1147] 3. The device uses the Google Maps API to mark the analysis results on a map in real time and display them to the user.

[1148] 4. The server collects and organizes landmine data from the past 10 years from the PostgreSQL database and inputs it into the AI ​​model.

[1149] 5. The server uses the LSTM model to generate a dashboard of predicted mine locations and send it to the device.

[1150] 6. The device displays the dashboard provided by the server, and the user checks the prediction results.

[1151] 7. The server uses an algorithm to create an optimal removal plan and displays it on the device.

[1152] 8. The user reviews the plan and approves it through the device interface.

[1153] 9. An automated robot uses a GPS device to navigate to a designated location and uses a robotic arm to dig up the mines.

[1154] 10. The server monitors the robot's operation in real time and displays the progress on the terminal.

[1155] 11. The server receives the user's facial expression data from the camera and detects feelings of anxiety.

[1156] 12. The device simplifies the interface and displays a reassuring message to the user.

[1157] These systems allow mine clearance work to be carried out quickly and safely, and also reduce the psychological burden on users.

[1158] Example prompt sentence:

[1159] "Please generate explanatory text for a mine removal system that detects buried mine locations from image data, makes predictions based on past mine data, and allows users to proceed with confirmation and removal work with confidence."

[1160] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1161] Step 1: Receiving image data

[1162] The server receives high-resolution image data transmitted from a satellite or drone. The input is the image data transmitted from the satellite or drone, and the output is the image data stored in the server's storage. Specifically, this can be done using cloud storage such as Amazon S3.

[1163] Step 2: Image analysis

[1164] The server uses the image data received by the server to identify buried landmine locations by applying a Convolutional Neural Network (CNN) model using machine learning libraries such as TensorFlow. The input is the image data stored in storage, and the output is the coordinate data of the area where the landmines are located. Specifically, the CNN model extracts image features and identifies the patterns of landmines.

[1165] Step 3: Viewing the analysis results

[1166] The server sends the results of the image analysis to the device. The device visually displays the analysis results using Google Maps API or similar. The input is the coordinate data sent from the server, and the output is the location of the mine marked on the map. Specifically, it marks specific coordinates on the map to show the user the location of the mine.

[1167] Step 4: Gather information

[1168] The server collects information on past mine placements and accidents from historical databases and local reports, and stores this information in a database. The input is mine information obtained from external data sources, and the output is information stored in the database. Specific operations use a database system such as PostgreSQL.

[1169] Step 5: Organize your data

[1170] The server normalizes and cleans the collected and stored data. The input is the stored minefield information, and the output is data formatted for AI analysis. Specifically, it uses data processing libraries such as Pandas to fill in missing values ​​and remove outliers.

[1171] Step 6: Predicting mine locations

[1172] The server inputs the organized data into an AI model to predict where landmines will be buried. The input is normalized landmine information, and the output is coordinate data of the predicted landmine locations. Specifically, a time series prediction model such as a Long Short-Term Memory (LSTM) network is used.

[1173] Step 7: View the prediction results

[1174] The server generates the prediction results in a dashboard format and sends them to the terminal. The terminal visualizes them so that the user can easily check them. The input is the prediction result data sent from the server, and the output is the prediction results displayed in graphs, maps, etc.

[1175] Step 8: Develop a removal plan

[1176] The server then creates a removal work plan based on the identified mine locations and the prediction results. The input is information on mine locations and the prediction results, and the output is a work plan. Specifically, an optimization algorithm is used to generate a plan that includes efficient work procedures and time schedules.

[1177] Step 9: Workplan Approval

[1178] The terminal presents the work plan to the user, who then reviews and approves the plan through the interface. The input is the work plan sent from the server, and the output is the plan approved by the user.

[1179] Step 10: Perform the removal work

[1180] The automated robot begins work in the designated area based on the work plan it receives. The input is the approved work plan, and the output is the number of mines removed and the progress of the work. Specifically, it uses GPS information to move autonomously and physically remove the mines.

[1181] Step 11: Monitoring the work

[1182] The server monitors the work status of the automated robot in real time. The input is progress information from the automated robot, and the output is progress status data updated in real time. Specifically, the data is displayed on the terminal as it is processed.

[1183] Step 12: Analyze emotional state

[1184] The server receives the user's facial expression and voice data and uses an emotion analysis algorithm to identify the user's emotional state. The input is data obtained from a camera or microphone, and the output is the analyzed emotional state. Specific operations use a voice recognition library and an emotion analysis library.

[1185] Step 13: Adjusting the Interface

[1186] The device dynamically adjusts the interface based on the user's emotional state. The input is the analyzed emotional state, and the output is an interface that corresponds to the user's emotional state. Specifically, if the user feels anxious, the interface is simplified to present information in a more understandable manner.

[1187] (Application example 2)

[1188] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1189] Conventional mine clearance systems are required to identify buried mines and remove them safely and efficiently, but there is a problem in that it is difficult for the entire system to perform the task quickly and accurately. Furthermore, they lack support functions that take into account the driver's emotional state, and do not improve safety or comfort while driving. Therefore, a system that can simultaneously remove mines and provide driver support was needed.

[1190] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring satellite images or drone images, means for analyzing the acquired images to identify buried mine locations, means for transmitting the analysis results to the terminal and visually displaying them, means for storing past mine placement information and accident information in a database, means for predicting mine locations based on the stored information, means for providing the prediction results to the terminal so that the user can check them, means for formulating a removal operation plan based on the buried mine locations and the prediction results, an automated robot for executing the formulated operation plan, means for monitoring the operation status of the automated robot in real time, means for analyzing the driver's emotions while driving and dynamically adjusting the interface, and means for providing hazard prediction information in accordance with the driver's emotional state. This not only enables efficient mine removal operations but also provides safe and comfortable assistance to the driver while driving.

[1191] A "server" is a computer system that sends, receives, and processes data over a network.

[1192] "Satellite imagery" refers to high-resolution images of the Earth's surface or objects taken by satellites.

[1193] "Drone imagery" refers to images of the earth's surface or objects taken by unmanned aerial vehicles (drones).

[1194] "Image analysis" is a technology that identifies and identifies objects or specific patterns based on acquired image data.

[1195] "Means for identifying buried landmine locations" refers to technology that uses image analysis to identify locations where landmines are buried.

[1196] A "terminal" is a device such as a computer or smartphone that displays data sent from a server and can be operated by a user.

[1197] "Past mine placement information" refers to data that indicates previously recorded mine placement locations and related information.

[1198] "Accident information" is data that records the location and circumstances of mine-related accidents.

[1199] A "database" is a data collection system that allows information to be efficiently stored, retrieved, and updated.

[1200] The "prediction results" are estimates of future mine locations based on past data.

[1201] A "clearance operation plan" is a plan that outlines specific procedures and schedules for safely clearing landmines based on the identified locations of the mines.

[1202] An "automatic robot" is a machine that autonomously carries out tasks as instructed.

[1203] "Real-time monitoring means" refers to a system for instantly monitoring and checking ongoing situations.

[1204] "Means for analyzing emotions" refers to technology for analyzing a user's facial expressions, voice, etc. to identify their emotional state.

[1205] "Means for dynamically adjusting the interface" refers to technology that automatically changes the display content and operation method according to the user's emotional state.

[1206] "Means for providing risk prediction information" refers to technology that predicts future risks based on past data and current conditions and notifies the user.

[1207] The embodiment of the present invention is based on a series of systems including a server, a terminal, an automatic robot, and an emotion analysis system. This system can analyze images in real time, identify the location of landmines, and efficiently remove them, while also grasping the emotional state of the driver and providing appropriate feedback.

[1208] The server receives image data from satellites and drones and performs image analysis based on this data. Specifically, the server uses a high-performance GPU and deep learning frameworks such as TensorFlow and Keras. This image analysis identifies specific patterns and locates buried mines. The coordinate data of identified mines is sent to the device and displayed visually.

[1209] Past mine placement and accident information is stored in a database. The server integrates this information and uses an AI model to predict where mines will be placed. The results of this prediction are also provided to the device, where users can check them. Predictions require cleansing and normalization of past data.

[1210] The server then creates a removal work plan based on the location of buried mines and the prediction results. This plan, which includes efficient work procedures and time schedules, is displayed on the terminal. Once the user confirms and approves the plan, work instructions are sent to the automated robot. The automated robot uses GPS information to automatically move to the designated area and remove the mines. The server also monitors the automated robot's work status in real time and immediately addresses any problems that arise.

[1211] The integration of an emotion engine is also a key element of this system. The server receives facial expression and voice data from the driver or operator via the device and performs emotion analysis. This analysis uses an emotion recognition algorithm, and if the user feels anxious or tired, the interface is dynamically adjusted. This allows the user to use information more easily and with peace of mind.

[1212] For example, a system can analyze live video feeds of a highway, detect obstacles ahead, and warn the driver. At the same time, if anxiety is detected from the driver's facial expression, the system simplifies the interface and activates reassuring voice guidance. In this way, both mine clearance and driver assistance can be carried out efficiently.

[1213] Example prompt sentence:

[1214] "Creating an application that detects obstacles ahead from live video footage on the highway, analyzes the driver's emotions, and provides appropriate feedback."

[1215] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1216] Step 1:

[1217] The server receives satellite or drone images. These images are sent to the server through a network and stored in a database. The input data is high-resolution image data, and the output is stored image data.

[1218] Step 2:

[1219] The server applies image analysis algorithms to the stored image data. Specifically, it uses deep learning frameworks such as TensorFlow and Keras to identify specific patterns and identify buried mine locations. The input data is the stored image data, and the output is the coordinate data of the mines.

[1220] Step 3:

[1221] The server sends coordinate data of the identified landmines to the terminal. The terminal visually displays the received data on a map and provides the user with information on the location of the landmines. The input data is the coordinate data of the landmines, and the output is the location information of the landmines marked on the map.

[1222] Step 4:

[1223] The server collects and organizes past mine laying and accident information into a database. The input data is past mine and accident information, and the output is an organized database.

[1224] Step 5:

[1225] The server uses an AI model to predict future mine locations based on the collected data. The input data is an organized database, and the output is predicted mine location data.

[1226] Step 6:

[1227] The server sends the prediction results to the terminal, which then visually displays the prediction results to the user. The input data is the predicted mine location data, and the output is the visually displayed prediction results.

[1228] Step 7:

[1229] The server then creates a removal plan based on the identified mine locations and prediction results. The plan includes efficient work procedures and time schedules. The input data is mine location information and prediction results, and the output is a detailed work plan.

[1230] Step 8:

[1231] The server sends the prepared work plan to the automated robot, which then uses GPS information to automatically move to the designated location and remove the mines based on the plan. The input data is the work plan, and the output is the progress of the mine removal work.

[1232] Step 9:

[1233] The server monitors the work status of the automated robot in real time and makes adjustments and instructions as necessary. The input data is progress data, and the output is the monitoring results and instructions.

[1234] Step 10:

[1235] The server receives the driver's facial expression and voice data through the terminal and performs emotion analysis. The input data is facial expression data and voice data, and the output is analyzed emotional state data.

[1236] Step 11:

[1237] The server dynamically adjusts the device interface based on the analyzed emotional state and provides risk prediction information according to the driver's emotional state. The input data are emotional state data and past prediction data, and the output is the dynamically adjusted interface and the provided risk prediction information.

[1238] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1239] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1240] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1241] [Fourth embodiment]

[1242] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1243] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1244] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1245] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1246] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1247] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1248] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1249] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1250] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1251] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1252] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1253] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1254] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1255] This invention relates to a system for safely and efficiently removing landmines using artificial intelligence technology. Specific embodiments for carrying out the invention will be described below.

[1256] Image Recognition

[1257] First, the server receives satellite or drone images, which are then sent to the server via a network.

[1258] The server then uses an image analysis algorithm to analyze the received image data. The algorithm identifies specific patterns and identifies the location of buried mines. The analysis results are output as coordinate data indicating the location of buried mines.

[1259] The coordinate data from the analysis is sent to the device, which receives it and visually displays it. Specifically, it marks specific coordinates on a map to show the user where the mines are buried.

[1260] Information gathering and prediction

[1261] The server collects and stores in a database information on past mine placements and accidents, which is obtained from historical databases and local reports.

[1262] Based on the stored information, the server makes predictions about mine locations. An AI model analyzes this data and predicts unknown mine locations. The predictions are then provided to the device and displayed as a dashboard for the user to access.

[1263] Removal work planning and execution

[1264] Based on the identified mine locations and prediction results, the server creates a removal operation plan, which includes efficient work procedures and time schedules.

[1265] The work plan is presented to the terminal and reviewed and approved by the user. Once approved, the plan is sent to the automated robot.

[1266] The automated robot will then begin clearing operations in the designated area based on the received work plan, and will use GPS information to navigate automatically and safely clear specific mines.

[1267] Finally, the server monitors the work of the automated robot in real time, tracking its progress and updating the data to ensure that the work is going smoothly.

[1268] Specific examples

[1269] For example, when carrying out mine clearance work in a certain area, the following specific steps are taken:

[1270] 1. The server receives the image data sent from the drone.

[1271] 2. The server performs image analysis and finds a mine at a specific coordinate, for example, (10.1234, 20.5678).

[1272] 3. The device displays the analysis results and provides coordinate information to the user.

[1273] 4. The server collects and organizes past mine placement data and accident information, and predicts where mines will be placed.

[1274] 5. The forecast results are displayed as a dashboard, providing information, for example, about the newly predicted coordinates (10.2345, 20.6789).

[1275] 6. The server will create a removal plan based on the identified mine locations and the prediction results.

[1276] 7. The terminal presents the work plan to the user and obtains confirmation.

[1277] 8. Autonomous robots will follow a work plan and head to the site to remove the mines.

[1278] 9. The server monitors the work in real time and tracks the progress.

[1279] This series of processes ensures that mine clearance work is carried out safely and efficiently, and the system protects many people from the dangers of landmines and minimizes damage.

[1280] The processing flow will be explained below.

[1281] Step 1:

[1282] The server receives image data from a satellite or drone. The image data is sent over the network, and the server saves the data in storage. Specifically, it calls the receive_image_data() function to retrieve the image. This data is saved in a variable called "image_data," for example.

[1283] Step 2:

[1284] The server analyzes the received image data. The acquired image data is input into the AI ​​model to identify the location of buried landmines. The AI ​​model contains a specific image recognition algorithm and executes the analyze_image(image_data) function, which identifies the coordinates of the buried landmines.

[1285] Step 3:

[1286] The server sends the analysis results to the device. It generates coordinate data for the identified mines and sends it to the device. It sends the analysis results using the send_results(analysis results, device ID) function.

[1287] Step 4:

[1288] The device receives the analysis results. The device visually displays the received coordinate data. For example, it marks specific coordinates on a map to show the user where the mines are buried. This is done using the display_results(analysis results) function.

[1289] Step 5:

[1290] The server collects information on past mine placements and accidents and stores it in a database. It retrieves information from historical databases and various reports and stores it in the database using the collect_data(data source) function. For example, it stores it in a variable called "historical_data."

[1291] Step 6:

[1292] Organize and consolidate the data collected by the server. Normalize and clean the data to make it analyzable by the AI ​​model. Use the normalize_data(collected_data) function.

[1293] Step 7:

[1294] The server uses an AI model to predict mine locations. The organized data is input into the AI ​​model to predict unknown mine locations. The predict_mine_locations(model, organized data) function is executed to obtain the predicted results.

[1295] Step 8:

[1296] The server provides the prediction results to the terminal. The prediction results are generated so that they can be displayed in dashboard format and sent to the terminal. The dashboard is generated using the generate_dashboard(prediction result) function.

[1297] Step 9:

[1298] The device displays the forecast results to the user, allowing the user to see the results in a visual format (e.g., a graph or map). This is done using the display_dashboard(forecastresult) function.

[1299] Step 10:

[1300] The server creates a mine removal plan. It creates an efficient removal plan based on the prediction results and the identified mine locations. It uses the create_removal_plan(identified location, prediction result) function.

[1301] Step 11:

[1302] The terminal presents the work plan to the user. The details of the work plan are displayed and the user confirms and approves it. The display_plan(plan) function is used.

[1303] Step 12:

[1304] The user checks and approves the work plan. The plan presented on the terminal is checked and determined to be feasible. After approval, the server is notified. The approve_plan() function is used.

[1305] Step 13:

[1306] The automated robot begins work based on the work plan received from the server. It follows GPS information to move to the designated mine site and carry out the removal work. The execute_removal_task(robot, plan) function is used.

[1307] Step 14:

[1308] The server monitors the work status of the automated robot in real time. It receives feedback data from the robot and monitors its progress. It uses the monitor_progress(robot) function.

[1309] These steps allow the system to safely and efficiently clear mines.

[1310] Example 1

[1311] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1312] Landmines are buried widely around the world as remnants of war and conflict, and their removal is extremely dangerous and time-consuming, making it a major social problem in many regions. Conventional mine removal methods rely on manual labor, which entails significant risks and costs. Furthermore, it is often difficult to identify or predict where mines will be buried, making the work inefficient. This invention aims to solve these problems by using artificial intelligence technology to remove landmines safely and efficiently.

[1313] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1314] In this invention, the server includes means for acquiring satellite images or unmanned aerial vehicle images, means for analyzing the acquired images to identify buried mine locations, means for transmitting the analysis results to a terminal and visually displaying them, means for storing information on past buried mines and accident information in a database, means for predicting buried mine locations based on the stored information, means for providing the prediction results to the terminal so that the user can check them, means for formulating a removal work plan based on the buried mine locations and the prediction results, an automated work device for executing the formulated work plan, and means for monitoring the work status of the automated work device in real time. This makes it possible to automate the process from identifying and predicting the location of mines to the removal work as a series of steps, thereby significantly improving the safety and efficiency of work.

[1315] "Satellite imagery" refers to image data of the Earth's surface taken from an artificial satellite in Earth's orbit.

[1316] "Unmanned aerial vehicle imagery" refers to image data of the earth's surface taken from an unmanned aerial vehicle such as a drone.

[1317] "Means of acquisition" refers to the equipment, software, and protocols necessary to receive image data transmitted from satellites and unmanned aerial vehicles.

[1318] "Means of analysis" refers to the algorithms, software, and hardware used to identify buried mine locations using acquired image data.

[1319] "Visual display means" refers to a device or application that provides the analysis results to the user in a visual format such as a map or graph.

[1320] "Means of storing information in a database" refers to the system or structure for managing and storing information on past mine burials and accidents.

[1321] "Predictive tools" refers to software and computational models that use artificial intelligence algorithms to predict new mine locations based on stored data.

[1322] "Means for providing and enabling users to check" refers to an interface or application that provides prediction results in a form that users can easily access and visually check.

[1323] "Planning tools for clearance operations" refers to algorithms and software that plan efficient and safe clearance procedures based on identified mine locations and predicted outcomes.

[1324] "Automated work equipment" refers to robots and mechanical devices that automatically remove mines according to a work plan.

[1325] "Monitoring means" means systems or software that monitor the progress or status of automated work equipment in real time and intervene or adjust as necessary.

[1326] This invention relates to a system for safely and efficiently removing landmines by utilizing artificial intelligence technology. Specific embodiments for carrying out this invention will be described below.

[1327] Image Recognition

[1328] First, the server acquires satellite and drone imagery, which includes, for example, downloading images uploaded to Amazon S3 using the AWS SDK.

[1329] The server then analyzes the acquired image data. Using Python's OpenCV and TensorFlow, it uses an object detection algorithm (such as YOLO) to extract the characteristics of the mines and identify their locations. Once the analysis is complete, it generates the results as coordinate data.

[1330] The server sends the analysis results to the device in JSON format, using WebSocket or REST API for communication. Specifically, the JSON data contains the coordinate information of the analyzed mines.

[1331] The device visually displays the received analysis results and uses the JavaScript Google Maps API to mark the coordinates of detected mines on a map and notify the user, allowing the user to intuitively confirm the location of specific mines.

[1332] Information gathering and prediction

[1333] The server then collects information on past mine placements and accidents and stores it in a database, which is managed using an RDBMS such as MySQL or PostgreSQL, and retrieves information from historical databases and local reports.

[1334] The server predicts mine locations based on the stored information. To do this, it uses machine learning libraries (such as Scikit-learn or TensorFlow) to build a predictive model and analyze past data to estimate unknown mine locations.

[1335] The predicted results are sent from the server to the device and displayed on a dashboard for the user to review. The dashboard, built using React and Vue.js, provides coordinate information for newly predicted mines, allowing users to easily view them.

[1336] Removal work planning and execution

[1337] Based on the prediction results and the identified mine locations, the server creates a removal plan, which includes efficient and safe work procedures, the equipment to be used, and a time schedule.

[1338] The terminal presents the work plan to the user. The user checks the presented plan and presses the approval button if there are no problems. The user's approval is recorded as a log on the server.

[1339] The approved work plan is then sent from the server to the automated work device, using MQTT or other real-time messaging protocols.

[1340] Based on the received plan, the automated device will begin mine clearance work in the designated area. It will use GPS to locate specific mines and move automatically to carry out the clearance work. The work status will be reported to the server in real time.

[1341] The server monitors the operation status of the automated work equipment and tracks its progress. The acquired data is updated to a database and displayed on a dashboard, allowing users to see the progress of the work in real time.

[1342] Specific examples

[1343] For example, if a mine clearance operation is envisaged in a certain area, the following steps would be taken:

[1344] 1. The server receives the image data sent from the drone.

[1345] 2. The server performs image analysis and detects a mine at a specific coordinate, for example, (10.1234, 20.5678).

[1346] 3. The device displays the analysis results and provides coordinate information to the user.

[1347] 4. The server collects and organizes past mine placement data and accident information, and predicts where mines will be placed.

[1348] 5. The forecast results are displayed as a dashboard, providing information about the newly predicted coordinates (10.2345, 20.6789).

[1349] 6. The server creates a removal plan based on the identified mine locations and the prediction results.

[1350] 7. The terminal presents the work plan to the user and obtains confirmation.

[1351] 8. Automated work equipment will head to the site according to the work plan and remove the mines.

[1352] 9. The server monitors the work in real time and tracks the progress.

[1353] Prompt Sentence Examples

[1354] Example prompts to input to a generative AI model:

[1355] "Please analyze satellite images to identify buried mine sites."

[1356] "Build a predictive model for mine placement using historical data."

[1357] "Please draw up a mine clearance operation plan for the specified coordinates."

[1358] This series of processes ensures that mine clearance work is carried out safely and efficiently, and that many people are protected from the dangers of landmines.

[1359] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1360] Program processing flow

[1361] Step 1:

[1362] The server receives image data from satellites and unmanned aerial vehicles. Specifically, it downloads the image data using HTTP or FTP protocols. The input is the image file, which becomes the data for processing in the next step.

[1363] Step 2:

[1364] The server analyzes the received image data and applies an object detection algorithm (such as YOLO) using Python's OpenCV or TensorFlow. The input for this analysis is the acquired image file, and the output is coordinate data indicating the location of buried mines. Specifically, the server extracts image features and identifies the parts that can be identified as mines using a trained model.

[1365] Step 3:

[1366] The server sends the analysis results, which identify the buried mine locations, to the device. Coordinate data is sent in JSON format using WebSocket or REST API. The input is the coordinate data from the analysis results, and the output is the JSON data transferred to the device. For example, coordinates are sent in the format {"latitude": 10.1234, "longitude": 20.5678}.

[1367] Step 4:

[1368] The device receives the analysis results sent from the server and displays them visually. It uses the Google Maps API in JavaScript to mark the identified coordinates on a map. The input is coordinate data in JSON format, and the output is a map that is displayed to the user. The device draws icons on the map to indicate the location of mines, allowing the user to intuitively identify them.

[1369] Step 5:

[1370] The server collects information on past mine placements and accidents and stores it in a database. Sources of information include historical databases and local reports, and the data is managed using MySQL or PostgreSQL. The input is data from each source, and the output is information stored in the database. Specifically, the server retrieves information through an API, standardizes it, and inserts it into the database.

[1371] Step 6:

[1372] The server predicts mine locations based on the stored information. It uses a machine learning library (such as Scikit-learn or TensorFlow) to learn from past data and build a predictive model. The input is past data stored in the database, and the output is the coordinates of predicted mine locations. The server uses the trained model to analyze the input data and predict unknown mine locations.

[1373] Step 7:

[1374] The server sends the prediction results to the device, where the user can view them on a dashboard. The prediction results are sent to the device in JSON format. The input is coordinate data from the prediction model, and the output is coordinate information displayed on the dashboard. The device uses front-end libraries such as React and Vue.js to display the information in a visually easy-to-understand format.

[1375] Step 8:

[1376] The server creates a removal operation plan based on the identified mine locations and prediction results. The input is mine coordinate data and prediction data, and the output is a detailed operation plan. The server calculates efficient and safe operation procedures and generates a timeline for each step and a list of required equipment.

[1377] Step 9:

[1378] The terminal presents the work plan to the user, who then reviews and approves the plan. The input is the work plan sent from the server, and the output is the user's approval data. The terminal displays the work plan to the user as an interactive guide and provides an approval button.

[1379] Step 10:

[1380] The server sends the approved work plan to the automated work device. It transmits data in real time using the MQTT protocol. The input is the work plan approved by the user, and the output is the data sent to the automated work device. The server starts work according to the instructions.

[1381] Step 11:

[1382] The automated work device carries out mine removal work based on the received work plan. It uses GPS to move to the designated area and remove the mines. The input is the work plan data, and the output is a report data on the completion of the removal work. The automated work device periodically sends its progress to the server.

[1383] Step 12:

[1384] The server monitors the work status of the automated work equipment in real time. The server receives the transmitted progress data and tracks the work progress. The input is feedback data from the automated work equipment, and the output is updated progress data. The server visually displays the progress to the user and issues warnings if any problems occur.

[1385] (Application example 1)

[1386] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1387] Conventional autonomous vehicle systems lack sufficient means to detect obstacles and accidents on the road in real time and provide safe detour routes. This has resulted in increased operational interruptions and dangers due to accidents and obstacles, resulting in problems with operational efficiency and safety. In addition, existing mine clearance systems are limited to mine detection and clearance and lack the ability to respond to dynamically changing situations.

[1388] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1389] In this invention, the server includes means for acquiring satellite or drone images, means for analyzing the acquired images to identify buried mine locations, means for transmitting the analysis results to a terminal and visually displaying them, means for storing information on past mine placements and accident information in a database, means for predicting mine locations based on the stored information, means for providing the prediction results to the terminal so that a user can check them, means for formulating a mine removal operation plan based on the buried mine locations and the prediction results, an automated machine for executing the operation plan, means for monitoring the operation status of the automated machine in real time, and means for detecting hazardous objects while the automated machine is operating and providing an alternative route. This enables automated vehicles to detect hazardous objects on roads in real time and provide safe detour routes, thereby improving safety and operating efficiency.

[1390] "Satellite imagery" refers to image data obtained by satellites photographing the Earth's surface.

[1391] "Drone images" are image data taken from the air by a drone, a small unmanned aerial vehicle.

[1392] "Analysis" is the process of applying specific algorithms to acquired image data to extract information and identify objects of interest.

[1393] "Mine site" refers to the location where a landmine is buried underground.

[1394] A "terminal" is an electronic device that a user uses to receive and visually view information.

[1395] A "database" is a computer system for organizing and storing data, allowing for efficient searching and updating of data.

[1396] "Prediction" refers to predicting future situations or events based on past data and current information.

[1397] A "clearance operation plan" is a plan that outlines the methods and procedures for safely and efficiently removing buried mine sites.

[1398] An "automated machine" refers to a robot or mechanical device that performs work automatically according to programmed instructions.

[1399] "Monitoring" refers to watching the progress of a system or task in real time and making adjustments as needed.

[1400] "Hazardous material" means any substance or condition that may cause injury or danger during operation or work.

[1401] "Alternate Route" means a proposed alternative route to avoid an accident or hazard.

[1402] MODE FOR CARRYING OUT THE INVENTION

[1403] The present invention provides a system for an autonomous vehicle to detect dangerous objects on the road and provide a safe detour route. The system comprises the following means.

[1404] First, the server acquires satellite or drone images, which are then transmitted in real time and aggregated on the server, where they are analyzed using image analysis algorithms such as OpenCV and Keras, utilizing high-performance computing resources.

[1405] As a result of the analysis, the locations of buried mines and other hazards are identified. This identified information is sent to the device and visually displayed to the user. For example, the coordinate information indicated by the analysis results is marked on a map to inform the user.

[1406] The server also stores information on past mine placements and accidents in a database. This information is used to predict mines and other hazards. An AI model analyzes this data and generates predictions. These predictions are also displayed on the device as a dashboard for users to review.

[1407] Based on the location of buried mines and the prediction results, the server creates a removal work plan. This work plan includes efficient work procedures and time schedules. The created work plan is displayed on the terminal and the user confirms and approves it. The approved plan is then sent to the automated machine.

[1408] The automated machines begin clearance operations in designated areas based on a work plan. They use GPS information to navigate automatically and safely remove specific mines and hazardous materials. Additionally, the automated machines have the ability to detect new hazards while in operation and can provide alternative routes based on detected hazards.

[1409] The server also monitors the work status of the automated machines in real time, allowing it to track the progress of the work and update the data to ensure that the work is progressing smoothly.

[1410] Specific examples

[1411] For example, when clearing landmines in an area, drones can take pictures of accidents on the road and send the images to a server, which analyzes the images and detects that an accident has occurred. This information is then sent to the autonomous vehicle's system, which then provides a safe detour route.

[1412] Prompt Sentence Examples

[1413] "As an autonomous vehicle travels from Tokyo to Los Angeles, a drone detects an accident on the road. Use the image analysis system to confirm the existence of the accident, use the Google Maps API to obtain a safe detour route, and provide that information to the autonomous vehicle."

[1414] With the introduction of this system, autonomous vehicles will be able to monitor road conditions in real time and quickly detect hazards and accidents, improving safety and operational efficiency.

[1415] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1416] Step 1:

[1417] The server receives image data transmitted from a satellite or drone. As input, the image data from the satellite or drone is passed to the server. As output, the raw data is obtained and stored in the server.

[1418] Step 2:

[1419] The server analyzes the image data it receives. It uses OpenCV and Keras as its analysis algorithms to identify landmines and hazardous objects in the image. The image data stored on the server is used as input. The output is the coordinate data of the identified landmines and hazardous objects.

[1420] Step 3:

[1421] The analysis results are sent to the terminal and displayed for the user to visually confirm. As input, the identified coordinate data is sent from the server to the terminal. As output, the coordinate information is displayed marked on a map on the terminal screen.

[1422] Step 4:

[1423] The server stores the past mine laying information and accident information in the database. As input, the past mine laying data and accident information are taken into the database. As output, the information storage is completed.

[1424] Step 5:

[1425] The server analyzes the stored information and predicts mine locations and hazardous materials. It uses a generative AI model to make predictions based on past data. The input is the stored data in the database. The output is the predicted coordinate data.

[1426] Step 6:

[1427] The prediction results are provided to the terminal and displayed for the user to check. As input, the predicted coordinate data is sent to the terminal. As output, the prediction information is displayed on the dashboard.

[1428] Step 7:

[1429] The server creates a removal operation plan based on the location of the mines and the prediction results. The input is the identified and predicted coordinate data. The output is a removal operation plan that includes the work procedure and time schedule.

[1430] Step 8:

[1431] The work plan is presented to the terminal and the user confirms and approves it. The work plan is sent to the terminal as input. The user's confirmation and approval is sent from the terminal to the server as output.

[1432] Step 9:

[1433] The automated machine starts the removal work in the designated area based on the received work plan. It uses GPS information to move automatically and safely remove the mines. The work plan and GPS information are used as inputs. The progress of the removal work is sent from the work site to the server as output.

[1434] Step 10:

[1435] The server monitors the work status of the automated machine in real time and tracks the progress. As input, work status data sent from the work site is taken into the server. As output, the progress status is updated in real time and displayed on the monitoring screen.

[1436] Step 11:

[1437] The automated machine detects new hazards while in operation and provides an alternative route. As input, sensor data acquired during operation is used. As output, an alternative route to avoid the hazard is provided to the automated machine.

[1438] This allows autonomous vehicle systems to monitor road conditions in real time, clear mines and provide safe detour routes.

[1439] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1440] This invention is a system for safely and efficiently removing landmines by using artificial intelligence technology and an emotion engine. Specific embodiments for carrying out the invention will be described below.

[1441] Image Recognition

[1442] First, the server receives image data from satellites or drones, and these images are sent to the server via a network and stored appropriately.

[1443] The server then uses the received image data to apply image analysis algorithms, using AI models to identify specific patterns and pinpoint the location of buried mines, generating coordinate data for the mines.

[1444] The analysis results are sent to the device, which receives this data and visually displays it, specifically marking specific coordinates on a map to show the user the location of the mines.

[1445] Information gathering and prediction

[1446] The server collects and stores information about past mine placements and accidents in a database, which is obtained from historical databases and local reports.

[1447] The server then organizes and consolidates the collected data, normalizing and cleaning it to make it suitable for analysis by AI models.

[1448] The server uses an AI model to predict where landmines are buried. The organized data is passed to the AI ​​model, which then predicts unknown landmine locations. The prediction results are generated in a dashboard format and provided to the device.

[1449] The device visually displays the prediction results to the user, specifically by showing the predicted locations using graphs or maps so that the user can easily check them.

[1450] Removal work planning and execution

[1451] The server then creates a removal plan based on the identified mine locations and the prediction results, generating a plan that includes efficient work procedures and a time schedule.

[1452] The work plan is presented on the terminal and an interface is provided for the user to review and approve. Once the user reviews and approves the plan, it is sent to the automated robot.

[1453] The automated robot will begin work in the designated area based on the received work plan, and will use GPS information to navigate automatically and remove mines.

[1454] The server monitors the work status of the automated robot in real time, constantly updating progress information to ensure that the work is progressing smoothly.

[1455] Emotion engine integration

[1456] Furthermore, the system is integrated with an emotion engine that recognizes the user's emotions. The server receives and analyzes the user's facial expression and voice data.

[1457] The server uses an emotion analysis algorithm to determine the user's emotional state, for example, if the user is feeling anxious, that emotional state is detected.

[1458] The device dynamically adjusts the interface based on the user's emotional state, for example simplifying the interface to present information more clearly when the user is feeling anxious.

[1459] Specific examples

[1460] As a concrete example, the procedure for carrying out mine clearance work in a certain area will be explained.

[1461] 1. The server receives the image data sent from the drone.

[1462] 2. The server performs image analysis and detects a mine at coordinates (10.1234, 20.5678).

[1463] 3. The device displays the analysis results on a map for the user.

[1464] 4. The server collects and organizes past mine data and predicts where mines are buried.

[1465] 5. The server generates the prediction results and provides them to the device as a dashboard.

[1466] 6. The device displays the prediction results for the user to confirm.

[1467] 7. The server creates a removal plan and displays it on the terminal.

[1468] 8. The user reviews and approves the work plan.

[1469] 9. The automated robot moves to the designated location and removes the mines.

[1470] 10. The server monitors the robot's work status in real time.

[1471] 11. The server analyzes the user's emotions and dynamically adjusts the interface.

[1472] This series of processes not only ensures safe and efficient mine clearance work, but also reduces the psychological burden on the user. This system protects many people from the dangers of landmines while also improving the efficiency of work.

[1473] The processing flow will be explained below.

[1474] Step 1:

[1475] The server receives image data from a satellite or drone. The image data is sent over the network, and the server saves the data in storage. Specifically, it calls the receive_image_data() function to retrieve the image. This data is saved in a variable called "image_data," for example.

[1476] Step 2:

[1477] The server analyzes the received image data. The acquired image data is input into the AI ​​model to identify the location of buried landmines. The AI ​​model contains a specific image recognition algorithm and executes the analyze_image(image_data) function, which identifies the coordinates of the buried landmines.

[1478] Step 3:

[1479] The server sends the analysis results to the device. It generates coordinate data for the identified mines and sends it to the device. It sends the analysis results using the send_results(analysis results, device ID) function.

[1480] Step 4:

[1481] The device receives the analysis results. The device visually displays the received coordinate data. For example, the display_results(analysis results) function is used to mark specific coordinates on a map to show the user where the mines are located.

[1482] Step 5:

[1483] The server collects information on past mine placements and accidents and stores it in a database. It retrieves information from historical databases and various reports and stores it in the database using the collect_data(data source) function. For example, it is stored in a variable called "historical_data."

[1484] Step 6:

[1485] Organize and consolidate the data collected by the server. Normalize and clean the data to make it analyzable by the AI ​​model. Use the normalize_data(collected_data) function.

[1486] Step 7:

[1487] The server uses an AI model to predict mine locations. The organized data is input into the AI ​​model to predict unknown mine locations. The predict_mine_locations(model, organized data) function is executed to obtain the predicted results.

[1488] Step 8:

[1489] The server provides the prediction results to the terminal. The prediction results are generated so that they can be displayed in dashboard format and sent to the terminal. The dashboard is generated using the generate_dashboard(prediction result) function.

[1490] Step 9:

[1491] The device displays the forecast results to the user, allowing the user to see the results in a visual format (e.g., a graph or map). This is done using the display_dashboard(forecastresult) function.

[1492] Step 10:

[1493] The server creates a mine removal plan. It creates an efficient removal plan based on the prediction results and the identified mine locations. It uses the create_removal_plan(identified location, prediction result) function.

[1494] Step 11:

[1495] The terminal presents the work plan to the user. The details of the work plan are displayed and the user confirms and approves it. The display_plan(plan) function is used.

[1496] Step 12:

[1497] The user checks and approves the work plan. The plan presented on the terminal is checked and determined to be feasible. After approval, the server is notified. The approve_plan() function is used.

[1498] Step 13:

[1499] The automated robot begins work based on the work plan received from the server. It follows GPS information to move to the designated mine site and carry out the removal work. The execute_removal_task(robot, plan) function is used.

[1500] Step 14:

[1501] The server monitors the work status of the automated robot in real time. It receives feedback data from the robot and monitors its progress. It uses the monitor_progress(robot) function.

[1502] Step 15:

[1503] The server receives the user's facial expression and voice data and performs emotion analysis. It analyzes the user's emotional state in real time and identifies the state using the emotion analysis engine. It uses the analyze_emotion(facial expression data, voice data) function.

[1504] Step 16:

[1505] The device dynamically adjusts the interface based on the user's emotional state. For example, if the user is feeling anxious, the interface can be simplified to make the information easier for the user to understand. This is done using the adjust_interface(emotional_state) function.

[1506] These steps allow the system to safely and efficiently remove mines and further optimize the user experience based on the user's emotional state. For example, in step 7 above, the system predicts that a new mine has been buried at coordinates (10.1234, 20.5678), and the removal process proceeds based on that prediction. The interface is simplified when the user feels anxious. In this way, the overall system further enhances safety and efficiency, and also provides psychological support.

[1507] Example 2

[1508] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1509] Conventional mine clearance systems have low analytical accuracy for accurately identifying the location of mines, limiting the efficiency of clearance work. Furthermore, they lack a method for reducing the psychological burden on users. Therefore, there is a need for a system that can clear mines quickly and safely while also reducing the psychological burden on users.

[1510] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for acquiring satellite images or drone images; means for analyzing the acquired images to identify buried mine locations; means for transmitting the analysis results to the terminal and visually displaying them; means for storing past mine placement information and accident information in a database; means for predicting mine locations based on the stored information; means for providing the prediction results to the terminal so that the user can confirm them; means for formulating a removal work plan based on the buried mine locations and the prediction results; an automatic robot for executing the formulated work plan; means for monitoring the work status of the automatic robot in real time; emotion engine means for receiving and analyzing facial expression data and voice data of the user; and means for dynamically adjusting the terminal interface based on the user's emotional state. This enables mine identification, prediction, and removal work to be performed quickly and safely while reducing the psychological burden on the user.

[1511] "Satellite imagery" refers to image data taken of a specific area on Earth by an artificial satellite.

[1512] "Drone images" are aerial image data taken by a camera mounted on a drone.

[1513] "Image analysis" is the technique of processing acquired image data to identify specific objects.

[1514] "Mine burial location" is information about the location where a mine is buried in the ground.

[1515] A "terminal" is a computing device through which a user can view information.

[1516] A "database" is a system that systematically stores and manages data.

[1517] "Prediction results" are estimated information derived using artificial intelligence algorithms.

[1518] A "clearance operation plan" is a specific procedure and schedule for efficiently clearing landmines.

[1519] An "automatic robot" is a mechanical device that operates autonomously based on set instructions to remove landmines.

[1520] "Real-time monitoring" is a technology that instantly monitors current progress.

[1521] An "emotion engine" is a technology that analyzes a user's facial expression data and voice data to identify their emotional state.

[1522] An "interface" is a screen or operating method that serves as a point of contact for the user and the system to exchange information with each other.

[1523] An "artificial intelligence algorithm" is a computational method that analyzes large amounts of data to derive patterns and predictions.

[1524] "Visually displaying" means displaying information on a screen in a format that is easy for the user to understand.

[1525] "Facial expression data" is information that captures the user's facial movements and expressions.

[1526] "Voice data" is information that records the user's utterances and voice characteristics.

[1527] "Dynamic adjustment" means changing the settings flexibly according to the situation at hand.

[1528] MODE FOR CARRYING OUT THE INVENTION

[1529] This invention is a system that combines artificial intelligence technology and an emotion engine to safely and efficiently remove landmines. Specific embodiments for carrying out this invention will be described below.

[1530] Image Recognition

[1531] First, the server receives image data sent from satellites or drones. This image data is then sent to the server via a network and stored in an appropriate storage. This process can use cloud storage such as Amazon S3.

[1532] The server then uses the received image data to apply image analysis algorithms, specifically using machine learning libraries such as TensorFlow to identify the locations of buried mines using a Convolutional Neural Network (CNN) model, and generates coordinate data for the areas where the mines are located.

[1533] The analysis results are sent to the device and displayed visually, and the device uses the Google Maps API or similar to mark specific coordinates on a map to show the user the location of the mines.

[1534] Information gathering and prediction

[1535] The server collects and stores information on past mine placements and accidents in a database. This information is obtained from historical databases and local reports. The database can be a database system such as PostgreSQL.

[1536] The server then normalizes and cleans the collected and stored data, converting it into a format suitable for AI analysis. This process uses data processing libraries such as Pandas.

[1537] The server inputs the organized data into an AI model to predict the location of buried mines, using a time-series prediction model such as a Long Short-Term Memory (LSTM) network. The prediction results are generated in the form of a dashboard and sent to the device.

[1538] The terminal visualizes the prediction results provided by the server, allowing users to easily check them. By using concrete diagrams and maps, users can intuitively understand the prediction results.

[1539] Removal work planning and execution

[1540] The server then creates a removal plan based on the identified mine locations and the predicted results. This plan includes efficient work procedures and time schedules. An optimization algorithm can be used to generate the most efficient removal plan.

[1541] The work plan is presented on the terminal and an interface is provided for the user to review and approve, and once the user approves the plan, it is sent to the automated robot.

[1542] The autonomous robot uses a GPS device to automatically navigate to the designated area and begin the clearance process. Specifically, the robot detects landmines and uses explosive ordnance disposal equipment to safely remove them.

[1543] The server monitors the work status of the automated robot in real time, and progress information is updated and displayed on the terminal.

[1544] Emotion engine integration

[1545] Furthermore, this system is integrated with an emotion engine that recognizes the user's emotions. The server receives and analyzes the user's facial expression and voice data. The analysis uses a voice recognition library and an expression analysis library.

[1546] The server uses an emotion analysis algorithm to identify the user's emotional state, for example, if the user is feeling anxious, that emotion is detected.

[1547] The device dynamically adjusts its interface based on the user's emotional state: when the user is feeling anxious, it simplifies the interface and provides more understandable information.

[1548] Examples of concrete examples and prompts

[1549] As a concrete example, the procedure for carrying out mine clearance work in a certain area is shown below.

[1550] 1. The server receives high-resolution image data sent from the drone and begins processing.

[1551] 2. The server uses a Convolutional Neural Network (CNN) to analyze the image and detect a landmine at coordinates (10.1234, 20.5678).

[1552] 3. The device uses the Google Maps API to mark the analysis results on a map in real time and display them to the user.

[1553] 4. The server collects and organizes landmine data from the past 10 years from the PostgreSQL database and inputs it into the AI ​​model.

[1554] 5. The server uses the LSTM model to generate a dashboard of predicted mine locations and send it to the device.

[1555] 6. The device displays the dashboard provided by the server, and the user checks the prediction results.

[1556] 7. The server uses an algorithm to create an optimal removal plan and displays it on the device.

[1557] 8. The user reviews the plan and approves it through the device interface.

[1558] 9. An automated robot uses a GPS device to navigate to a designated location and uses a robotic arm to dig up the mines.

[1559] 10. The server monitors the robot's operation in real time and displays the progress on the terminal.

[1560] 11. The server receives the user's facial expression data from the camera and detects feelings of anxiety.

[1561] 12. The device simplifies the interface and displays a reassuring message to the user.

[1562] These systems allow mine clearance work to be carried out quickly and safely, and also reduce the psychological burden on users.

[1563] Example prompt sentence:

[1564] "Please generate explanatory text for a mine removal system that detects buried mine locations from image data, makes predictions based on past mine data, and allows users to proceed with confirmation and removal work with confidence."

[1565] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1566] Step 1: Receiving image data

[1567] The server receives high-resolution image data transmitted from a satellite or drone. The input is the image data transmitted from the satellite or drone, and the output is the image data stored in the server's storage. Specifically, this can be done using cloud storage such as Amazon S3.

[1568] Step 2: Image analysis

[1569] The server uses the image data received by the server to identify buried landmine locations by applying a Convolutional Neural Network (CNN) model using machine learning libraries such as TensorFlow. The input is the image data stored in storage, and the output is the coordinate data of the area where the landmines are located. Specifically, the CNN model extracts image features and identifies the patterns of landmines.

[1570] Step 3: Viewing the analysis results

[1571] The server sends the results of the image analysis to the device. The device visually displays the analysis results using Google Maps API or similar. The input is the coordinate data sent from the server, and the output is the location of the mine marked on the map. Specifically, it marks specific coordinates on the map to show the user the location of the mine.

[1572] Step 4: Gather information

[1573] The server collects information on past mine placements and accidents from historical databases and local reports, and stores this information in a database. The input is mine information obtained from external data sources, and the output is information stored in the database. Specific operations use a database system such as PostgreSQL.

[1574] Step 5: Organize your data

[1575] The server normalizes and cleans the collected and stored data. The input is the stored minefield information, and the output is data formatted for AI analysis. Specifically, it uses data processing libraries such as Pandas to fill in missing values ​​and remove outliers.

[1576] Step 6: Predicting mine locations

[1577] The server inputs the organized data into an AI model to predict where landmines will be buried. The input is normalized landmine information, and the output is coordinate data of the predicted landmine locations. Specifically, a time series prediction model such as a Long Short-Term Memory (LSTM) network is used.

[1578] Step 7: View the prediction results

[1579] The server generates the prediction results in a dashboard format and sends them to the terminal. The terminal visualizes them so that the user can easily check them. The input is the prediction result data sent from the server, and the output is the prediction results displayed in graphs, maps, etc.

[1580] Step 8: Develop a removal plan

[1581] The server then creates a removal work plan based on the identified mine locations and the prediction results. The input is information on mine locations and the prediction results, and the output is a work plan. Specifically, an optimization algorithm is used to generate a plan that includes efficient work procedures and time schedules.

[1582] Step 9: Workplan Approval

[1583] The terminal presents the work plan to the user, who then reviews and approves the plan through the interface. The input is the work plan sent from the server, and the output is the plan approved by the user.

[1584] Step 10: Perform the removal work

[1585] The automated robot begins work in the designated area based on the work plan it receives. The input is the approved work plan, and the output is the number of mines removed and the progress of the work. Specifically, it uses GPS information to move autonomously and physically remove the mines.

[1586] Step 11: Monitoring the work

[1587] The server monitors the work status of the automated robot in real time. The input is progress information from the automated robot, and the output is progress status data updated in real time. Specifically, the data is displayed on the terminal as it is processed.

[1588] Step 12: Analyze emotional state

[1589] The server receives the user's facial expression and voice data and uses an emotion analysis algorithm to identify the user's emotional state. The input is data obtained from a camera or microphone, and the output is the analyzed emotional state. Specific operations use a voice recognition library and an emotion analysis library.

[1590] Step 13: Adjusting the Interface

[1591] The device dynamically adjusts the interface based on the user's emotional state. The input is the analyzed emotional state, and the output is an interface that corresponds to the user's emotional state. Specifically, if the user feels anxious, the interface is simplified to present information in a more understandable manner.

[1592] (Application example 2)

[1593] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1594] Conventional mine clearance systems are required to identify buried mines and remove them safely and efficiently, but there is a problem in that it is difficult for the entire system to perform the task quickly and accurately. Furthermore, they lack support functions that take into account the driver's emotional state, and do not improve safety or comfort while driving. Therefore, a system that can simultaneously remove mines and provide driver support was needed.

[1595] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring satellite images or drone images, means for analyzing the acquired images to identify buried mine locations, means for transmitting the analysis results to the terminal and visually displaying them, means for storing past mine placement information and accident information in a database, means for predicting mine locations based on the stored information, means for providing the prediction results to the terminal so that the user can check them, means for formulating a removal operation plan based on the buried mine locations and the prediction results, an automated robot for executing the formulated operation plan, means for monitoring the operation status of the automated robot in real time, means for analyzing the driver's emotions while driving and dynamically adjusting the interface, and means for providing hazard prediction information in accordance with the driver's emotional state. This not only enables efficient mine removal operations but also provides safe and comfortable assistance to the driver while driving.

[1596] A "server" is a computer system that sends, receives, and processes data over a network.

[1597] "Satellite imagery" refers to high-resolution images of the Earth's surface or objects taken by satellites.

[1598] "Drone imagery" refers to images of the earth's surface or objects taken by unmanned aerial vehicles (drones).

[1599] "Image analysis" is a technology that identifies and identifies objects or specific patterns based on acquired image data.

[1600] "Means for identifying buried landmine locations" refers to technology that uses image analysis to identify locations where landmines are buried.

[1601] A "terminal" is a device such as a computer or smartphone that displays data sent from a server and can be operated by a user.

[1602] "Past mine placement information" refers to data that indicates previously recorded mine placement locations and related information.

[1603] "Accident information" is data that records the location and circumstances of mine-related accidents.

[1604] A "database" is a data collection system that allows information to be efficiently stored, retrieved, and updated.

[1605] The "prediction results" are estimates of future mine locations based on past data.

[1606] A "clearance operation plan" is a plan that outlines specific procedures and schedules for safely clearing landmines based on the identified locations of the mines.

[1607] An "automatic robot" is a machine that autonomously carries out tasks as instructed.

[1608] "Real-time monitoring means" refers to a system for instantly monitoring and checking ongoing situations.

[1609] "Means for analyzing emotions" refers to technology for analyzing a user's facial expressions, voice, etc. to identify their emotional state.

[1610] "Means for dynamically adjusting the interface" refers to technology that automatically changes the display content and operation method according to the user's emotional state.

[1611] "Means for providing risk prediction information" refers to technology that predicts future risks based on past data and current conditions and notifies the user.

[1612] The embodiment of the present invention is based on a series of systems including a server, a terminal, an automatic robot, and an emotion analysis system. This system can analyze images in real time, identify the location of landmines, and efficiently remove them, while also grasping the emotional state of the driver and providing appropriate feedback.

[1613] The server receives image data from satellites and drones and performs image analysis based on this data. Specifically, the server uses a high-performance GPU and deep learning frameworks such as TensorFlow and Keras. This image analysis identifies specific patterns and locates buried mines. The coordinate data of identified mines is sent to the device and displayed visually.

[1614] Past mine placement and accident information is stored in a database. The server integrates this information and uses an AI model to predict where mines will be placed. The results of this prediction are also provided to the device, where users can check them. Predictions require cleansing and normalization of past data.

[1615] The server then creates a removal work plan based on the location of buried mines and the prediction results. This plan, which includes efficient work procedures and time schedules, is displayed on the terminal. Once the user confirms and approves the plan, work instructions are sent to the automated robot. The automated robot uses GPS information to automatically move to the designated area and remove the mines. The server also monitors the automated robot's work status in real time and immediately addresses any problems that arise.

[1616] The integration of an emotion engine is also a key element of this system. The server receives facial expression and voice data from the driver or operator via the device and performs emotion analysis. This analysis uses an emotion recognition algorithm, and if the user feels anxious or tired, the interface is dynamically adjusted. This allows the user to use information more easily and with peace of mind.

[1617] For example, a system can analyze live video feeds of a highway, detect obstacles ahead, and warn the driver. At the same time, if anxiety is detected from the driver's facial expression, the system simplifies the interface and activates reassuring voice guidance. In this way, both mine clearance and driver assistance can be carried out efficiently.

[1618] Example prompt sentence:

[1619] "Creating an application that detects obstacles ahead from live video footage on the highway, analyzes the driver's emotions, and provides appropriate feedback."

[1620] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1621] Step 1:

[1622] The server receives satellite or drone images. These images are sent to the server through a network and stored in a database. The input data is high-resolution image data, and the output is stored image data.

[1623] Step 2:

[1624] The server applies image analysis algorithms to the stored image data. Specifically, it uses deep learning frameworks such as TensorFlow and Keras to identify specific patterns and identify buried mine locations. The input data is the stored image data, and the output is the coordinate data of the mines.

[1625] Step 3:

[1626] The server sends coordinate data of the identified landmines to the terminal. The terminal visually displays the received data on a map and provides the user with information on the location of the landmines. The input data is the coordinate data of the landmines, and the output is the location information of the landmines marked on the map.

[1627] Step 4:

[1628] The server collects and organizes past mine laying and accident information into a database. The input data is past mine and accident information, and the output is an organized database.

[1629] Step 5:

[1630] The server uses an AI model to predict future mine locations based on the collected data. The input data is an organized database, and the output is predicted mine location data.

[1631] Step 6:

[1632] The server sends the prediction results to the terminal, which then visually displays the prediction results to the user. The input data is the predicted mine location data, and the output is the visually displayed prediction results.

[1633] Step 7:

[1634] The server then creates a removal plan based on the identified mine locations and prediction results. The plan includes efficient work procedures and time schedules. The input data is mine location information and prediction results, and the output is a detailed work plan.

[1635] Step 8:

[1636] The server sends the prepared work plan to the automated robot, which then uses GPS information to automatically move to the designated location and remove the mines based on the plan. The input data is the work plan, and the output is the progress of the mine removal work.

[1637] Step 9:

[1638] The server monitors the work status of the automated robot in real time and makes adjustments and instructions as necessary. The input data is progress data, and the output is the monitoring results and instructions.

[1639] Step 10:

[1640] The server receives the driver's facial expression and voice data through the terminal and performs emotion analysis. The input data is facial expression data and voice data, and the output is analyzed emotional state data.

[1641] Step 11:

[1642] The server dynamically adjusts the device interface based on the analyzed emotional state and provides risk prediction information according to the driver's emotional state. The input data are emotional state data and past prediction data, and the output is the dynamically adjusted interface and the provided risk prediction information.

[1643] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1644] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1645] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1646] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1647] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1648] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1649] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1650] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1651] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1652] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1653] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users ...

Claims

1. a means for acquiring satellite or drone imagery; means for analyzing the acquired image to identify buried mine locations; means for transmitting the analysis results to a terminal and visually displaying them; a means for storing past mine burial information and accident information in a database; a means for predicting mine placement locations based on the stored information; A means for providing the prediction result to the terminal so that the user can check it; a means for planning removal operations based on the location of buried mines and the predicted results; an automatic robot for executing the planned work plan; a means for monitoring the working status of the automatic robot in real time; A system including:

2. 2. The system according to claim 1, further comprising means for identifying buried mine locations based on the acquired images using an artificial intelligence algorithm.

3. 2. The system of claim 1, wherein the automated robot further comprises means for automatically moving and removing mines using GPS information based on a removal operation plan.

Images