System

Abstract

A system is provided.SOLUTION: A system comprising: means for storing a miniaturized artificial intelligence model in a storage medium for data processing at multiple locations; means for transmitting a latest artificial intelligence model through a network to periodically update the artificial intelligence model stored in the storage medium; and means for analyzing collected data in real time using the artificial intelligence model stored in the storage medium and providing an analysis result to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In remote locations where communication environments are unstable or in disaster areas where communication networks are easily disrupted, it is difficult to perform real-time data analysis. This challenge is particularly pronounced when researchers need rapid data analysis or when disaster relief teams need to make immediate decisions on the ground. Cloud-based data analysis is highly dependent on the communication environment and is therefore likely to malfunction in these situations. Therefore, there is a need for technology that can perform high-precision data analysis on-site, even in situations where communication environments are unstable or unavailable. [Means for solving the problem]

[0005] The present invention provides a system that enables on-site data processing by storing a miniaturized artificial intelligence model on a storage medium. Specifically, the system includes a means for storing the miniaturized artificial intelligence model on a storage medium for multi-site data processing, and a means for transmitting the latest artificial intelligence model via a network to periodically update the artificial intelligence model stored on the storage medium. The system also includes a means for analyzing collected data in real time using the artificial intelligence model stored on the storage medium and providing the analysis results to a user. This system allows data analysis using the artificial intelligence model stored on the storage medium even in situations where the network is unstable or unavailable, enabling researchers and disaster relief teams to quickly obtain highly accurate information.

[0006] A "miniaturized artificial intelligence model" is a machine learning model trained on a server that has been optimized to run on fewer resources.

[0007] A "storage medium" is a device or apparatus that can store digital data and read it as needed.

[0008] A "network" is a communications infrastructure that connects multiple computers and devices together to exchange data.

[0009] "Data analytics" is the process of using collected data to organize and analyze the information using specific algorithms and models to derive meaningful results.

[0010] A "user" is an entity or individual who operates the system and uses the results.

[0011] A "server" is a computer system connected to a network that trains data, updates models, and receives and transmits data.

[0012] "Remote areas" are areas that are far from major communications infrastructure and where internet connectivity is unstable or nonexistent.

[0013] A "disaster relief team" is a specialized group or organization that rescues lives and assesses damage at disaster sites.

[0014] "Communication environment" is a term that indicates the status and quality of the network for sending and receiving data. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention relates to a system that stores a miniaturized artificial intelligence model in a storage medium and performs real-time data analysis even in an unstable or unavailable communication environment. Specific examples of the system based on the present invention are described in detail below.

[0037] 1. The server updates the AI ​​model

[0038] The server collects the latest training data and optimizes the AI ​​model. After training a new AI model using the training data, the server converts it into a portable AI memory format and transmits it to the device via the network. Therefore, the server uses its high-performance computing resources to efficiently generate and distribute AI models.

[0039] 2. The device receives the AI ​​model

[0040] The device receives the latest AI model update notification from the server and downloads the model via the network. After downloading, the device stores the new model in its internal memory and installs it. This ensures that the device always has the latest AI model.

[0041] 3. Users collect data

[0042] Users collect necessary data in remote locations or disaster sites. The collected data is diverse, including images, videos, audio, and sensor data. This data is input into a terminal and stored on a storage medium. Users use data collection tools (e.g., cameras, sensors) to efficiently collect data.

[0043] 4. The device analyzes the data

[0044] The device uses the AI ​​model stored on the storage medium to perform real-time data analysis based on data provided by the user. As data is input, the device preprocesses the data and converts it into a suitable format for analysis. From there, the AI ​​model begins its analysis and generates results. The analysis results are provided to the user via a user interface or display device.

[0045] 5. Users consume the results

[0046] Users who receive the analysis results can make decisions in real time based on that information. For example, researchers in remote locations can use the analysis results to determine the direction and next steps of their research, allowing for more efficient research. Disaster relief teams can also use the analysis results to carry out rapid and effective rescue operations. As a result, highly accurate data analysis and rapid decision-making are possible even in situations where communication is unstable or unavailable.

[0047] Specific examples

[0048] Example 1: Disaster Relief Scenario

[0049] In a disaster site where communication networks are unavailable, disaster relief teams collect images and audio to assess the damage. The collected data is input into a portable AI memory and instantly analyzed by an artificial intelligence model. The analysis results include the scale of damage, high-risk areas, and optimal rescue routes. Rescue teams use this information to carry out rapid and effective rescue operations.

[0050] Example 2: Remote research scenario

[0051] Researchers studying new species in remote locations input data collected on-site into a portable AI memory. The AI ​​model analyzes the input data in real time and provides insights into the characteristics and habitat of the new species. This allows researchers to decide on the next research direction on the spot and advance their research more efficiently.

[0052] The above is a detailed description of one embodiment of the present invention, which enables advanced data analysis and rapid decision-making even in an unstable communication environment.

[0053] The processing flow will be explained below.

[0054] Step 1:

[0055] The server collects the latest training data.

[0056] The server acquires weather data, earthquake data, and other related data and stores it in a database.

[0057] Step 2:

[0058] The server preprocesses the acquired training data.

[0059] It removes noise, normalizes, and fills missing values, converting the data into a format suitable for training AI models.

[0060] Step 3:

[0061] The server trains the artificial intelligence model.

[0062] Apply machine learning algorithms to build AI models and train them using the acquired training data.

[0063] Step 4:

[0064] The server converts the artificial intelligence model after training into a portable AI memory device.

[0065] The model is optimized, compressed, converted to a memory-compatible format, and saved.

[0066] Step 5:

[0067] The server sends the latest AI model to the device via the network.

[0068] The trained AI model is encoded and data transmission is performed using a secure communication method.

[0069] Step 6:

[0070] The device detects an update notification for the AI ​​model received from the server.

[0071] It periodically checks for update notifications from the server and starts receiving them if a new model is available.

[0072] Step 7:

[0073] The device downloads the new AI model.

[0074] The AI ​​model is downloaded via the network and quickly saved to the device.

[0075] Step 8:

[0076] The device will install the new AI model.

[0077] It unpacks the saved AI model into memory, deletes the previous model, and installs the new one.

[0078] Step 9:

[0079] Users collect data at disaster sites and remote locations.

[0080] The necessary data (images, audio, sensor data, etc.) is collected using cameras, sensors, etc. and input into the terminal.

[0081] Step 10:

[0082] The terminal receives and pre-processes data provided by the user.

[0083] Check the format of the input data and remove outliers and noise to make it analyzable.

[0084] Step 11:

[0085] The device inputs the preprocessed data into an AI model for real-time analysis.

[0086] The AI ​​model begins the analysis and generates analytical results.

[0087] Step 12:

[0088] The terminal obtains the analysis results and provides them to the user through a user interface.

[0089] The results of the analysis are then formatted appropriately and presented to the user via a display or other output device.

[0090] Step 13:

[0091] Users make decisions based on the analysis results.

[0092] Based on the analysis results received, we will formulate and implement research directions and rescue operation plans.

[0093] Example 1

[0094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] Real-time data analysis is required in situations where data processing is required at multiple locations, such as in medical care, disaster relief, or remote research. However, in these situations, communication environments are often unstable or unavailable, resulting in delays in data analysis and processing. In addition, because the amount of information transmitted is large, efficient processing on the terminal side is also important. Conventional methods have made it difficult to make quick decisions under such circumstances.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0097] In this invention, the server includes means for using high-performance computing resources to train an AI model based on the latest training data, converting it for portable use, and transmitting it to the terminal, means for the terminal to receive update notifications from the server and download and install new AI models, means for the user to input various data collected using a data collection tool into the terminal and store it in a storage medium, means for the terminal to preprocess the data and analyze it in real time using the AI ​​model stored in the storage medium, and provide the user with the analysis results, and means for the user to make decisions based on the analysis results. This enables fast and accurate data analysis and efficient decision-making even in situations where the communication environment is unstable or unavailable.

[0098] "High-performance computing resources" refers to hardware and software for efficiently and quickly processing data and performing calculations, and specifically includes high-performance GPUs and servers.

[0099] "Training data" refers to various types of data used to train artificial intelligence models, including, for example, image data and text data.

[0100] "Artificial intelligence model" refers to a model designed using machine learning algorithms and trained to perform specific tasks automatically.

[0101] "Converting for portable use" refers to converting a trained artificial intelligence model into a format that can run efficiently on a device that processes data.

[0102] "Update Notification" refers to a message sent from a server to a device informing the device that a newer version of an artificial intelligence model is available.

[0103] "Downloading and installing" refers to the device receiving (downloading) a new artificial intelligence model from the server and setting it up for use (installing).

[0104] "Data collection tools" refers to various devices and sensors that users use to collect data, including cameras and thermometers.

[0105] "Real-time analysis" refers to analyzing data as soon as it is entered and generating results.

[0106] "Providing the analysis results to the user" refers to the terminal presenting the information obtained by the analysis to the user through a user interface or display device.

[0107] "Making a decision" refers to a user deciding on a future course of action or strategy based on the information provided.

[0108] "Unstable or unavailable network environment" refers to a situation where the network connection is of poor quality or completely unavailable.

[0109] This invention relates to a system that stores a miniaturized artificial intelligence model on a storage medium and performs real-time data analysis even in situations where communication is unstable or unavailable. This system is realized through the cooperation of a server, terminals, and users.

[0110] server

[0111] The server uses high-performance computing resources (e.g., NVIDIA Tesla V100 GPU) to train an artificial intelligence model based on the latest training data. The server collects the necessary training data from databases and cloud storage and optimizes the model using machine learning algorithms. After training is complete, the server uses specific tools such as TensorFlow Lite Converter to convert the model for portable use. The converted model is then transmitted to the device via a network (e.g., 5G communication). A secure communication protocol (e.g., SSL / TLS) is used for this transmission.

[0112] Terminal

[0113] The device receives update notifications from the server and downloads and installs the new AI model. Update notifications are sent using protocols such as HTTP / 2. After downloading, the device stores the new model in its internal memory (e.g., an SD card) and installs it in a usable state. The device uses the downloaded model to analyze data provided by the user in real time.

[0114] User

[0115] Users collect the necessary data using data collection tools (e.g., cameras, sensors). For example, at disaster sites, they collect photos of the damage and sensor data. The collected data is entered into the device and stored in the internal memory. Users use data collection tools to efficiently collect data.

[0116] Data analysis

[0117] The device performs preprocessing such as noise removal and normalization on the data provided by the user. This preprocessing improves the accuracy of the analysis. The device then analyzes the preprocessed data in real time using an artificial intelligence model. The analysis results are provided to the user as formatted data (e.g., JSON format), and the user can view the results on the device's display or through a dedicated application.

[0118] Specific examples

[0119] Example 1: Disaster Relief Scenario

[0120] When communication networks are unavailable at disaster sites, disaster relief teams collect images and audio to assess the damage situation. The collected data is input into a terminal, and an artificial intelligence model immediately analyzes it. For example, a prompt might be used, such as, "Please analyze the damage situation using this image data." The analysis results include the scale of damage, high-risk areas, and optimal rescue routes. Rescue teams use this information to carry out prompt and effective rescue operations.

[0121] Example 2: Remote research scenario

[0122] Researchers studying new species in remote locations input data collected on-site into a terminal, using a prompt such as, "Analyze the collected biological data and determine whether it is a new species." The AI ​​model analyzes the input data in real time and provides insights into the characteristics and habitat of the new species. This allows researchers to decide on the next research direction on the spot and advance their research more efficiently.

[0123] This system enables rapid and accurate data analysis and efficient decision-making even in situations where communication is unstable or unavailable, and is expected to have a wide range of applications, including disaster relief and remote research.

[0124] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0125] The flow of this system's program processing

[0126] Step 1: Server collects training data

[0127] The server collects training data from various data sources (e.g., databases, cloud storage). The input is raw data (e.g., image data, text data) obtained from multiple data sources and stores this data in local storage. Specific operations include executing API calls and database queries.

[0128] Step 2: The server trains the AI ​​model

[0129] The server uses a high-performance GPU (e.g., NVIDIA Tesla V100) to train an AI model based on the collected training data. The input is the stored training data, and the output is an optimized AI model. Specifically, the training process is performed using a machine learning library (e.g., TensorFlow, PyTorch).

[0130] Step 3: The server converts the AI ​​model for portable use

[0131] The server converts the trained AI model into a format that can be used on portable devices (e.g., ONNX format). The input is the trained AI model, and the output is an AI model converted for portable devices. Specific operations involve using tools such as TensorFlow Lite Converter.

[0132] Step 4: The server sends the model to the device

[0133] The server sends the converted AI model to the device via a network (e.g., 5G communication). The input is the AI ​​model converted for the portable device, and the output is the model data to be sent. Specifically, data is transmitted using a secure communication protocol (e.g., SSL / TLS).

[0134] Step 5: Your device receives an update notification

[0135] The terminal receives an update notification from the server. The input is a notification message from the server, and the output is that the terminal recognizes the model update. The specific operation is to notify using the HTTP / 2 protocol.

[0136] Step 6: Your device will download the new model

[0137] After the device confirms the update notification, it downloads the new model file from the server. The input is the model data from the server, and the output is the downloaded model file. The specific operation is to execute the file download process.

[0138] Step 7: The device installs the model

[0139] The device stores the downloaded model in its internal memory (e.g., SD card) and installs it. The input is the downloaded model file, and the output is the installed model. Specific operations include unpacking and deploying the model file.

[0140] Step 8: User prepares data collection tools

[0141] The user prepares a data collection tool, such as a camera (e.g., a high-resolution camera) or a sensor (e.g., an environmental sensor). The input is the readiness of the tool to be used, and the output is that the data collection tool is ready for use. Specific actions include calibrating the device and checking the battery.

[0142] Step 9: Collect the required data

[0143] The user uses the prepared tools to collect images, videos, audio, sensor data, etc. The input is the raw data obtained from the collection tool, and the output is the collected data. Specific actions include taking photos and recording audio.

[0144] Step 10: User enters data into terminal

[0145] The collected data is connected to a terminal and transferred to the internal memory. The input is the collected data, and the output is the data stored in the terminal. Specific operations include USB connection and wireless communication (e.g., Bluetooth).

[0146] Step 11: The device preprocesses the data

[0147] The device receives data provided by the user and performs preprocessing such as noise removal and normalization. The input is unprocessed data and the output is preprocessed data. Specific operations include adjusting the image resolution and filtering audio data.

[0148] Step 12: The device analyzes the data using the AI ​​model

[0149] The device uses an AI model to perform real-time analysis based on the preprocessed data. The input is the preprocessed data, and the output is the analysis results. Specific operations include image recognition and voice analysis.

[0150] Step 13: The device generates the analysis results

[0151] Once the analysis is complete, the terminal generates the results and outputs them as formatted data (e.g., JSON format). The input is the analysis result, and the output is the formatted result data. The specific operation is to convert the results into a data format.

[0152] Step 14: User receives analysis results

[0153] The user receives the analysis results from the terminal's display device or a dedicated application. The input is the formatted result data, and the output is the result displayed to the user. The specific operation is to display the results on the display device.

[0154] Step 15: User makes decision based on results

[0155] The user makes a decision based on the analysis results. The input is the displayed analysis results, and the output is the decision content. The specific action is to determine the optimal response depending on the situation.

[0156] The above is the specific flow of the program processing of this system.

[0157] (Application example 1)

[0158] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0159] In recent years, advances in autonomous driving technology have generated much expectation for autonomous vehicles, such as reducing traffic accidents and easing traffic congestion. However, when communication networks are unstable, necessary data analysis can be delayed, making it impossible to ensure safety. To address this issue, a new system capable of performing data analysis with high accuracy and in real time is required.

[0160] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0161] In this invention, the server includes means for storing a miniaturized artificial intelligence model in a storage medium, means for transmitting the latest artificial intelligence model via a network to periodically update the artificial intelligence model stored in the storage medium, means for analyzing collected data in real time using the artificial intelligence model stored in the storage medium and providing the analysis results to a user, and means for processing video data acquired from a camera mounted on the vehicle in real time even in an unstable communication environment to detect the presence or absence of obstacles and lane deviations, thereby enabling autonomous vehicles to operate safely and effectively even in situations where the communication network is unstable or unavailable.

[0162] A "miniaturized artificial intelligence model" is an artificial intelligence model that has been optimized and scaled down so that it can operate in environments with limited computing resources and memory capacity.

[0163] "Storage media" means any device or material for storing data electronically or magnetically.

[0164] A "network" is a system or infrastructure that connects multiple devices and enables data communication.

[0165] "Data" means any form of information, such as information, signals, or measurements, that is collected, stored, or analyzed.

[0166] "Real-time" means that input data is processed and analyzed almost as soon as it is generated.

[0167] A "user" is someone who uses or benefits from the use of a system or device.

[0168] A "vehicle-mounted camera" is a device installed in a vehicle to capture images of its surroundings.

[0169] "Video data" is digital data containing visual information captured by a camera.

[0170] "Processing" is a series of operations performed on data to manipulate and analyze it and obtain useful information or results.

[0171] An "obstacle" is any object or obstacle that may impede the vehicle's progress.

[0172] "Lane deviation" refers to a state in which the vehicle's current direction of travel deviates from the lane of the road.

[0173] "Discovery" is the act of finding data or situations that meet specified conditions or characteristics.

[0174] The server collects the latest training data and optimizes the AI ​​model. It uses high-performance computing resources to train a new AI model and converts it for portable AI memory. It then sends it to the device over the network, allowing the device to always have the latest AI model. The main software used is Keras and TensorFlow, and a high-performance server (e.g., a server with a GPU) is required for hardware.

[0175] The device receives update notifications from the server, downloads new AI models, and stores them in its internal memory. All the device requires is a network connection and local storage, which can be found on common devices such as smartphones and tablets.

[0176] The user collects the necessary data (video data) using a camera mounted on the vehicle. This video data is input into the terminal and saved in a storage medium. The collected video data is preprocessed using software such as Keras and PIL.

[0177] The device then analyzes the data in real time using an artificial intelligence model stored on the storage media. Preprocessing of the video data includes standardizing and normalizing the image size, converting the data into a format that the model can use to achieve optimal results. Analysis is performed using a high-performance processor (e.g., Apple A14 Bionic) and software such as Keras and TensorFlow.

[0178] The analysis results include information such as the presence or absence of obstacles and lane deviations. These results are provided to the user via the device's user interface. Based on this information, the user can make decisions in real time to ensure safe autonomous driving.

[0179] Examples:

[0180] This autonomous driving support AI assistant analyzes images from a camera in front of the vehicle in real time while driving on the highway. For example, even if communication is lost, it can detect pedestrians or obstacles around the vehicle and issue a warning or make an emergency stop. The autonomous vehicle uses this information to decide its next course of action.

[0181] Example prompts to input to a generative AI model:

[0182] "Detect pedestrians in real time from the forward camera image and display the results."

[0183] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0184] Step 1:

[0185] The server trains the latest artificial intelligence models and converts them for portable AI memory.

[0186] Input: High-performance computing resources (e.g., GPU-equipped servers), up-to-date training data

[0187] Output: Optimized artificial intelligence model

[0188] How it works: The server uses Keras and TensorFlow to extract features from the collected dataset, train the model, and optimize the weights. It then converts the optimized model into a portable format (e.g., .h5 file).

[0189] Step 2:

[0190] The server sends the artificial intelligence model to the terminal.

[0191] Input: Optimized artificial intelligence model, network connection

[0192] Output: AI model downloaded to device

[0193] Specific operation: The server sends the optimized model to the device using the HTTP protocol, etc. The device downloads the model via the network and stores it in its internal memory.

[0194] Step 3:

[0195] A user collects video data using a camera mounted on a vehicle.

[0196] Input: Vehicle-mounted camera

[0197] Output: Camera video data

[0198] Specific operation: The camera starts up at the timing and conditions specified by the user and continuously captures images of the area in front of the vehicle. This image data is collected frame by frame and transferred to the terminal.

[0199] Step 4:

[0200] The device preprocesses the video data and converts it into an analyzable data format.

[0201] Input: Camera video data

[0202] Output: Pre-processed video data

[0203] Specific operation: The device uses the Python Image Library (PIL) and Keras to perform preprocessing such as standardizing the image size (e.g., resizing to 224x224 pixels) and normalizing (scaling pixel values ​​to the 0-1 range).

[0204] Step 5:

[0205] The device uses artificial intelligence models to analyze the pre-processed data in real time.

[0206] Input: Preprocessed video data, artificial intelligence model

[0207] Output: Analysis results (presence or absence of obstacles, lane deviation, etc.)

[0208] Specific operation: The device uses Keras and TensorFlow to input preprocessed data into an artificial intelligence model and make predictions to detect the presence or absence of obstacles and lane deviations. The model's output includes specific classification results and location information.

[0209] Step 6:

[0210] The terminal provides the analysis results to a user interface.

[0211] Input: Analysis results

[0212] Output: A visual or audio notification provided to the user

[0213] How it works: The device provides the user with real-time analysis results via a user interface (e.g., on-screen alerts, voice guidance system), allowing the user to make driving decisions based on this information.

[0214] Example prompts based on concrete examples:

[0215] "Detect pedestrians in real time from the forward camera image and display the results."

[0216] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0217] The present invention relates to a system that stores a miniaturized artificial intelligence model on a storage medium and performs real-time data analysis even in situations where communication is unstable or unavailable. Another feature of the present invention is that it provides feedback and suggestions based on the user's state by combining it with an emotion engine that recognizes the user's emotions. A specific example of a system based on the present invention is described in detail below.

[0218] 1. The server updates the AI ​​model

[0219] The server collects the latest training data and optimizes the AI ​​model. After training a new AI model using the training data, the server converts it into a portable AI memory format and transmits it to the device via the network. Therefore, the server uses its high-performance computing resources to efficiently generate and distribute AI models.

[0220] 2. The device receives the AI ​​model

[0221] The device receives the latest AI model update notification from the server and downloads the model via the network. After downloading, the device stores the new model in its internal memory and installs it. This ensures that the device always has the latest AI model.

[0222] 3. Users collect data

[0223] Users collect necessary data in remote locations or disaster sites. The collected data is diverse, including images, videos, audio, and sensor data. This data is input into a terminal and stored on a storage medium. Users use data collection tools (e.g., cameras, sensors) to efficiently collect data.

[0224] 4. The device analyzes the data

[0225] The device uses the AI ​​model stored on the storage medium to perform real-time data analysis based on data provided by the user. As data is input, the device preprocesses the data and converts it into a suitable format for analysis. From there, the AI ​​model begins its analysis and generates results. The analysis results are provided to the user via a user interface or display device.

[0226] 5. Emotion Recognition by Emotion Engine

[0227] The device is equipped with an emotion engine that recognizes the user's emotions. This engine uses data acquired from sensors such as cameras and microphones to analyze the user's facial expressions, voice, and behavioral patterns. Based on this data, the emotion engine identifies the user's emotional state in real time and provides the user with the analysis results.

[0228] 6. Users consume the results and receive emotional feedback

[0229] Users can then make real-time decisions based on the analysis results. The emotion engine provides appropriate feedback and suggestions based on the user's emotional state. For example, if the stress level is high, it will suggest relaxation techniques and rest. If the user is in a positive emotional state, it will immediately display encouraging messages to take the next step.

[0230] Specific examples

[0231] Example 1: Disaster Relief Scenario

[0232] When communication networks are unavailable at a disaster site, disaster relief teams collect images and audio to assess the damage. The collected data is input into a portable AI memory and instantly analyzed by an artificial intelligence model. The analysis results include the scale of damage, high-risk areas, and optimal rescue routes. At the same time, an emotion engine analyzes the emotional state of the rescue team and suggests appropriate rest if stress levels are high. The rescue team then uses this information to carry out prompt and effective rescue operations.

[0233] Example 2: Remote research scenario

[0234] Researchers studying new species in remote locations input data collected on-site into a portable AI memory. The AI ​​model analyzes the input data in real time and provides insights into the characteristics and habitat of the new species. The emotion engine assesses the researcher's emotional state from their facial expressions and voice, and sends encouraging messages or suggests rest as needed. This allows researchers to decide on the next research direction on the spot and advance their research more efficiently.

[0235] This concludes the detailed description of one embodiment of the present invention, which enables advanced data analysis and appropriate feedback based on the user's emotions even in unstable communication environments, resulting in more effective and efficient decision-making.

[0236] The processing flow will be explained below.

[0237] Step 1:

[0238] The server collects the latest training data.

[0239] The server acquires weather data, earthquake data, and other related data and stores it in a database.

[0240] Step 2:

[0241] The server preprocesses the acquired training data.

[0242] It removes noise, normalizes, and fills missing values, converting the data into a format suitable for training AI models.

[0243] Step 3:

[0244] The server trains the artificial intelligence model.

[0245] Apply machine learning algorithms to build AI models and train them using the acquired training data.

[0246] Step 4:

[0247] The server converts the artificial intelligence model after training into a portable AI memory device.

[0248] The model is optimized, compressed, converted to a memory-compatible format, and saved.

[0249] Step 5:

[0250] The server sends the latest AI model to the device via the network.

[0251] The trained AI model is encoded and data transmission is performed using a secure communication method.

[0252] Step 6:

[0253] The device detects an update notification for the AI ​​model received from the server.

[0254] It periodically checks for update notifications from the server and starts receiving them if a new model is available.

[0255] Step 7:

[0256] The device downloads the new AI model.

[0257] The AI ​​model is downloaded via the network and quickly saved to the device.

[0258] Step 8:

[0259] The device will install the new AI model.

[0260] It unpacks the saved AI model into memory, deletes the previous model, and installs the new one.

[0261] Step 9:

[0262] Users collect data at disaster sites and remote locations.

[0263] The necessary data (images, audio, sensor data, etc.) is collected using cameras, sensors, etc. and input into the terminal.

[0264] Step 10:

[0265] The terminal receives and pre-processes data provided by the user.

[0266] Check the format of the input data and remove outliers and noise to make it analyzable.

[0267] Step 11:

[0268] The device inputs the preprocessed data into an AI model for real-time analysis.

[0269] The AI ​​model begins the analysis and generates analytical results.

[0270] Step 12:

[0271] The terminal obtains the analysis results and provides them to the user through a user interface.

[0272] The results of the analysis are then formatted appropriately and presented to the user via a display or other output device.

[0273] Step 13:

[0274] The device analyzes the user's facial expressions and voice using an emotion engine.

[0275] The device uses data from the camera and microphone to determine the user's emotional state.

[0276] Step 14:

[0277] The device provides feedback and suggestions based on the emotional state it identifies.

[0278] If the user is feeling stressed, it will suggest ways to relax, and if they are feeling positive, it will display an encouraging message.

[0279] Step 15:

[0280] Users use emotion-based feedback to make decisions.

[0281] Choose the appropriate action based on your mental state and move on to the next step.

[0282] Example 2

[0283] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0284] Conventional systems have difficulty analyzing data in unstable communication environments, which can delay real-time decision-making based on collected data. Furthermore, they do not provide feedback or suggestions that take into account the user's emotional state, hindering efficient activities.

[0285] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for storing a miniaturized artificial intelligence model in a storage medium for multi-site data processing, means for transmitting the latest artificial intelligence model via a network to periodically update the artificial intelligence model stored in the storage medium, means for analyzing collected data in real time using the artificial intelligence model stored in the storage medium and providing the analysis results to the user, means for performing real-time emotion analysis using data acquired from a sensor to recognize the user's emotional state, and means for providing appropriate feedback and suggestions based on the user's emotional state. This enables advanced data analysis and appropriate feedback based on the user's emotions even in an unstable communication environment, thereby realizing more effective and efficient decision-making.

[0286] "Miniaturized AI models" refer to AI algorithms or neural networks that are reduced in size and designed to run efficiently on portable devices.

[0287] "Storage media" refers to computer hardware components used for long-term data storage, including HDDs, SSDs, and USB memory sticks.

[0288] "Network" refers to the infrastructure for data communication between multiple computers and devices, and specifically includes Wi-Fi, 4G / 5G, Ethernet, etc.

[0289] "Data analytics" refers to the set of processes that are undertaken to process collected data and extract meaningful information and insights.

[0290] "Means for providing to the user" refers to an interface or mechanism that visually or audibly conveys analysis results and feedback information to the user.

[0291] "Emotion analysis" refers to a set of algorithms and methods for identifying a user's emotional or psychological state based on data obtained from sensors.

[0292] "Means for providing feedback and suggestions" refers to a system or interface that provides information to encourage or motivate a user to take appropriate action based on the user's emotional state or the results of data analysis.

[0293] The present invention relates to a system that stores a miniaturized artificial intelligence model on a storage medium and performs real-time data analysis even in unstable or unavailable communication environments. Another feature of the present invention is that it combines an emotion engine that recognizes the user's emotions to provide feedback and suggestions based on the user's state.

[0294] The system is implemented using the following hardware and software.

[0295] Update of artificial intelligence models by the server

[0296] The server optimizes the AI ​​model using high-performance computing resources. Specifically, it utilizes deep learning frameworks such as TensorFlow and PyTorch using GPUs on cloud services (e.g., Google Cloud Platform and Amazon Web Services). The server periodically collects the latest training data (e.g., image and audio data) and uses this data to train the AI ​​model. The server then converts the model into an optimal format for portable AI memory (e.g., ONNX) and transmits it to the device via a network (e.g., Wi-Fi, 4G / 5G).

[0297] Receiving artificial intelligence models and analyzing data on the device

[0298] The terminal receives the latest AI model update notification from the server and downloads the new model via the network. After downloading, the terminal stores the model in its internal memory and installs it. The terminal is a portable device such as a smartphone or laptop, and performs real-time data analysis based on the received model. Based on the data collected by the user (e.g., images, audio, sensor data), the data is preprocessed and converted into an appropriate format for analysis, and the AI ​​model analyzes the data. The analysis results are provided to the user via a user interface (e.g., a display).

[0299] Emotion recognition and feedback by emotion engine

[0300] The device is equipped with sensors such as a camera and microphone, which are used to collect the user's emotional data. The emotion engine uses this data to analyze the user's facial expressions, voice, and behavioral patterns to identify the user's emotional state in real time. Based on the analysis results, it provides appropriate feedback to the user. For example, if the user's stress level is high, it suggests taking a break, and if the user is in a positive emotional state, it displays an encouraging message to take the next step.

[0301] Prompt Sentence Examples

[0302] Example 1: Disaster Relief Scenario

[0303] A disaster relief team took 100 images at the scene and entered them into a mobile device. Explain how the device's AI model analyzed the extent of the damage, identified high-risk areas, and suggested rest based on stress levels.

[0304] Example 2: Remote research scenario

[0305] A researcher remotely collects photos of 30 new plant species and inputs them into a device. The device's AI model analyzes the data, provides characteristics of the new species, and an emotion engine detects when the researcher is fatigued and provides appropriate feedback. Explain the process.

[0306] This concludes the detailed description of one embodiment of the present invention, which enables advanced data analysis and appropriate feedback based on the user's emotions even in unstable communication environments, resulting in more effective and efficient decision-making.

[0307] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0308] Step 1:

[0309] A server collects training data.

[0310] Input: Large amounts of image and audio data stored in cloud storage (e.g., Amazon S3).

[0311] Specific operation: The server downloads the training data via the cloud API.

[0312] Output: Training data saved in local storage.

[0313] Step 2:

[0314] The server trains the artificial intelligence model.

[0315] Input: The training data collected in step 1.

[0316] What it does: The server trains a neural network using a deep learning framework such as TensorFlow or PyTorch.

[0317] Output: A fully trained artificial intelligence model.

[0318] Step 3:

[0319] The server converts the model into a portable format.

[0320] Input: The artificial intelligence model trained in step 2.

[0321] Specific operation: The server converts the model into a compatible format such as ONNX.

[0322] Output: An artificial intelligence model in a portable format.

[0323] Step 4:

[0324] The server sends the model to the device.

[0325] Input: The artificial intelligence model in portable format converted in step 3.

[0326] Specific operation: The server uploads the model to the device via the network (e.g., Wi-Fi, 5G) using the HTTP or FTP protocol.

[0327] Output: The artificial intelligence model downloaded to the device.

[0328] Step 5:

[0329] Your device will receive an update notification.

[0330] Input: Update notifications sent by the server.

[0331] Specific behavior: The device receives a push notification and checks for the existence of a new model.

[0332] Output: Update notification reception status.

[0333] Step 6:

[0334] The device will download and install the new model.

[0335] Input: The artificial intelligence model sent from the server in step 4.

[0336] Specific operation: The device sends an HTTP request to the server to download the model file. After the download is complete, the device stores the model in its internal memory, replacing the old model.

[0337] Output: The latest installed artificial intelligence model.

[0338] Step 7:

[0339] The user collects the data.

[0340] Input: Real-time data from cameras and sensors (e.g., images, video, audio, sensor data).

[0341] Specific operation: The user takes photos of the disaster site with a camera and collects environmental data from sensors.

[0342] Output: The dataset entered on the terminal.

[0343] Step 8:

[0344] The device preprocesses the data.

[0345] Input: Data collected in step 7.

[0346] Specific operation: The device performs preprocessing such as noise removal and resolution adjustment.

[0347] Output: Preprocessed data.

[0348] Step 9:

[0349] The device analyzes the preprocessed data.

[0350] Input: The data preprocessed in step 8 and the latest artificial intelligence model installed on the device.

[0351] Specific operation: The device makes an API call, inputs the preprocessed data into the model, and begins analysis.

[0352] Output: Analysis results (e.g., identification of high-risk areas, assessment of damage scale).

[0353] Step 10:

[0354] The terminal provides the analysis results to the user.

[0355] Input: Analysis results obtained in step 9.

[0356] Specific operation: The device visualizes and displays the analysis results on a display or other user interface.

[0357] Output: The analytical information that is presented visually to the user.

[0358] Step 11:

[0359] The device uses an emotion engine to collect and analyze emotion data.

[0360] Input: User facial and voice data obtained from sensors such as cameras and microphones.

[0361] Specific operation: The device runs an emotion analysis algorithm based on sensor data to determine the user's emotional state.

[0362] Output: Data indicating the user's emotional state (e.g., stress level, joy, anger).

[0363] Step 12:

[0364] The user receives emotional feedback.

[0365] Input: Emotion data obtained in step 11 and analysis results provided in step 10.

[0366] Specific operation: The device displays appropriate feedback (e.g., suggestions for rest, encouraging messages) based on the user's emotional state.

[0367] Output: The feedback message provided to the user.

[0368] This is the specific processing flow of the system's program. This enables advanced data analysis and appropriate feedback based on the user's emotions even in unstable communication environments, resulting in more effective and efficient decision-making.

[0369] (Application example 2)

[0370] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0371] In many brick-and-mortar stores today, it is difficult to understand customers' emotions and states in real time and provide appropriate customer service. In particular, in environments with unstable communication lines or when no connection is available, it is difficult to provide advanced support using artificial intelligence technology. Therefore, while there is a demand for improved customer satisfaction and effective customer support, current systems are unable to adequately address these challenges.

[0372] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0373] In this invention, the server includes means for storing a miniaturized artificial intelligence model in a storage medium, means for transmitting the latest artificial intelligence model via a communication line, means for analyzing collected information in real time using the artificial intelligence model stored in the storage medium and providing the analysis results to the user, and means for using an emotion engine that analyzes the user's emotional state to provide emotion-based feedback and suggestions. This makes it possible to analyze the emotional state of customers in real time and provide appropriate customer service support even in an environment where communication lines are unstable, such as in a physical store.

[0374] "Multi-location information processing" refers to the collection of information at multiple physical or virtual locations and the integration and processing of that information.

[0375] A "miniaturized artificial intelligence model" refers to an artificial intelligence model that is smaller than a general artificial intelligence model and is designed to be able to operate with limited resources.

[0376] "Storage medium" refers to a device or medium for permanently or temporarily storing data.

[0377] "Communication Line" refers to the physical or wireless network infrastructure for transmitting and receiving data.

[0378] "Means for transmitting the latest artificial intelligence model" refers to the mechanism by which the updated AI model is transferred to other devices over the network.

[0379] "Means for analyzing information in real time" refers to mechanisms for immediately processing collected data and providing the results to users immediately.

[0380] An "emotion engine that analyzes the user's emotional state" refers to a system that uses emotion recognition technology to identify emotions from the user's facial expressions and voice.

[0381] "Means for providing emotion-based feedback and suggestions" refers to mechanisms that provide appropriate advice or instructions for action based on the user's emotional state.

[0382] A "physical store" refers to a physical sales or service location that customers can visit in person.

[0383] "Customer service support" refers to systems and means that assist sales staff and service providers in the customer service activities they provide to customers.

[0384] MODE FOR CARRYING OUT THE INVENTION

[0385] To implement this invention, the following system configuration and procedures are required.

[0386] System Configuration

[0387] Hardware

[0388] 1. Device: Smart glasses (with built-in camera and display).

[0389] 2. Server: A server with high-performance computing resources.

[0390] software

[0391] 1. Artificial intelligence model: The miniaturized AI model is stored on the device's storage media.

[0392] 2. Data analysis engine: Uses OpenCV and DeepFace to perform real-time analysis of collected data.

[0393] 3. Emotion Engine: Uses emotion recognition technology based on DeepFace.

[0394] 4. Communication Protocol: The communication protocol to support data communication between the terminal and the server.

[0395] Feedback of results

[0396] 1. Analysis results: The device analyzes the collected data and displays the analysis results on the screen in real time.

[0397] 2. Emotional Feedback: The emotion engine identifies the user's emotional state and provides contextual feedback and suggestions.

[0398] 3. Model update from the server: The server sends the latest artificial intelligence model to the terminal, and the terminal stores the model in its storage medium.

[0399] Operation explanation

[0400] The device uses the smart glasses' camera to capture the facial expressions and behavior of customers visiting a physical store. The captured data is processed in real time using the device's data analysis engine (OpenCV, DeepFace). Based on the analysis results, the customer's emotional state is identified. For example, DeepFace can recognize emotions such as "happiness," "sadness," and "surprise" from the customer's facial image.

[0401] The emotion recognition results from the emotion engine are fed back to the user (store clerk), and appropriate customer service methods (e.g., "If the customer seems anxious, explain gently") are suggested.

[0402] The server also periodically trains the latest AI model and sends it to the terminal, which then stores the received model in a storage medium, contributing to improving analysis accuracy.

[0403] Specific examples

[0404] For example, consider a scenario in which a customer is being served by a sales clerk wearing smart glasses in a brick-and-mortar store. The emotion of "anxiety" is recognized based on the customer's facial expression captured by the camera. In this case, the sales clerk's display will recommend "speaking kindly and explaining the product." This will enable the sales clerk to provide appropriate service to the customer, improving customer satisfaction.

[0405] Prompt Sentence Examples

[0406] Predefine the user's emotional state and enter it in text format, such as:

[0407] user_emotion: "anxiety"

[0408] customer_behavior: "looking at products"

[0409] Based on this, the generative AI model provides the following feedback:

[0410] Suggestion: "They may ask you to explain this product, so be gentle."

[0411] This system makes it possible to analyze customers' emotional states in real time and provide appropriate customer support, even in physical stores where communication lines are unstable.

[0412] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0413] Step 1:

[0414] The server trains the latest artificial intelligence models and sends model updates to the devices.

[0415] Input: Training data collected by the server.

[0416] Data processing and computation: The server uses high-performance computing resources to train new artificial intelligence models, then converts the trained models into portable AI memory.

[0417] Output: The latest AI model sent to the device.

[0418] Specific operation: The server trains the latest model according to a regular schedule and sends update data to the terminal via the network.

[0419] Step 2:

[0420] The terminal receives the latest artificial intelligence model from the server and stores it in a storage medium.

[0421] Input: The latest AI model data sent from the server.

[0422] Data processing and calculation: The device downloads the received model data to its internal memory and installs it automatically.

[0423] Output: The latest AI model stored on a storage medium.

[0424] Specific operation: The device monitors the network connection status, and when it receives a model update notification from the server, it automatically downloads and installs the data.

[0425] Step 3:

[0426] A user (store clerk) collects facial expression data of customers using the camera in the smart glasses.

[0427] Input: Customer face image.

[0428] Data processing and computation: Collecting image data captured by the smart glasses camera and converting it into a format suitable for analysis.

[0429] Output: Facial image data in a format suitable for analysis.

[0430] How it works: A store clerk wears smart glasses, and a camera automatically captures facial expression data while interacting with customers.

[0431] Step 4:

[0432] The device analyzes the collected facial expression data to identify the emotional state.

[0433] Input: Collected customer facial image data.

[0434] Data processing and computation: Using OpenCV and DeepFace, image data is analyzed in real time to identify the customer's emotional state (e.g., joy, anxiety, surprise, etc.).

[0435] Output: Customer emotional state data.

[0436] How it works: Facial image data is input into the analysis engine, DeepFace analyzes and identifies the emotional state, and displays the results on the device.

[0437] Step 5:

[0438] The emotion engine generates appropriate feedback and suggestions based on the emotional state identified.

[0439] Input: Customer emotional state data.

[0440] Data processing and computation: The emotion engine generates pre-defined feedback and suggestions based on the captured emotional state.

[0441] Output: Feedback regarding specific ways to respond to store staff and customer support.

[0442] Specific behavior: If the emotion engine identifies "anxiety," it will display recommended actions, such as "explain things gently" to the store clerk, on the smart glasses display.

[0443] Step 6:

[0444] The user (store clerk) provides appropriate customer service based on the analysis results and feedback.

[0445] Input: Analysis results and feedback provided by the device.

[0446] Data processing and calculation: The store clerk uses the analysis results and feedback to provide appropriate support to the customer.

[0447] Output: Hospitality and product suggestions to customers.

[0448] Specific actions: The store clerk follows the instructions of the smart glasses and provides customer service by speaking to them in a friendly manner and explaining the products in detail.

[0449] Step 7:

[0450] The server periodically collects new training data and continuously trains the model to improve its accuracy.

[0451] Input: Emotion recognition data and customer service outcome data collected in physical stores.

[0452] Data processing and calculation: The server uses the collected data to retrain the AI ​​model to improve its accuracy.

[0453] Output: Improved AI model data.

[0454] Specific operation: The server periodically analyzes the data collected from the physical store and improves the model. As a result, a new AI model is distributed to the device again.

[0455] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0456] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0457] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0458] [Second embodiment]

[0459] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0460] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0461] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0462] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0463] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0464] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0465] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0466] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0467] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0468] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0469] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0470] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0471] The present invention relates to a system that stores a miniaturized artificial intelligence model in a storage medium and performs real-time data analysis even in an unstable or unavailable communication environment. Specific examples of the system based on the present invention are described in detail below.

[0472] 1. The server updates the AI ​​model

[0473] The server collects the latest training data and optimizes the AI ​​model. After training a new AI model using the training data, the server converts it into a portable AI memory format and transmits it to the device via the network. Therefore, the server uses its high-performance computing resources to efficiently generate and distribute AI models.

[0474] 2. The device receives the AI ​​model

[0475] The device receives the latest AI model update notification from the server and downloads the model via the network. After downloading, the device stores the new model in its internal memory and installs it. This ensures that the device always has the latest AI model.

[0476] 3. Users collect data

[0477] Users collect necessary data in remote locations or disaster sites. The collected data is diverse, including images, videos, audio, and sensor data. This data is input into a terminal and stored on a storage medium. Users use data collection tools (e.g., cameras, sensors) to efficiently collect data.

[0478] 4. The device analyzes the data

[0479] The device uses the AI ​​model stored on the storage medium to perform real-time data analysis based on data provided by the user. As data is input, the device preprocesses the data and converts it into a suitable format for analysis. From there, the AI ​​model begins its analysis and generates results. The analysis results are provided to the user via a user interface or display device.

[0480] 5. Users consume the results

[0481] Users who receive the analysis results can make decisions in real time based on that information. For example, researchers in remote locations can use the analysis results to determine the direction and next steps of their research, allowing for more efficient research. Disaster relief teams can also use the analysis results to carry out rapid and effective rescue operations. As a result, highly accurate data analysis and rapid decision-making are possible even in situations where communication is unstable or unavailable.

[0482] Specific examples

[0483] Example 1: Disaster Relief Scenario

[0484] In a disaster site where communication networks are unavailable, disaster relief teams collect images and audio to assess the damage. The collected data is input into a portable AI memory and instantly analyzed by an artificial intelligence model. The analysis results include the scale of damage, high-risk areas, and optimal rescue routes. Rescue teams use this information to carry out rapid and effective rescue operations.

[0485] Example 2: Remote research scenario

[0486] Researchers studying new species in remote locations input data collected on-site into a portable AI memory. The AI ​​model analyzes the input data in real time and provides insights into the characteristics and habitat of the new species. This allows researchers to decide on the next research direction on the spot and advance their research more efficiently.

[0487] The above is a detailed description of one embodiment of the present invention, which enables advanced data analysis and rapid decision-making even in an unstable communication environment.

[0488] The processing flow will be explained below.

[0489] Step 1:

[0490] The server collects the latest training data.

[0491] The server acquires weather data, earthquake data, and other related data and stores it in a database.

[0492] Step 2:

[0493] The server preprocesses the acquired training data.

[0494] It removes noise, normalizes, and fills missing values, converting the data into a format suitable for training AI models.

[0495] Step 3:

[0496] The server trains the artificial intelligence model.

[0497] Apply machine learning algorithms to build AI models and train them using the acquired training data.

[0498] Step 4:

[0499] The server converts the artificial intelligence model after training into a portable AI memory device.

[0500] The model is optimized, compressed, converted to a memory-compatible format, and saved.

[0501] Step 5:

[0502] The server sends the latest AI model to the device via the network.

[0503] The trained AI model is encoded and data transmission is performed using a secure communication method.

[0504] Step 6:

[0505] The device detects an update notification for the AI ​​model received from the server.

[0506] It periodically checks for update notifications from the server and starts receiving them if a new model is available.

[0507] Step 7:

[0508] The device downloads the new AI model.

[0509] The AI ​​model is downloaded via the network and quickly saved to the device.

[0510] Step 8:

[0511] The device will install the new AI model.

[0512] It unpacks the saved AI model into memory, deletes the previous model, and installs the new one.

[0513] Step 9:

[0514] Users collect data at disaster sites and remote locations.

[0515] The necessary data (images, audio, sensor data, etc.) is collected using cameras, sensors, etc. and input into the terminal.

[0516] Step 10:

[0517] The terminal receives and pre-processes data provided by the user.

[0518] Check the format of the input data and remove outliers and noise to make it analyzable.

[0519] Step 11:

[0520] The device inputs the preprocessed data into an AI model for real-time analysis.

[0521] The AI ​​model begins the analysis and generates analytical results.

[0522] Step 12:

[0523] The terminal obtains the analysis results and provides them to the user through a user interface.

[0524] The results of the analysis are then formatted appropriately and presented to the user via a display or other output device.

[0525] Step 13:

[0526] Users make decisions based on the analysis results.

[0527] Based on the analysis results received, we will formulate and implement research directions and rescue operation plans.

[0528] Example 1

[0529] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0530] Real-time data analysis is required in situations where data processing is required at multiple locations, such as in medical care, disaster relief, or remote research. However, in these situations, communication environments are often unstable or unavailable, resulting in delays in data analysis and processing. In addition, because the amount of information transmitted is large, efficient processing on the terminal side is also important. Conventional methods have made it difficult to make quick decisions under such circumstances.

[0531] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0532] In this invention, the server includes means for using high-performance computing resources to train an AI model based on the latest training data, converting it for portable use, and transmitting it to the terminal, means for the terminal to receive update notifications from the server and download and install new AI models, means for the user to input various data collected using a data collection tool into the terminal and store it in a storage medium, means for the terminal to preprocess the data and analyze it in real time using the AI ​​model stored in the storage medium, and provide the user with the analysis results, and means for the user to make decisions based on the analysis results. This enables fast and accurate data analysis and efficient decision-making even in situations where the communication environment is unstable or unavailable.

[0533] "High-performance computing resources" refers to hardware and software for efficiently and quickly processing data and performing calculations, and specifically includes high-performance GPUs and servers.

[0534] "Training data" refers to various types of data used to train artificial intelligence models, including, for example, image data and text data.

[0535] "Artificial intelligence model" refers to a model designed using machine learning algorithms and trained to perform specific tasks automatically.

[0536] "Converting for portable use" refers to converting a trained artificial intelligence model into a format that can run efficiently on a device that processes data.

[0537] "Update Notification" refers to a message sent from a server to a device informing the device that a newer version of an artificial intelligence model is available.

[0538] "Downloading and installing" refers to the device receiving (downloading) a new artificial intelligence model from the server and setting it up for use (installing).

[0539] "Data collection tools" refers to various devices and sensors that users use to collect data, including cameras and thermometers.

[0540] "Real-time analysis" refers to analyzing data as soon as it is entered and generating results.

[0541] "Providing the analysis results to the user" refers to the terminal presenting the information obtained by the analysis to the user through a user interface or display device.

[0542] "Making a decision" refers to a user deciding on a future course of action or strategy based on the information provided.

[0543] "Unstable or unavailable network environment" refers to a situation where the network connection is of poor quality or completely unavailable.

[0544] This invention relates to a system that stores a miniaturized artificial intelligence model on a storage medium and performs real-time data analysis even in situations where communication is unstable or unavailable. This system is realized through the cooperation of a server, terminals, and users.

[0545] server

[0546] The server uses high-performance computing resources (e.g., NVIDIA Tesla V100 GPU) to train an artificial intelligence model based on the latest training data. The server collects the necessary training data from databases and cloud storage and optimizes the model using machine learning algorithms. After training is complete, the server uses specific tools such as TensorFlow Lite Converter to convert the model for portable use. The converted model is then transmitted to the device via a network (e.g., 5G communication). A secure communication protocol (e.g., SSL / TLS) is used for this transmission.

[0547] Terminal

[0548] The device receives update notifications from the server and downloads and installs the new AI model. Update notifications are sent using protocols such as HTTP / 2. After downloading, the device stores the new model in its internal memory (e.g., an SD card) and installs it in a usable state. The device uses the downloaded model to analyze data provided by the user in real time.

[0549] User

[0550] Users collect the necessary data using data collection tools (e.g., cameras, sensors). For example, at disaster sites, they collect photos of the damage and sensor data. The collected data is entered into the device and stored in the internal memory. Users use data collection tools to efficiently collect data.

[0551] Data analysis

[0552] The device performs preprocessing such as noise removal and normalization on the data provided by the user. This preprocessing improves the accuracy of the analysis. The device then analyzes the preprocessed data in real time using an artificial intelligence model. The analysis results are provided to the user as formatted data (e.g., JSON format), and the user can view the results on the device's display or through a dedicated application.

[0553] Specific examples

[0554] Example 1: Disaster Relief Scenario

[0555] When communication networks are unavailable at disaster sites, disaster relief teams collect images and audio to assess the damage situation. The collected data is input into a terminal, and an artificial intelligence model immediately analyzes it. For example, a prompt might be used, such as, "Please analyze the damage situation using this image data." The analysis results include the scale of damage, high-risk areas, and optimal rescue routes. Rescue teams use this information to carry out prompt and effective rescue operations.

[0556] Example 2: Remote research scenario

[0557] Researchers studying new species in remote locations input data collected on-site into a terminal, using a prompt such as, "Analyze the collected biological data and determine whether it is a new species." The AI ​​model analyzes the input data in real time and provides insights into the characteristics and habitat of the new species. This allows researchers to decide on the next research direction on the spot and advance their research more efficiently.

[0558] This system enables rapid and accurate data analysis and efficient decision-making even in situations where communication is unstable or unavailable, and is expected to have a wide range of applications, including disaster relief and remote research.

[0559] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0560] The flow of this system's program processing

[0561] Step 1: Server collects training data

[0562] The server collects training data from various data sources (e.g., databases, cloud storage). The input is raw data (e.g., image data, text data) obtained from multiple data sources and stores this data in local storage. Specific operations include executing API calls and database queries.

[0563] Step 2: The server trains the AI ​​model

[0564] The server uses a high-performance GPU (e.g., NVIDIA Tesla V100) to train an AI model based on the collected training data. The input is the stored training data, and the output is an optimized AI model. Specifically, the training process is performed using a machine learning library (e.g., TensorFlow, PyTorch).

[0565] Step 3: The server converts the AI ​​model for portable use

[0566] The server converts the trained AI model into a format that can be used on portable devices (e.g., ONNX format). The input is the trained AI model, and the output is an AI model converted for portable devices. Specific operations involve using tools such as TensorFlow Lite Converter.

[0567] Step 4: The server sends the model to the device

[0568] The server sends the converted AI model to the device via a network (e.g., 5G communication). The input is the AI ​​model converted for the portable device, and the output is the model data to be sent. Specifically, data is transmitted using a secure communication protocol (e.g., SSL / TLS).

[0569] Step 5: Your device receives an update notification

[0570] The terminal receives an update notification from the server. The input is a notification message from the server, and the output is that the terminal recognizes the model update. The specific operation is to notify using the HTTP / 2 protocol.

[0571] Step 6: Your device will download the new model

[0572] After the device confirms the update notification, it downloads the new model file from the server. The input is the model data from the server, and the output is the downloaded model file. The specific operation is to execute the file download process.

[0573] Step 7: The device installs the model

[0574] The device stores the downloaded model in its internal memory (e.g., SD card) and installs it. The input is the downloaded model file, and the output is the installed model. Specific operations include unpacking and deploying the model file.

[0575] Step 8: User prepares data collection tools

[0576] The user prepares a data collection tool, such as a camera (e.g., a high-resolution camera) or a sensor (e.g., an environmental sensor). The input is the readiness of the tool to be used, and the output is that the data collection tool is ready for use. Specific actions include calibrating the device and checking the battery.

[0577] Step 9: Collect the required data

[0578] The user uses the prepared tools to collect images, videos, audio, sensor data, etc. The input is the raw data obtained from the collection tool, and the output is the collected data. Specific actions include taking photos and recording audio.

[0579] Step 10: User enters data into terminal

[0580] The collected data is connected to a terminal and transferred to the internal memory. The input is the collected data, and the output is the data stored in the terminal. Specific operations include USB connection and wireless communication (e.g., Bluetooth).

[0581] Step 11: The device preprocesses the data

[0582] The device receives data provided by the user and performs preprocessing such as noise removal and normalization. The input is unprocessed data and the output is preprocessed data. Specific operations include adjusting the image resolution and filtering audio data.

[0583] Step 12: The device analyzes the data using the AI ​​model

[0584] The device uses an AI model to perform real-time analysis based on the preprocessed data. The input is the preprocessed data, and the output is the analysis results. Specific operations include image recognition and voice analysis.

[0585] Step 13: The device generates the analysis results

[0586] Once the analysis is complete, the terminal generates the results and outputs them as formatted data (e.g., JSON format). The input is the analysis result, and the output is the formatted result data. The specific operation is to convert the results into a data format.

[0587] Step 14: User receives analysis results

[0588] The user receives the analysis results from the terminal's display device or a dedicated application. The input is the formatted result data, and the output is the result displayed to the user. The specific operation is to display the results on the display device.

[0589] Step 15: User makes decision based on results

[0590] The user makes a decision based on the analysis results. The input is the displayed analysis results, and the output is the decision content. The specific action is to determine the optimal response depending on the situation.

[0591] The above is the specific flow of the program processing of this system.

[0592] (Application example 1)

[0593] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0594] In recent years, advances in autonomous driving technology have generated much expectation for autonomous vehicles, such as reducing traffic accidents and easing traffic congestion. However, when communication networks are unstable, necessary data analysis can be delayed, making it impossible to ensure safety. To address this issue, a new system capable of performing data analysis with high accuracy and in real time is required.

[0595] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0596] In this invention, the server includes means for storing a miniaturized artificial intelligence model in a storage medium, means for transmitting the latest artificial intelligence model via a network to periodically update the artificial intelligence model stored in the storage medium, means for analyzing collected data in real time using the artificial intelligence model stored in the storage medium and providing the analysis results to a user, and means for processing video data acquired from a camera mounted on the vehicle in real time even in an unstable communication environment to detect the presence or absence of obstacles and lane deviations, thereby enabling autonomous vehicles to operate safely and effectively even in situations where the communication network is unstable or unavailable.

[0597] A "miniaturized artificial intelligence model" is an artificial intelligence model that has been optimized and scaled down so that it can operate in environments with limited computing resources and memory capacity.

[0598] "Storage media" means any device or material for storing data electronically or magnetically.

[0599] A "network" is a system or infrastructure that connects multiple devices and enables data communication.

[0600] "Data" means any form of information, such as information, signals, or measurements, that is collected, stored, or analyzed.

[0601] "Real-time" means that input data is processed and analyzed almost as soon as it is generated.

[0602] A "user" is someone who uses or benefits from the use of a system or device.

[0603] A "vehicle-mounted camera" is a device installed in a vehicle to capture images of its surroundings.

[0604] "Video data" is digital data containing visual information captured by a camera.

[0605] "Processing" is a series of operations performed on data to manipulate and analyze it and obtain useful information or results.

[0606] An "obstacle" is any object or obstacle that may impede the vehicle's progress.

[0607] "Lane deviation" refers to a state in which the vehicle's current direction of travel deviates from the lane of the road.

[0608] "Discovery" is the act of finding data or situations that meet specified conditions or characteristics.

[0609] The server collects the latest training data and optimizes the AI ​​model. It uses high-performance computing resources to train a new AI model and converts it for portable AI memory. It then sends it to the device over the network, allowing the device to always have the latest AI model. The main software used is Keras and TensorFlow, and a high-performance server (e.g., a server with a GPU) is required for hardware.

[0610] The device receives update notifications from the server, downloads new AI models, and stores them in its internal memory. All the device requires is a network connection and local storage, which can be found on common devices such as smartphones and tablets.

[0611] The user collects the necessary data (video data) using a camera mounted on the vehicle. This video data is input into the terminal and saved in a storage medium. The collected video data is preprocessed using software such as Keras and PIL.

[0612] The device then analyzes the data in real time using an artificial intelligence model stored on the storage media. Preprocessing of the video data includes standardizing and normalizing the image size, converting the data into a format that the model can use to achieve optimal results. Analysis is performed using a high-performance processor (e.g., Apple A14 Bionic) and software such as Keras and TensorFlow.

[0613] The analysis results include information such as the presence or absence of obstacles and lane deviations. These results are provided to the user via the device's user interface. Based on this information, the user can make decisions in real time to ensure safe autonomous driving.

[0614] Examples:

[0615] This autonomous driving support AI assistant analyzes images from a camera in front of the vehicle in real time while driving on the highway. For example, even if communication is lost, it can detect pedestrians or obstacles around the vehicle and issue a warning or make an emergency stop. The autonomous vehicle uses this information to decide its next course of action.

[0616] Example prompts to input to a generative AI model:

[0617] "Detect pedestrians in real time from the forward camera image and display the results."

[0618] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0619] Step 1:

[0620] The server trains the latest artificial intelligence models and converts them for portable AI memory.

[0621] Input: High-performance computing resources (e.g., GPU-equipped servers), up-to-date training data

[0622] Output: Optimized artificial intelligence model

[0623] How it works: The server uses Keras and TensorFlow to extract features from the collected dataset, train the model, and optimize the weights. It then converts the optimized model into a portable format (e.g., .h5 file).

[0624] Step 2:

[0625] The server sends the artificial intelligence model to the terminal.

[0626] Input: Optimized artificial intelligence model, network connection

[0627] Output: AI model downloaded to device

[0628] Specific operation: The server sends the optimized model to the device using the HTTP protocol, etc. The device downloads the model via the network and stores it in its internal memory.

[0629] Step 3:

[0630] A user collects video data using a camera mounted on a vehicle.

[0631] Input: Vehicle-mounted camera

[0632] Output: Camera video data

[0633] Specific operation: The camera starts up at the timing and conditions specified by the user and continuously captures images of the area in front of the vehicle. This image data is collected frame by frame and transferred to the terminal.

[0634] Step 4:

[0635] The device preprocesses the video data and converts it into an analyzable data format.

[0636] Input: Camera video data

[0637] Output: Pre-processed video data

[0638] Specific operation: The device uses the Python Image Library (PIL) and Keras to perform preprocessing such as standardizing the image size (e.g., resizing to 224x224 pixels) and normalizing (scaling pixel values ​​to the 0-1 range).

[0639] Step 5:

[0640] The device uses artificial intelligence models to analyze the pre-processed data in real time.

[0641] Input: Preprocessed video data, artificial intelligence model

[0642] Output: Analysis results (presence or absence of obstacles, lane deviation, etc.)

[0643] Specific operation: The device uses Keras and TensorFlow to input preprocessed data into an artificial intelligence model and make predictions to detect the presence or absence of obstacles and lane deviations. The model's output includes specific classification results and location information.

[0644] Step 6:

[0645] The terminal provides the analysis results to a user interface.

[0646] Input: Analysis results

[0647] Output: A visual or audio notification provided to the user

[0648] How it works: The device provides the user with real-time analysis results via a user interface (e.g., on-screen alerts, voice guidance system), allowing the user to make driving decisions based on this information.

[0649] Example prompts based on concrete examples:

[0650] "Detect pedestrians in real time from the forward camera image and display the results."

[0651] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0652] The present invention relates to a system that stores a miniaturized artificial intelligence model on a storage medium and performs real-time data analysis even in situations where communication is unstable or unavailable. Another feature of the present invention is that it provides feedback and suggestions based on the user's state by combining it with an emotion engine that recognizes the user's emotions. A specific example of a system based on the present invention is described in detail below.

[0653] 1. The server updates the AI ​​model

[0654] The server collects the latest training data and optimizes the AI ​​model. After training a new AI model using the training data, the server converts it into a portable AI memory format and transmits it to the device via the network. Therefore, the server uses its high-performance computing resources to efficiently generate and distribute AI models.

[0655] 2. The device receives the AI ​​model

[0656] The device receives the latest AI model update notification from the server and downloads the model via the network. After downloading, the device stores the new model in its internal memory and installs it. This ensures that the device always has the latest AI model.

[0657] 3. Users collect data

[0658] Users collect necessary data in remote locations or disaster sites. The collected data is diverse, including images, videos, audio, and sensor data. This data is input into a terminal and stored on a storage medium. Users use data collection tools (e.g., cameras, sensors) to efficiently collect data.

[0659] 4. The device analyzes the data

[0660] The device uses the AI ​​model stored on the storage medium to perform real-time data analysis based on data provided by the user. As data is input, the device preprocesses the data and converts it into a suitable format for analysis. From there, the AI ​​model begins its analysis and generates results. The analysis results are provided to the user via a user interface or display device.

[0661] 5. Emotion Recognition by Emotion Engine

[0662] The device is equipped with an emotion engine that recognizes the user's emotions. This engine uses data acquired from sensors such as cameras and microphones to analyze the user's facial expressions, voice, and behavioral patterns. Based on this data, the emotion engine identifies the user's emotional state in real time and provides the user with the analysis results.

[0663] 6. Users consume the results and receive emotional feedback

[0664] Users can then make real-time decisions based on the analysis results. The emotion engine provides appropriate feedback and suggestions based on the user's emotional state. For example, if the stress level is high, it will suggest relaxation techniques and rest. If the user is in a positive emotional state, it will immediately display encouraging messages to take the next step.

[0665] Specific examples

[0666] Example 1: Disaster Relief Scenario

[0667] When communication networks are unavailable at a disaster site, disaster relief teams collect images and audio to assess the damage. The collected data is input into a portable AI memory and instantly analyzed by an artificial intelligence model. The analysis results include the scale of damage, high-risk areas, and optimal rescue routes. At the same time, an emotion engine analyzes the emotional state of the rescue team and suggests appropriate rest if stress levels are high. The rescue team then uses this information to carry out prompt and effective rescue operations.

[0668] Example 2: Remote research scenario

[0669] Researchers studying new species in remote locations input data collected on-site into a portable AI memory. The AI ​​model analyzes the input data in real time and provides insights into the characteristics and habitat of the new species. The emotion engine assesses the researcher's emotional state from their facial expressions and voice, and sends encouraging messages or suggests rest as needed. This allows researchers to decide on the next research direction on the spot and advance their research more efficiently.

[0670] This concludes the detailed description of one embodiment of the present invention, which enables advanced data analysis and appropriate feedback based on the user's emotions even in unstable communication environments, resulting in more effective and efficient decision-making.

[0671] The processing flow will be explained below.

[0672] Step 1:

[0673] The server collects the latest training data.

[0674] The server acquires weather data, earthquake data, and other related data and stores it in a database.

[0675] Step 2:

[0676] The server preprocesses the acquired training data.

[0677] It removes noise, normalizes, and fills missing values, converting the data into a format suitable for training AI models.

[0678] Step 3:

[0679] The server trains the artificial intelligence model.

[0680] Apply machine learning algorithms to build AI models and train them using the acquired training data.

[0681] Step 4:

[0682] The server converts the artificial intelligence model after training into a portable AI memory device.

[0683] The model is optimized, compressed, converted to a memory-compatible format, and saved.

[0684] Step 5:

[0685] The server sends the latest AI model to the device via the network.

[0686] The trained AI model is encoded and data transmission is performed using a secure communication method.

[0687] Step 6:

[0688] The device detects an update notification for the AI ​​model received from the server.

[0689] It periodically checks for update notifications from the server and starts receiving them if a new model is available.

[0690] Step 7:

[0691] The device downloads the new AI model.

[0692] The AI ​​model is downloaded via the network and quickly saved to the device.

[0693] Step 8:

[0694] The device will install the new AI model.

[0695] It unpacks the saved AI model into memory, deletes the previous model, and installs the new one.

[0696] Step 9:

[0697] Users collect data at disaster sites and remote locations.

[0698] The necessary data (images, audio, sensor data, etc.) is collected using cameras, sensors, etc. and input into the terminal.

[0699] Step 10:

[0700] The terminal receives and pre-processes data provided by the user.

[0701] Check the format of the input data and remove outliers and noise to make it analyzable.

[0702] Step 11:

[0703] The device inputs the preprocessed data into an AI model for real-time analysis.

[0704] The AI ​​model begins the analysis and generates analytical results.

[0705] Step 12:

[0706] The terminal obtains the analysis results and provides them to the user through a user interface.

[0707] The results of the analysis are then formatted appropriately and presented to the user via a display or other output device.

[0708] Step 13:

[0709] The device analyzes the user's facial expressions and voice using an emotion engine.

[0710] The device uses data from the camera and microphone to determine the user's emotional state.

[0711] Step 14:

[0712] The device provides feedback and suggestions based on the emotional state it identifies.

[0713] If the user is feeling stressed, it will suggest ways to relax, and if they are feeling positive, it will display an encouraging message.

[0714] Step 15:

[0715] Users use emotion-based feedback to make decisions.

[0716] Choose the appropriate action based on your mental state and move on to the next step.

[0717] Example 2

[0718] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0719] Conventional systems have difficulty analyzing data in unstable communication environments, which can delay real-time decision-making based on collected data. Furthermore, they do not provide feedback or suggestions that take into account the user's emotional state, hindering efficient activities.

[0720] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for storing a miniaturized artificial intelligence model in a storage medium for multi-site data processing, means for transmitting the latest artificial intelligence model via a network to periodically update the artificial intelligence model stored in the storage medium, means for analyzing collected data in real time using the artificial intelligence model stored in the storage medium and providing the analysis results to the user, means for performing real-time emotion analysis using data acquired from a sensor to recognize the user's emotional state, and means for providing appropriate feedback and suggestions based on the user's emotional state. This enables advanced data analysis and appropriate feedback based on the user's emotions even in an unstable communication environment, thereby realizing more effective and efficient decision-making.

[0721] "Miniaturized AI models" refer to AI algorithms or neural networks that are reduced in size and designed to run efficiently on portable devices.

[0722] "Storage media" refers to computer hardware components used for long-term data storage, including HDDs, SSDs, and USB memory sticks.

[0723] "Network" refers to the infrastructure for data communication between multiple computers and devices, and specifically includes Wi-Fi, 4G / 5G, Ethernet, etc.

[0724] "Data analytics" refers to the set of processes that are undertaken to process collected data and extract meaningful information and insights.

[0725] "Means for providing to the user" refers to an interface or mechanism that visually or audibly conveys analysis results and feedback information to the user.

[0726] "Emotion analysis" refers to a set of algorithms and methods for identifying a user's emotional or psychological state based on data obtained from sensors.

[0727] "Means for providing feedback and suggestions" refers to a system or interface that provides information to encourage or motivate a user to take appropriate action based on the user's emotional state or the results of data analysis.

[0728] The present invention relates to a system that stores a miniaturized artificial intelligence model on a storage medium and performs real-time data analysis even in unstable or unavailable communication environments. Another feature of the present invention is that it combines an emotion engine that recognizes the user's emotions to provide feedback and suggestions based on the user's state.

[0729] The system is implemented using the following hardware and software.

[0730] Update of artificial intelligence models by the server

[0731] The server optimizes the AI ​​model using high-performance computing resources. Specifically, it utilizes deep learning frameworks such as TensorFlow and PyTorch using GPUs on cloud services (e.g., Google Cloud Platform and Amazon Web Services). The server periodically collects the latest training data (e.g., image and audio data) and uses this data to train the AI ​​model. The server then converts the model into an optimal format for portable AI memory (e.g., ONNX) and transmits it to the device via a network (e.g., Wi-Fi, 4G / 5G).

[0732] Receiving artificial intelligence models and analyzing data on the device

[0733] The terminal receives the latest AI model update notification from the server and downloads the new model via the network. After downloading, the terminal stores the model in its internal memory and installs it. The terminal is a portable device such as a smartphone or laptop, and performs real-time data analysis based on the received model. Based on the data collected by the user (e.g., images, audio, sensor data), the data is preprocessed and converted into an appropriate format for analysis, and the AI ​​model analyzes the data. The analysis results are provided to the user via a user interface (e.g., a display).

[0734] Emotion recognition and feedback by emotion engine

[0735] The device is equipped with sensors such as a camera and microphone, which are used to collect the user's emotional data. The emotion engine uses this data to analyze the user's facial expressions, voice, and behavioral patterns to identify the user's emotional state in real time. Based on the analysis results, it provides appropriate feedback to the user. For example, if the user's stress level is high, it suggests taking a break, and if the user is in a positive emotional state, it displays an encouraging message to take the next step.

[0736] Prompt Sentence Examples

[0737] Example 1: Disaster Relief Scenario

[0738] A disaster relief team took 100 images at the scene and entered them into a mobile device. Explain how the device's AI model analyzed the extent of the damage, identified high-risk areas, and suggested rest based on stress levels.

[0739] Example 2: Remote research scenario

[0740] A researcher remotely collects photos of 30 new plant species and inputs them into a device. The device's AI model analyzes the data, provides characteristics of the new species, and an emotion engine detects when the researcher is fatigued and provides appropriate feedback. Explain the process.

[0741] This concludes the detailed description of one embodiment of the present invention, which enables advanced data analysis and appropriate feedback based on the user's emotions even in unstable communication environments, resulting in more effective and efficient decision-making.

[0742] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0743] Step 1:

[0744] A server collects training data.

[0745] Input: Large amounts of image and audio data stored in cloud storage (e.g., Amazon S3).

[0746] Specific operation: The server downloads the training data via the cloud API.

[0747] Output: Training data saved in local storage.

[0748] Step 2:

[0749] The server trains the artificial intelligence model.

[0750] Input: The training data collected in step 1.

[0751] What it does: The server trains a neural network using a deep learning framework such as TensorFlow or PyTorch.

[0752] Output: A fully trained artificial intelligence model.

[0753] Step 3:

[0754] The server converts the model into a portable format.

[0755] Input: The artificial intelligence model trained in step 2.

[0756] Specific operation: The server converts the model into a compatible format such as ONNX.

[0757] Output: An artificial intelligence model in a portable format.

[0758] Step 4:

[0759] The server sends the model to the device.

[0760] Input: The artificial intelligence model in portable format converted in step 3.

[0761] Specific operation: The server uploads the model to the device via the network (e.g., Wi-Fi, 5G) using the HTTP or FTP protocol.

[0762] Output: The artificial intelligence model downloaded to the device.

[0763] Step 5:

[0764] Your device will receive an update notification.

[0765] Input: Update notifications sent by the server.

[0766] Specific behavior: The device receives a push notification and checks for the existence of a new model.

[0767] Output: Update notification reception status.

[0768] Step 6:

[0769] The device will download and install the new model.

[0770] Input: The artificial intelligence model sent from the server in step 4.

[0771] Specific operation: The device sends an HTTP request to the server to download the model file. After the download is complete, the device stores the model in its internal memory, replacing the old model.

[0772] Output: The latest installed artificial intelligence model.

[0773] Step 7:

[0774] The user collects the data.

[0775] Input: Real-time data from cameras and sensors (e.g., images, video, audio, sensor data).

[0776] Specific operation: The user takes photos of the disaster site with a camera and collects environmental data from sensors.

[0777] Output: The dataset entered on the terminal.

[0778] Step 8:

[0779] The device preprocesses the data.

[0780] Input: Data collected in step 7.

[0781] Specific operation: The device performs preprocessing such as noise removal and resolution adjustment.

[0782] Output: Preprocessed data.

[0783] Step 9:

[0784] The device analyzes the preprocessed data.

[0785] Input: The data preprocessed in step 8 and the latest artificial intelligence model installed on the device.

[0786] Specific operation: The device makes an API call, inputs the preprocessed data into the model, and begins analysis.

[0787] Output: Analysis results (e.g., identification of high-risk areas, assessment of damage scale).

[0788] Step 10:

[0789] The terminal provides the analysis results to the user.

[0790] Input: Analysis results obtained in step 9.

[0791] Specific operation: The device visualizes and displays the analysis results on a display or other user interface.

[0792] Output: The analytical information that is presented visually to the user.

[0793] Step 11:

[0794] The device uses an emotion engine to collect and analyze emotion data.

[0795] Input: User facial and voice data obtained from sensors such as cameras and microphones.

[0796] Specific operation: The device runs an emotion analysis algorithm based on sensor data to determine the user's emotional state.

[0797] Output: Data indicating the user's emotional state (e.g., stress level, joy, anger).

[0798] Step 12:

[0799] The user receives emotional feedback.

[0800] Input: Emotion data obtained in step 11 and analysis results provided in step 10.

[0801] Specific operation: The device displays appropriate feedback (e.g., suggestions for rest, encouraging messages) based on the user's emotional state.

[0802] Output: The feedback message provided to the user.

[0803] This is the specific processing flow of the system's program. This enables advanced data analysis and appropriate feedback based on the user's emotions even in unstable communication environments, resulting in more effective and efficient decision-making.

[0804] (Application example 2)

[0805] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0806] In many brick-and-mortar stores today, it is difficult to understand customers' emotions and states in real time and provide appropriate customer service. In particular, in environments with unstable communication lines or when no connection is available, it is difficult to provide advanced support using artificial intelligence technology. Therefore, while there is a demand for improved customer satisfaction and effective customer support, current systems are unable to adequately address these challenges.

[0807] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0808] In this invention, the server includes means for storing a miniaturized artificial intelligence model in a storage medium, means for transmitting the latest artificial intelligence model via a communication line, means for analyzing collected information in real time using the artificial intelligence model stored in the storage medium and providing the analysis results to the user, and means for using an emotion engine that analyzes the user's emotional state to provide emotion-based feedback and suggestions. This makes it possible to analyze the emotional state of customers in real time and provide appropriate customer service support even in an environment where communication lines are unstable, such as in a physical store.

[0809] "Multi-location information processing" refers to the collection of information at multiple physical or virtual locations and the integration and processing of that information.

[0810] A "miniaturized artificial intelligence model" refers to an artificial intelligence model that is smaller than a general artificial intelligence model and is designed to be able to operate with limited resources.

[0811] "Storage medium" refers to a device or medium for permanently or temporarily storing data.

[0812] "Communication Line" refers to the physical or wireless network infrastructure for transmitting and receiving data.

[0813] "Means for transmitting the latest artificial intelligence model" refers to the mechanism by which the updated AI model is transferred to other devices over the network.

[0814] "Means for analyzing information in real time" refers to mechanisms for immediately processing collected data and providing the results to users immediately.

[0815] An "emotion engine that analyzes the user's emotional state" refers to a system that uses emotion recognition technology to identify emotions from the user's facial expressions and voice.

[0816] "Means for providing emotion-based feedback and suggestions" refers to mechanisms that provide appropriate advice or instructions for action based on the user's emotional state.

[0817] A "physical store" refers to a physical sales or service location that customers can visit in person.

[0818] "Customer service support" refers to systems and means that assist sales staff and service providers in the customer service activities they provide to customers.

[0819] MODE FOR CARRYING OUT THE INVENTION

[0820] To implement this invention, the following system configuration and procedures are required.

[0821] System Configuration

[0822] Hardware

[0823] 1. Device: Smart glasses (with built-in camera and display).

[0824] 2. Server: A server with high-performance computing resources.

[0825] software

[0826] 1. Artificial intelligence model: The miniaturized AI model is stored on the device's storage media.

[0827] 2. Data analysis engine: Uses OpenCV and DeepFace to perform real-time analysis of collected data.

[0828] 3. Emotion Engine: Uses emotion recognition technology based on DeepFace.

[0829] 4. Communication Protocol: The communication protocol to support data communication between the terminal and the server.

[0830] Feedback of results

[0831] 1. Analysis results: The device analyzes the collected data and displays the analysis results on the screen in real time.

[0832] 2. Emotional Feedback: The emotion engine identifies the user's emotional state and provides contextual feedback and suggestions.

[0833] 3. Model update from the server: The server sends the latest artificial intelligence model to the terminal, and the terminal stores the model in its storage medium.

[0834] Operation explanation

[0835] The device uses the smart glasses' camera to capture the facial expressions and behavior of customers visiting a physical store. The captured data is processed in real time using the device's data analysis engine (OpenCV, DeepFace). Based on the analysis results, the customer's emotional state is identified. For example, DeepFace can recognize emotions such as "happiness," "sadness," and "surprise" from the customer's facial image.

[0836] The emotion recognition results from the emotion engine are fed back to the user (store clerk), and appropriate customer service methods (e.g., "If the customer seems anxious, explain gently") are suggested.

[0837] The server also periodically trains the latest AI model and sends it to the terminal, which then stores the received model in a storage medium, contributing to improving analysis accuracy.

[0838] Specific examples

[0839] For example, consider a scenario in which a customer is being served by a sales clerk wearing smart glasses in a brick-and-mortar store. The emotion of "anxiety" is recognized based on the customer's facial expression captured by the camera. In this case, the sales clerk's display will recommend "speaking kindly and explaining the product." This will enable the sales clerk to provide appropriate service to the customer, improving customer satisfaction.

[0840] Prompt Sentence Examples

[0841] Predefine the user's emotional state and enter it in text format, such as:

[0842] user_emotion: "anxiety"

[0843] customer_behavior: "looking at products"

[0844] Based on this, the generative AI model provides the following feedback:

[0845] Suggestion: "They may ask you to explain this product, so be gentle."

[0846] This system makes it possible to analyze customers' emotional states in real time and provide appropriate customer support, even in physical stores where communication lines are unstable.

[0847] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0848] Step 1:

[0849] The server trains the latest artificial intelligence models and sends model updates to the devices.

[0850] Input: Training data collected by the server.

[0851] Data processing and computation: The server uses high-performance computing resources to train new artificial intelligence models, then converts the trained models into portable AI memory.

[0852] Output: The latest AI model sent to the device.

[0853] Specific operation: The server trains the latest model according to a regular schedule and sends update data to the terminal via the network.

[0854] Step 2:

[0855] The terminal receives the latest artificial intelligence model from the server and stores it in a storage medium.

[0856] Input: The latest AI model data sent from the server.

[0857] Data processing and calculation: The device downloads the received model data to its internal memory and installs it automatically.

[0858] Output: The latest AI model stored on a storage medium.

[0859] Specific operation: The device monitors the network connection status, and when it receives a model update notification from the server, it automatically downloads and installs the data.

[0860] Step 3:

[0861] A user (store clerk) collects facial expression data of customers using the camera in the smart glasses.

[0862] Input: Customer face image.

[0863] Data processing and computation: Collecting image data captured by the smart glasses camera and converting it into a format suitable for analysis.

[0864] Output: Facial image data in a format suitable for analysis.

[0865] How it works: A store clerk wears smart glasses, and a camera automatically captures facial expression data while interacting with customers.

[0866] Step 4:

[0867] The device analyzes the collected facial expression data to identify the emotional state.

[0868] Input: Collected customer facial image data.

[0869] Data processing and computation: Using OpenCV and DeepFace, image data is analyzed in real time to identify the customer's emotional state (e.g., joy, anxiety, surprise, etc.).

[0870] Output: Customer emotional state data.

[0871] How it works: Facial image data is input into the analysis engine, DeepFace analyzes and identifies the emotional state, and displays the results on the device.

[0872] Step 5:

[0873] The emotion engine generates appropriate feedback and suggestions based on the emotional state identified.

[0874] Input: Customer emotional state data.

[0875] Data processing and computation: The emotion engine generates pre-defined feedback and suggestions based on the captured emotional state.

[0876] Output: Feedback regarding specific ways to respond to store staff and customer support.

[0877] Specific behavior: If the emotion engine identifies "anxiety," it will display recommended actions, such as "explain things gently" to the store clerk, on the smart glasses display.

[0878] Step 6:

[0879] The user (store clerk) provides appropriate customer service based on the analysis results and feedback.

[0880] Input: Analysis results and feedback provided by the device.

[0881] Data processing and calculation: The store clerk uses the analysis results and feedback to provide appropriate support to the customer.

[0882] Output: Hospitality and product suggestions to customers.

[0883] Specific actions: The store clerk follows the instructions of the smart glasses and provides customer service by speaking to them in a friendly manner and explaining the products in detail.

[0884] Step 7:

[0885] The server periodically collects new training data and continuously trains the model to improve its accuracy.

[0886] Input: Emotion recognition data and customer service outcome data collected in physical stores.

[0887] Data processing and calculation: The server uses the collected data to retrain the AI ​​model to improve its accuracy.

[0888] Output: Improved AI model data.

[0889] Specific operation: The server periodically analyzes the data collected from the physical store and improves the model. As a result, a new AI model is distributed to the device again.

[0890] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0891] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0892] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0893] [Third embodiment]

[0894] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0895] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0896] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0897] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0898] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0899] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0900] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0901] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0902] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0903] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0904] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0905] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0906] The present invention relates to a system that stores a miniaturized artificial intelligence model in a storage medium and performs real-time data analysis even in an unstable or unavailable communication environment. Specific examples of the system based on the present invention are described in detail below.

[0907] 1. The server updates the AI ​​model

[0908] The server collects the latest training data and optimizes the AI ​​model. After training a new AI model using the training data, the server converts it into a portable AI memory format and transmits it to the device via the network. Therefore, the server uses its high-performance computing resources to efficiently generate and distribute AI models.

[0909] 2. The device receives the AI ​​model

[0910] The device receives the latest AI model update notification from the server and downloads the model via the network. After downloading, the device stores the new model in its internal memory and installs it. This ensures that the device always has the latest AI model.

[0911] 3. Users collect data

[0912] Users collect necessary data in remote locations or disaster sites. The collected data is diverse, including images, videos, audio, and sensor data. This data is input into a terminal and stored on a storage medium. Users use data collection tools (e.g., cameras, sensors) to efficiently collect data.

[0913] 4. The device analyzes the data

[0914] The device uses the AI ​​model stored on the storage medium to perform real-time data analysis based on data provided by the user. As data is input, the device preprocesses the data and converts it into a suitable format for analysis. From there, the AI ​​model begins its analysis and generates results. The analysis results are provided to the user via a user interface or display device.

[0915] 5. Users consume the results

[0916] Users who receive the analysis results can make decisions in real time based on that information. For example, researchers in remote locations can use the analysis results to determine the direction and next steps of their research, allowing for more efficient research. Disaster relief teams can also use the analysis results to carry out rapid and effective rescue operations. As a result, highly accurate data analysis and rapid decision-making are possible even in situations where communication is unstable or unavailable.

[0917] Specific examples

[0918] Example 1: Disaster Relief Scenario

[0919] In a disaster site where communication networks are unavailable, disaster relief teams collect images and audio to assess the damage. The collected data is input into a portable AI memory and instantly analyzed by an artificial intelligence model. The analysis results include the scale of damage, high-risk areas, and optimal rescue routes. Rescue teams use this information to carry out rapid and effective rescue operations.

[0920] Example 2: Remote research scenario

[0921] Researchers studying new species in remote locations input data collected on-site into a portable AI memory. The AI ​​model analyzes the input data in real time and provides insights into the characteristics and habitat of the new species. This allows researchers to decide on the next research direction on the spot and advance their research more efficiently.

[0922] The above is a detailed description of one embodiment of the present invention, which enables advanced data analysis and rapid decision-making even in an unstable communication environment.

[0923] The processing flow will be explained below.

[0924] Step 1:

[0925] The server collects the latest training data.

[0926] The server acquires weather data, earthquake data, and other related data and stores it in a database.

[0927] Step 2:

[0928] The server preprocesses the acquired training data.

[0929] It removes noise, normalizes, and fills missing values, converting the data into a format suitable for training AI models.

[0930] Step 3:

[0931] The server trains the artificial intelligence model.

[0932] Apply machine learning algorithms to build AI models and train them using the acquired training data.

[0933] Step 4:

[0934] The server converts the artificial intelligence model after training into a portable AI memory device.

[0935] The model is optimized, compressed, converted to a memory-compatible format, and saved.

[0936] Step 5:

[0937] The server sends the latest AI model to the device via the network.

[0938] The trained AI model is encoded and data transmission is performed using a secure communication method.

[0939] Step 6:

[0940] The device detects an update notification for the AI ​​model received from the server.

[0941] It periodically checks for update notifications from the server and starts receiving them if a new model is available.

[0942] Step 7:

[0943] The device downloads the new AI model.

[0944] The AI ​​model is downloaded via the network and quickly saved to the device.

[0945] Step 8:

[0946] The device will install the new AI model.

[0947] It unpacks the saved AI model into memory, deletes the previous model, and installs the new one.

[0948] Step 9:

[0949] Users collect data at disaster sites and remote locations.

[0950] The necessary data (images, audio, sensor data, etc.) is collected using cameras, sensors, etc. and input into the terminal.

[0951] Step 10:

[0952] The terminal receives and pre-processes data provided by the user.

[0953] Check the format of the input data and remove outliers and noise to make it analyzable.

[0954] Step 11:

[0955] The device inputs the preprocessed data into an AI model for real-time analysis.

[0956] The AI ​​model begins the analysis and generates analytical results.

[0957] Step 12:

[0958] The terminal obtains the analysis results and provides them to the user through a user interface.

[0959] The results of the analysis are then formatted appropriately and presented to the user via a display or other output device.

[0960] Step 13:

[0961] Users make decisions based on the analysis results.

[0962] Based on the analysis results received, we will formulate and implement research directions and rescue operation plans.

[0963] Example 1

[0964] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0965] Real-time data analysis is required in situations where data processing is required at multiple locations, such as in medical care, disaster relief, or remote research. However, in these situations, communication environments are often unstable or unavailable, resulting in delays in data analysis and processing. In addition, because the amount of information transmitted is large, efficient processing on the terminal side is also important. Conventional methods have made it difficult to make quick decisions under such circumstances.

[0966] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0967] In this invention, the server includes means for using high-performance computing resources to train an AI model based on the latest training data, converting it for portable use, and transmitting it to the terminal, means for the terminal to receive update notifications from the server and download and install new AI models, means for the user to input various data collected using a data collection tool into the terminal and store it in a storage medium, means for the terminal to preprocess the data and analyze it in real time using the AI ​​model stored in the storage medium, and provide the user with the analysis results, and means for the user to make decisions based on the analysis results. This enables fast and accurate data analysis and efficient decision-making even in situations where the communication environment is unstable or unavailable.

[0968] "High-performance computing resources" refers to hardware and software for efficiently and quickly processing data and performing calculations, and specifically includes high-performance GPUs and servers.

[0969] "Training data" refers to various types of data used to train artificial intelligence models, including, for example, image data and text data.

[0970] "Artificial intelligence model" refers to a model designed using machine learning algorithms and trained to perform specific tasks automatically.

[0971] "Converting for portable use" refers to converting a trained artificial intelligence model into a format that can run efficiently on a device that processes data.

[0972] "Update Notification" refers to a message sent from a server to a device informing the device that a newer version of an artificial intelligence model is available.

[0973] "Downloading and installing" refers to the device receiving (downloading) a new artificial intelligence model from the server and setting it up for use (installing).

[0974] "Data collection tools" refers to various devices and sensors that users use to collect data, including cameras and thermometers.

[0975] "Real-time analysis" refers to analyzing data as soon as it is entered and generating results.

[0976] "Providing the analysis results to the user" refers to the terminal presenting the information obtained by the analysis to the user through a user interface or display device.

[0977] "Making a decision" refers to a user deciding on a future course of action or strategy based on the information provided.

[0978] "Unstable or unavailable network environment" refers to a situation where the network connection is of poor quality or completely unavailable.

[0979] This invention relates to a system that stores a miniaturized artificial intelligence model on a storage medium and performs real-time data analysis even in situations where communication is unstable or unavailable. This system is realized through the cooperation of a server, terminals, and users.

[0980] server

[0981] The server uses high-performance computing resources (e.g., NVIDIA Tesla V100 GPU) to train an artificial intelligence model based on the latest training data. The server collects the necessary training data from databases and cloud storage and optimizes the model using machine learning algorithms. After training is complete, the server uses specific tools such as TensorFlow Lite Converter to convert the model for portable use. The converted model is then transmitted to the device via a network (e.g., 5G communication). A secure communication protocol (e.g., SSL / TLS) is used for this transmission.

[0982] Terminal

[0983] The device receives update notifications from the server and downloads and installs the new AI model. Update notifications are sent using protocols such as HTTP / 2. After downloading, the device stores the new model in its internal memory (e.g., an SD card) and installs it in a usable state. The device uses the downloaded model to analyze data provided by the user in real time.

[0984] User

[0985] Users collect the necessary data using data collection tools (e.g., cameras, sensors). For example, at disaster sites, they collect photos of the damage and sensor data. The collected data is entered into the device and stored in the internal memory. Users use data collection tools to efficiently collect data.

[0986] Data analysis

[0987] The device performs preprocessing such as noise removal and normalization on the data provided by the user. This preprocessing improves the accuracy of the analysis. The device then analyzes the preprocessed data in real time using an artificial intelligence model. The analysis results are provided to the user as formatted data (e.g., JSON format), and the user can view the results on the device's display or through a dedicated application.

[0988] Specific examples

[0989] Example 1: Disaster Relief Scenario

[0990] When communication networks are unavailable at disaster sites, disaster relief teams collect images and audio to assess the damage situation. The collected data is input into a terminal, and an artificial intelligence model immediately analyzes it. For example, a prompt might be used, such as, "Please analyze the damage situation using this image data." The analysis results include the scale of damage, high-risk areas, and optimal rescue routes. Rescue teams use this information to carry out prompt and effective rescue operations.

[0991] Example 2: Remote research scenario

[0992] Researchers studying new species in remote locations input data collected on-site into a terminal, using a prompt such as, "Analyze the collected biological data and determine whether it is a new species." The AI ​​model analyzes the input data in real time and provides insights into the characteristics and habitat of the new species. This allows researchers to decide on the next research direction on the spot and advance their research more efficiently.

[0993] This system enables rapid and accurate data analysis and efficient decision-making even in situations where communication is unstable or unavailable, and is expected to have a wide range of applications, including disaster relief and remote research.

[0994] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0995] The flow of this system's program processing

[0996] Step 1: Server collects training data

[0997] The server collects training data from various data sources (e.g., databases, cloud storage). The input is raw data (e.g., image data, text data) obtained from multiple data sources and stores this data in local storage. Specific operations include executing API calls and database queries.

[0998] Step 2: The server trains the AI ​​model

[0999] The server uses a high-performance GPU (e.g., NVIDIA Tesla V100) to train an AI model based on the collected training data. The input is the stored training data, and the output is an optimized AI model. Specifically, the training process is performed using a machine learning library (e.g., TensorFlow, PyTorch).

[1000] Step 3: The server converts the AI ​​model for portable use

[1001] The server converts the trained AI model into a format that can be used on portable devices (e.g., ONNX format). The input is the trained AI model, and the output is an AI model converted for portable devices. Specific operations involve using tools such as TensorFlow Lite Converter.

[1002] Step 4: The server sends the model to the device

[1003] The server sends the converted AI model to the device via a network (e.g., 5G communication). The input is the AI ​​model converted for the portable device, and the output is the model data to be sent. Specifically, data is transmitted using a secure communication protocol (e.g., SSL / TLS).

[1004] Step 5: Your device receives an update notification

[1005] The terminal receives an update notification from the server. The input is a notification message from the server, and the output is that the terminal recognizes the model update. The specific operation is to notify using the HTTP / 2 protocol.

[1006] Step 6: Your device will download the new model

[1007] After the device confirms the update notification, it downloads the new model file from the server. The input is the model data from the server, and the output is the downloaded model file. The specific operation is to execute the file download process.

[1008] Step 7: The device installs the model

[1009] The device stores the downloaded model in its internal memory (e.g., SD card) and installs it. The input is the downloaded model file, and the output is the installed model. Specific operations include unpacking and deploying the model file.

[1010] Step 8: User prepares data collection tools

[1011] The user prepares a data collection tool, such as a camera (e.g., a high-resolution camera) or a sensor (e.g., an environmental sensor). The input is the readiness of the tool to be used, and the output is that the data collection tool is ready for use. Specific actions include calibrating the device and checking the battery.

[1012] Step 9: Collect the required data

[1013] The user uses the prepared tools to collect images, videos, audio, sensor data, etc. The input is the raw data obtained from the collection tool, and the output is the collected data. Specific actions include taking photos and recording audio.

[1014] Step 10: User enters data into terminal

[1015] The collected data is connected to a terminal and transferred to the internal memory. The input is the collected data, and the output is the data stored in the terminal. Specific operations include USB connection and wireless communication (e.g., Bluetooth).

[1016] Step 11: The device preprocesses the data

[1017] The device receives data provided by the user and performs preprocessing such as noise removal and normalization. The input is unprocessed data and the output is preprocessed data. Specific operations include adjusting the image resolution and filtering audio data.

[1018] Step 12: The device analyzes the data using the AI ​​model

[1019] The device uses an AI model to perform real-time analysis based on the preprocessed data. The input is the preprocessed data, and the output is the analysis results. Specific operations include image recognition and voice analysis.

[1020] Step 13: The device generates the analysis results

[1021] Once the analysis is complete, the terminal generates the results and outputs them as formatted data (e.g., JSON format). The input is the analysis result, and the output is the formatted result data. The specific operation is to convert the results into a data format.

[1022] Step 14: User receives analysis results

[1023] The user receives the analysis results from the terminal's display device or a dedicated application. The input is the formatted result data, and the output is the result displayed to the user. The specific operation is to display the results on the display device.

[1024] Step 15: User makes decision based on results

[1025] The user makes a decision based on the analysis results. The input is the displayed analysis results, and the output is the decision content. The specific action is to determine the optimal response depending on the situation.

[1026] The above is the specific flow of the program processing of this system.

[1027] (Application example 1)

[1028] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1029] In recent years, advances in autonomous driving technology have generated much expectation for autonomous vehicles, such as reducing traffic accidents and easing traffic congestion. However, when communication networks are unstable, necessary data analysis can be delayed, making it impossible to ensure safety. To address this issue, a new system capable of performing data analysis with high accuracy and in real time is required.

[1030] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1031] In this invention, the server includes means for storing a miniaturized artificial intelligence model in a storage medium, means for transmitting the latest artificial intelligence model via a network to periodically update the artificial intelligence model stored in the storage medium, means for analyzing collected data in real time using the artificial intelligence model stored in the storage medium and providing the analysis results to a user, and means for processing video data acquired from a camera mounted on the vehicle in real time even in an unstable communication environment to detect the presence or absence of obstacles and lane deviations, thereby enabling autonomous vehicles to operate safely and effectively even in situations where the communication network is unstable or unavailable.

[1032] A "miniaturized artificial intelligence model" is an artificial intelligence model that has been optimized and scaled down so that it can operate in environments with limited computing resources and memory capacity.

[1033] "Storage media" means any device or material for storing data electronically or magnetically.

[1034] A "network" is a system or infrastructure that connects multiple devices and enables data communication.

[1035] "Data" means any form of information, such as information, signals, or measurements, that is collected, stored, or analyzed.

[1036] "Real-time" means that input data is processed and analyzed almost as soon as it is generated.

[1037] A "user" is someone who uses or benefits from the use of a system or device.

[1038] A "vehicle-mounted camera" is a device installed in a vehicle to capture images of its surroundings.

[1039] "Video data" is digital data containing visual information captured by a camera.

[1040] "Processing" is a series of operations performed on data to manipulate and analyze it and obtain useful information or results.

[1041] An "obstacle" is any object or obstacle that may impede the vehicle's progress.

[1042] "Lane deviation" refers to a state in which the vehicle's current direction of travel deviates from the lane of the road.

[1043] "Discovery" is the act of finding data or situations that meet specified conditions or characteristics.

[1044] The server collects the latest training data and optimizes the AI ​​model. It uses high-performance computing resources to train a new AI model and converts it for portable AI memory. It then sends it to the device over the network, allowing the device to always have the latest AI model. The main software used is Keras and TensorFlow, and a high-performance server (e.g., a server with a GPU) is required for hardware.

[1045] The device receives update notifications from the server, downloads new AI models, and stores them in its internal memory. All the device requires is a network connection and local storage, which can be found on common devices such as smartphones and tablets.

[1046] The user collects the necessary data (video data) using a camera mounted on the vehicle. This video data is input into the terminal and saved in a storage medium. The collected video data is preprocessed using software such as Keras and PIL.

[1047] The device then analyzes the data in real time using an artificial intelligence model stored on the storage media. Preprocessing of the video data includes standardizing and normalizing the image size, converting the data into a format that the model can use to achieve optimal results. Analysis is performed using a high-performance processor (e.g., Apple A14 Bionic) and software such as Keras and TensorFlow.

[1048] The analysis results include information such as the presence or absence of obstacles and lane deviations. These results are provided to the user via the device's user interface. Based on this information, the user can make decisions in real time to ensure safe autonomous driving.

[1049] Examples:

[1050] This autonomous driving support AI assistant analyzes images from a camera in front of the vehicle in real time while driving on the highway. For example, even if communication is lost, it can detect pedestrians or obstacles around the vehicle and issue a warning or make an emergency stop. The autonomous vehicle uses this information to decide its next course of action.

[1051] Example prompts to input to a generative AI model:

[1052] "Detect pedestrians in real time from the forward camera image and display the results."

[1053] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1054] Step 1:

[1055] The server trains the latest artificial intelligence models and converts them for portable AI memory.

[1056] Input: High-performance computing resources (e.g., GPU-equipped servers), up-to-date training data

[1057] Output: Optimized artificial intelligence model

[1058] How it works: The server uses Keras and TensorFlow to extract features from the collected dataset, train the model, and optimize the weights. It then converts the optimized model into a portable format (e.g., .h5 file).

[1059] Step 2:

[1060] The server sends the artificial intelligence model to the terminal.

[1061] Input: Optimized artificial intelligence model, network connection

[1062] Output: AI model downloaded to device

[1063] Specific operation: The server sends the optimized model to the device using the HTTP protocol, etc. The device downloads the model via the network and stores it in its internal memory.

[1064] Step 3:

[1065] A user collects video data using a camera mounted on a vehicle.

[1066] Input: Vehicle-mounted camera

[1067] Output: Camera video data

[1068] Specific operation: The camera starts up at the timing and conditions specified by the user and continuously captures images of the area in front of the vehicle. This image data is collected frame by frame and transferred to the terminal.

[1069] Step 4:

[1070] The device preprocesses the video data and converts it into an analyzable data format.

[1071] Input: Camera video data

[1072] Output: Pre-processed video data

[1073] Specific operation: The device uses the Python Image Library (PIL) and Keras to perform preprocessing such as standardizing the image size (e.g., resizing to 224x224 pixels) and normalizing (scaling pixel values ​​to the 0-1 range).

[1074] Step 5:

[1075] The device uses artificial intelligence models to analyze the pre-processed data in real time.

[1076] Input: Preprocessed video data, artificial intelligence model

[1077] Output: Analysis results (presence or absence of obstacles, lane deviation, etc.)

[1078] Specific operation: The device uses Keras and TensorFlow to input preprocessed data into an artificial intelligence model and make predictions to detect the presence or absence of obstacles and lane deviations. The model's output includes specific classification results and location information.

[1079] Step 6:

[1080] The terminal provides the analysis results to a user interface.

[1081] Input: Analysis results

[1082] Output: A visual or audio notification provided to the user

[1083] How it works: The device provides the user with real-time analysis results via a user interface (e.g., on-screen alerts, voice guidance system), allowing the user to make driving decisions based on this information.

[1084] Example prompts based on concrete examples:

[1085] "Detect pedestrians in real time from the forward camera image and display the results."

[1086] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1087] The present invention relates to a system that stores a miniaturized artificial intelligence model on a storage medium and performs real-time data analysis even in situations where communication is unstable or unavailable. Another feature of the present invention is that it provides feedback and suggestions based on the user's state by combining it with an emotion engine that recognizes the user's emotions. A specific example of a system based on the present invention is described in detail below.

[1088] 1. The server updates the AI ​​model

[1089] The server collects the latest training data and optimizes the AI ​​model. After training a new AI model using the training data, the server converts it into a portable AI memory format and transmits it to the device via the network. Therefore, the server uses its high-performance computing resources to efficiently generate and distribute AI models.

[1090] 2. The device receives the AI ​​model

[1091] The device receives the latest AI model update notification from the server and downloads the model via the network. After downloading, the device stores the new model in its internal memory and installs it. This ensures that the device always has the latest AI model.

[1092] 3. Users collect data

[1093] Users collect necessary data in remote locations or disaster sites. The collected data is diverse, including images, videos, audio, and sensor data. This data is input into a terminal and stored on a storage medium. Users use data collection tools (e.g., cameras, sensors) to efficiently collect data.

[1094] 4. The device analyzes the data

[1095] The device uses the AI ​​model stored on the storage medium to perform real-time data analysis based on data provided by the user. As data is input, the device preprocesses the data and converts it into a suitable format for analysis. From there, the AI ​​model begins its analysis and generates results. The analysis results are provided to the user via a user interface or display device.

[1096] 5. Emotion Recognition by Emotion Engine

[1097] The device is equipped with an emotion engine that recognizes the user's emotions. This engine uses data acquired from sensors such as cameras and microphones to analyze the user's facial expressions, voice, and behavioral patterns. Based on this data, the emotion engine identifies the user's emotional state in real time and provides the user with the analysis results.

[1098] 6. Users consume the results and receive emotional feedback

[1099] Users can then make real-time decisions based on the analysis results. The emotion engine provides appropriate feedback and suggestions based on the user's emotional state. For example, if the stress level is high, it will suggest relaxation techniques and rest. If the user is in a positive emotional state, it will immediately display encouraging messages to take the next step.

[1100] Specific examples

[1101] Example 1: Disaster Relief Scenario

[1102] When communication networks are unavailable at a disaster site, disaster relief teams collect images and audio to assess the damage. The collected data is input into a portable AI memory and instantly analyzed by an artificial intelligence model. The analysis results include the scale of damage, high-risk areas, and optimal rescue routes. At the same time, an emotion engine analyzes the emotional state of the rescue team and suggests appropriate rest if stress levels are high. The rescue team then uses this information to carry out prompt and effective rescue operations.

[1103] Example 2: Remote research scenario

[1104] Researchers studying new species in remote locations input data collected on-site into a portable AI memory. The AI ​​model analyzes the input data in real time and provides insights into the characteristics and habitat of the new species. The emotion engine assesses the researcher's emotional state from their facial expressions and voice, and sends encouraging messages or suggests rest as needed. This allows researchers to decide on the next research direction on the spot and advance their research more efficiently.

[1105] This concludes the detailed description of one embodiment of the present invention, which enables advanced data analysis and appropriate feedback based on the user's emotions even in unstable communication environments, resulting in more effective and efficient decision-making.

[1106] The processing flow will be explained below.

[1107] Step 1:

[1108] The server collects the latest training data.

[1109] The server acquires weather data, earthquake data, and other related data and stores it in a database.

[1110] Step 2:

[1111] The server preprocesses the acquired training data.

[1112] It removes noise, normalizes, and fills missing values, converting the data into a format suitable for training AI models.

[1113] Step 3:

[1114] The server trains the artificial intelligence model.

[1115] Apply machine learning algorithms to build AI models and train them using the acquired training data.

[1116] Step 4:

[1117] The server converts the artificial intelligence model after training into a portable AI memory device.

[1118] The model is optimized, compressed, converted to a memory-compatible format, and saved.

[1119] Step 5:

[1120] The server sends the latest AI model to the device via the network.

[1121] The trained AI model is encoded and data transmission is performed using a secure communication method.

[1122] Step 6:

[1123] The device detects an update notification for the AI ​​model received from the server.

[1124] It periodically checks for update notifications from the server and starts receiving them if a new model is available.

[1125] Step 7:

[1126] The device downloads the new AI model.

[1127] The AI ​​model is downloaded via the network and quickly saved to the device.

[1128] Step 8:

[1129] The device will install the new AI model.

[1130] It unpacks the saved AI model into memory, deletes the previous model, and installs the new one.

[1131] Step 9:

[1132] Users collect data at disaster sites and remote locations.

[1133] The necessary data (images, audio, sensor data, etc.) is collected using cameras, sensors, etc. and input into the terminal.

[1134] Step 10:

[1135] The terminal receives and pre-processes data provided by the user.

[1136] Check the format of the input data and remove outliers and noise to make it analyzable.

[1137] Step 11:

[1138] The device inputs the preprocessed data into an AI model for real-time analysis.

[1139] The AI ​​model begins the analysis and generates analytical results.

[1140] Step 12:

[1141] The terminal obtains the analysis results and provides them to the user through a user interface.

[1142] The results of the analysis are then formatted appropriately and presented to the user via a display or other output device.

[1143] Step 13:

[1144] The device analyzes the user's facial expressions and voice using an emotion engine.

[1145] The device uses data from the camera and microphone to determine the user's emotional state.

[1146] Step 14:

[1147] The device provides feedback and suggestions based on the emotional state it identifies.

[1148] If the user is feeling stressed, it will suggest ways to relax, and if they are feeling positive, it will display an encouraging message.

[1149] Step 15:

[1150] Users use emotion-based feedback to make decisions.

[1151] Choose the appropriate action based on your mental state and move on to the next step.

[1152] Example 2

[1153] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1154] Conventional systems have difficulty analyzing data in unstable communication environments, which can delay real-time decision-making based on collected data. Furthermore, they do not provide feedback or suggestions that take into account the user's emotional state, hindering efficient activities.

[1155] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for storing a miniaturized artificial intelligence model in a storage medium for multi-site data processing, means for transmitting the latest artificial intelligence model via a network to periodically update the artificial intelligence model stored in the storage medium, means for analyzing collected data in real time using the artificial intelligence model stored in the storage medium and providing the analysis results to the user, means for performing real-time emotion analysis using data acquired from a sensor to recognize the user's emotional state, and means for providing appropriate feedback and suggestions based on the user's emotional state. This enables advanced data analysis and appropriate feedback based on the user's emotions even in an unstable communication environment, thereby realizing more effective and efficient decision-making.

[1156] "Miniaturized AI models" refer to AI algorithms or neural networks that are reduced in size and designed to run efficiently on portable devices.

[1157] "Storage media" refers to computer hardware components used for long-term data storage, including HDDs, SSDs, and USB memory sticks.

[1158] "Network" refers to the infrastructure for data communication between multiple computers and devices, and specifically includes Wi-Fi, 4G / 5G, Ethernet, etc.

[1159] "Data analytics" refers to the set of processes that are undertaken to process collected data and extract meaningful information and insights.

[1160] "Means for providing to the user" refers to an interface or mechanism that visually or audibly conveys analysis results and feedback information to the user.

[1161] "Emotion analysis" refers to a set of algorithms and methods for identifying a user's emotional or psychological state based on data obtained from sensors.

[1162] "Means for providing feedback and suggestions" refers to a system or interface that provides information to encourage or motivate a user to take appropriate action based on the user's emotional state or the results of data analysis.

[1163] The present invention relates to a system that stores a miniaturized artificial intelligence model on a storage medium and performs real-time data analysis even in unstable or unavailable communication environments. Another feature of the present invention is that it combines an emotion engine that recognizes the user's emotions to provide feedback and suggestions based on the user's state.

[1164] The system is implemented using the following hardware and software.

[1165] Update of artificial intelligence models by the server

[1166] The server optimizes the AI ​​model using high-performance computing resources. Specifically, it utilizes deep learning frameworks such as TensorFlow and PyTorch using GPUs on cloud services (e.g., Google Cloud Platform and Amazon Web Services). The server periodically collects the latest training data (e.g., image and audio data) and uses this data to train the AI ​​model. The server then converts the model into an optimal format for portable AI memory (e.g., ONNX) and transmits it to the device via a network (e.g., Wi-Fi, 4G / 5G).

[1167] Receiving artificial intelligence models and analyzing data on the device

[1168] The terminal receives the latest AI model update notification from the server and downloads the new model via the network. After downloading, the terminal stores the model in its internal memory and installs it. The terminal is a portable device such as a smartphone or laptop, and performs real-time data analysis based on the received model. Based on the data collected by the user (e.g., images, audio, sensor data), the data is preprocessed and converted into an appropriate format for analysis, and the AI ​​model analyzes the data. The analysis results are provided to the user via a user interface (e.g., a display).

[1169] Emotion recognition and feedback by emotion engine

[1170] The device is equipped with sensors such as a camera and microphone, which are used to collect the user's emotional data. The emotion engine uses this data to analyze the user's facial expressions, voice, and behavioral patterns to identify the user's emotional state in real time. Based on the analysis results, it provides appropriate feedback to the user. For example, if the user's stress level is high, it suggests taking a break, and if the user is in a positive emotional state, it displays an encouraging message to take the next step.

[1171] Prompt Sentence Examples

[1172] Example 1: Disaster Relief Scenario

[1173] A disaster relief team took 100 images at the scene and entered them into a mobile device. Explain how the device's AI model analyzed the extent of the damage, identified high-risk areas, and suggested rest based on stress levels.

[1174] Example 2: Remote research scenario

[1175] A researcher remotely collects photos of 30 new plant species and inputs them into a device. The device's AI model analyzes the data, provides characteristics of the new species, and an emotion engine detects when the researcher is fatigued and provides appropriate feedback. Explain the process.

[1176] This concludes the detailed description of one embodiment of the present invention, which enables advanced data analysis and appropriate feedback based on the user's emotions even in unstable communication environments, resulting in more effective and efficient decision-making.

[1177] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1178] Step 1:

[1179] A server collects training data.

[1180] Input: Large amounts of image and audio data stored in cloud storage (e.g., Amazon S3).

[1181] Specific operation: The server downloads the training data via the cloud API.

[1182] Output: Training data saved in local storage.

[1183] Step 2:

[1184] The server trains the artificial intelligence model.

[1185] Input: The training data collected in step 1.

[1186] What it does: The server trains a neural network using a deep learning framework such as TensorFlow or PyTorch.

[1187] Output: A fully trained artificial intelligence model.

[1188] Step 3:

[1189] The server converts the model into a portable format.

[1190] Input: The artificial intelligence model trained in step 2.

[1191] Specific operation: The server converts the model into a compatible format such as ONNX.

[1192] Output: An artificial intelligence model in a portable format.

[1193] Step 4:

[1194] The server sends the model to the device.

[1195] Input: The artificial intelligence model in portable format converted in step 3.

[1196] Specific operation: The server uploads the model to the device via the network (e.g., Wi-Fi, 5G) using the HTTP or FTP protocol.

[1197] Output: The artificial intelligence model downloaded to the device.

[1198] Step 5:

[1199] Your device will receive an update notification.

[1200] Input: Update notifications sent by the server.

[1201] Specific behavior: The device receives a push notification and checks for the existence of a new model.

[1202] Output: Update notification reception status.

[1203] Step 6:

[1204] The device will download and install the new model.

[1205] Input: The artificial intelligence model sent from the server in step 4.

[1206] Specific operation: The device sends an HTTP request to the server to download the model file. After the download is complete, the device stores the model in its internal memory, replacing the old model.

[1207] Output: The latest installed artificial intelligence model.

[1208] Step 7:

[1209] The user collects the data.

[1210] Input: Real-time data from cameras and sensors (e.g., images, video, audio, sensor data).

[1211] Specific operation: The user takes photos of the disaster site with a camera and collects environmental data from sensors.

[1212] Output: The dataset entered on the terminal.

[1213] Step 8:

[1214] The device preprocesses the data.

[1215] Input: Data collected in step 7.

[1216] Specific operation: The device performs preprocessing such as noise removal and resolution adjustment.

[1217] Output: Preprocessed data.

[1218] Step 9:

[1219] The device analyzes the preprocessed data.

[1220] Input: The data preprocessed in step 8 and the latest artificial intelligence model installed on the device.

[1221] Specific operation: The device makes an API call, inputs the preprocessed data into the model, and begins analysis.

[1222] Output: Analysis results (e.g., identification of high-risk areas, assessment of damage scale).

[1223] Step 10:

[1224] The terminal provides the analysis results to the user.

[1225] Input: Analysis results obtained in step 9.

[1226] Specific operation: The device visualizes and displays the analysis results on a display or other user interface.

[1227] Output: The analytical information that is presented visually to the user.

[1228] Step 11:

[1229] The device uses an emotion engine to collect and analyze emotion data.

[1230] Input: User facial and voice data obtained from sensors such as cameras and microphones.

[1231] Specific operation: The device runs an emotion analysis algorithm based on sensor data to determine the user's emotional state.

[1232] Output: Data indicating the user's emotional state (e.g., stress level, joy, anger).

[1233] Step 12:

[1234] The user receives emotional feedback.

[1235] Input: Emotion data obtained in step 11 and analysis results provided in step 10.

[1236] Specific operation: The device displays appropriate feedback (e.g., suggestions for rest, encouraging messages) based on the user's emotional state.

[1237] Output: The feedback message provided to the user.

[1238] This is the specific processing flow of the system's program. This enables advanced data analysis and appropriate feedback based on the user's emotions even in unstable communication environments, resulting in more effective and efficient decision-making.

[1239] (Application example 2)

[1240] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1241] In many brick-and-mortar stores today, it is difficult to understand customers' emotions and states in real time and provide appropriate customer service. In particular, in environments with unstable communication lines or when no connection is available, it is difficult to provide advanced support using artificial intelligence technology. Therefore, while there is a demand for improved customer satisfaction and effective customer support, current systems are unable to adequately address these challenges.

[1242] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1243] In this invention, the server includes means for storing a miniaturized artificial intelligence model in a storage medium, means for transmitting the latest artificial intelligence model via a communication line, means for analyzing collected information in real time using the artificial intelligence model stored in the storage medium and providing the analysis results to the user, and means for using an emotion engine that analyzes the user's emotional state to provide emotion-based feedback and suggestions. This makes it possible to analyze the emotional state of customers in real time and provide appropriate customer service support even in an environment where communication lines are unstable, such as in a physical store.

[1244] "Multi-location information processing" refers to the collection of information at multiple physical or virtual locations and the integration and processing of that information.

[1245] A "miniaturized artificial intelligence model" refers to an artificial intelligence model that is smaller than a general artificial intelligence model and is designed to be able to operate with limited resources.

[1246] "Storage medium" refers to a device or medium for permanently or temporarily storing data.

[1247] "Communication Line" refers to the physical or wireless network infrastructure for transmitting and receiving data.

[1248] "Means for transmitting the latest artificial intelligence model" refers to the mechanism by which the updated AI model is transferred to other devices over the network.

[1249] "Means for analyzing information in real time" refers to mechanisms for immediately processing collected data and providing the results to users immediately.

[1250] An "emotion engine that analyzes the user's emotional state" refers to a system that uses emotion recognition technology to identify emotions from the user's facial expressions and voice.

[1251] "Means for providing emotion-based feedback and suggestions" refers to mechanisms that provide appropriate advice or instructions for action based on the user's emotional state.

[1252] A "physical store" refers to a physical sales or service location that customers can visit in person.

[1253] "Customer service support" refers to systems and means that assist sales staff and service providers in the customer service activities they provide to customers.

[1254] MODE FOR CARRYING OUT THE INVENTION

[1255] To implement this invention, the following system configuration and procedures are required.

[1256] System Configuration

[1257] Hardware

[1258] 1. Device: Smart glasses (with built-in camera and display).

[1259] 2. Server: A server with high-performance computing resources.

[1260] software

[1261] 1. Artificial intelligence model: The miniaturized AI model is stored on the device's storage media.

[1262] 2. Data analysis engine: Uses OpenCV and DeepFace to perform real-time analysis of collected data.

[1263] 3. Emotion Engine: Uses emotion recognition technology based on DeepFace.

[1264] 4. Communication Protocol: The communication protocol to support data communication between the terminal and the server.

[1265] Feedback of results

[1266] 1. Analysis results: The device analyzes the collected data and displays the analysis results on the screen in real time.

[1267] 2. Emotional Feedback: The emotion engine identifies the user's emotional state and provides contextual feedback and suggestions.

[1268] 3. Model update from the server: The server sends the latest artificial intelligence model to the terminal, and the terminal stores the model in its storage medium.

[1269] Operation explanation

[1270] The device uses the smart glasses' camera to capture the facial expressions and behavior of customers visiting a physical store. The captured data is processed in real time using the device's data analysis engine (OpenCV, DeepFace). Based on the analysis results, the customer's emotional state is identified. For example, DeepFace can recognize emotions such as "happiness," "sadness," and "surprise" from the customer's facial image.

[1271] The emotion recognition results from the emotion engine are fed back to the user (store clerk), and appropriate customer service methods (e.g., "If the customer seems anxious, explain gently") are suggested.

[1272] The server also periodically trains the latest AI model and sends it to the terminal, which then stores the received model in a storage medium, contributing to improving analysis accuracy.

[1273] Specific examples

[1274] For example, consider a scenario in which a customer is being served by a sales clerk wearing smart glasses in a brick-and-mortar store. The emotion of "anxiety" is recognized based on the customer's facial expression captured by the camera. In this case, the sales clerk's display will recommend "speaking kindly and explaining the product." This will enable the sales clerk to provide appropriate service to the customer, improving customer satisfaction.

[1275] Prompt Sentence Examples

[1276] Predefine the user's emotional state and enter it in text format, such as:

[1277] user_emotion: "anxiety"

[1278] customer_behavior: "looking at products"

[1279] Based on this, the generative AI model provides the following feedback:

[1280] Suggestion: "They may ask you to explain this product, so be gentle."

[1281] This system makes it possible to analyze customers' emotional states in real time and provide appropriate customer support, even in physical stores where communication lines are unstable.

[1282] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1283] Step 1:

[1284] The server trains the latest artificial intelligence models and sends model updates to the devices.

[1285] Input: Training data collected by the server.

[1286] Data processing and computation: The server uses high-performance computing resources to train new artificial intelligence models, then converts the trained models into portable AI memory.

[1287] Output: The latest AI model sent to the device.

[1288] Specific operation: The server trains the latest model according to a regular schedule and sends update data to the terminal via the network.

[1289] Step 2:

[1290] The terminal receives the latest artificial intelligence model from the server and stores it in a storage medium.

[1291] Input: The latest AI model data sent from the server.

[1292] Data processing and calculation: The device downloads the received model data to its internal memory and installs it automatically.

[1293] Output: The latest AI model stored on a storage medium.

[1294] Specific operation: The device monitors the network connection status, and when it receives a model update notification from the server, it automatically downloads and installs the data.

[1295] Step 3:

[1296] A user (store clerk) collects facial expression data of customers using the camera in the smart glasses.

[1297] Input: Customer face image.

[1298] Data processing and computation: Collecting image data captured by the smart glasses camera and converting it into a format suitable for analysis.

[1299] Output: Facial image data in a format suitable for analysis.

[1300] How it works: A store clerk wears smart glasses, and a camera automatically captures facial expression data while interacting with customers.

[1301] Step 4:

[1302] The device analyzes the collected facial expression data to identify the emotional state.

[1303] Input: Collected customer facial image data.

[1304] Data processing and computation: Using OpenCV and DeepFace, image data is analyzed in real time to identify the customer's emotional state (e.g., joy, anxiety, surprise, etc.).

[1305] Output: Customer emotional state data.

[1306] How it works: Facial image data is input into the analysis engine, DeepFace analyzes and identifies the emotional state, and displays the results on the device.

[1307] Step 5:

[1308] The emotion engine generates appropriate feedback and suggestions based on the emotional state identified.

[1309] Input: Customer emotional state data.

[1310] Data processing and computation: The emotion engine generates pre-defined feedback and suggestions based on the captured emotional state.

[1311] Output: Feedback regarding specific ways to respond to store staff and customer support.

[1312] Specific behavior: If the emotion engine identifies "anxiety," it will display recommended actions, such as "explain things gently" to the store clerk, on the smart glasses display.

[1313] Step 6:

[1314] The user (store clerk) provides appropriate customer service based on the analysis results and feedback.

[1315] Input: Analysis results and feedback provided by the device.

[1316] Data processing and calculation: The store clerk uses the analysis results and feedback to provide appropriate support to the customer.

[1317] Output: Hospitality and product suggestions to customers.

[1318] Specific actions: The store clerk follows the instructions of the smart glasses and provides customer service by speaking to them in a friendly manner and explaining the products in detail.

[1319] Step 7:

[1320] The server periodically collects new training data and continuously trains the model to improve its accuracy.

[1321] Input: Emotion recognition data and customer service outcome data collected in physical stores.

[1322] Data processing and calculation: The server uses the collected data to retrain the AI ​​model to improve its accuracy.

[1323] Output: Improved AI model data.

[1324] Specific operation: The server periodically analyzes the data collected from the physical store and improves the model. As a result, a new AI model is distributed to the device again.

[1325] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1326] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1327] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1328] [Fourth embodiment]

[1329] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1330] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1331] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1332] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1333] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1334] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1335] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1336] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1337] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1338] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1339] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1340] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1341] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1342] The present invention relates to a system that stores a miniaturized artificial intelligence model in a storage medium and performs real-time data analysis even in an unstable or unavailable communication environment. Specific examples of the system based on the present invention are described in detail below.

[1343] 1. The server updates the AI ​​model

[1344] The server collects the latest training data and optimizes the AI ​​model. After training a new AI model using the training data, the server converts it into a portable AI memory format and transmits it to the device via the network. Therefore, the server uses its high-performance computing resources to efficiently generate and distribute AI models.

[1345] 2. The device receives the AI ​​model

[1346] The device receives the latest AI model update notification from the server and downloads the model via the network. After downloading, the device stores the new model in its internal memory and installs it. This ensures that the device always has the latest AI model.

[1347] 3. Users collect data

[1348] Users collect necessary data in remote locations or disaster sites. The collected data is diverse, including images, videos, audio, and sensor data. This data is input into a terminal and stored on a storage medium. Users use data collection tools (e.g., cameras, sensors) to efficiently collect data.

[1349] 4. The device analyzes the data

[1350] The device uses the AI ​​model stored on the storage medium to perform real-time data analysis based on data provided by the user. As data is input, the device preprocesses the data and converts it into a suitable format for analysis. From there, the AI ​​model begins its analysis and generates results. The analysis results are provided to the user via a user interface or display device.

[1351] 5. Users consume the results

[1352] Users who receive the analysis results can make decisions in real time based on that information. For example, researchers in remote locations can use the analysis results to determine the direction and next steps of their research, allowing for more efficient research. Disaster relief teams can also use the analysis results to carry out rapid and effective rescue operations. As a result, highly accurate data analysis and rapid decision-making are possible even in situations where communication is unstable or unavailable.

[1353] Specific examples

[1354] Example 1: Disaster Relief Scenario

[1355] In a disaster site where communication networks are unavailable, disaster relief teams collect images and audio to assess the damage. The collected data is input into a portable AI memory and instantly analyzed by an artificial intelligence model. The analysis results include the scale of damage, high-risk areas, and optimal rescue routes. Rescue teams use this information to carry out rapid and effective rescue operations.

[1356] Example 2: Remote research scenario

[1357] Researchers studying new species in remote locations input data collected on-site into a portable AI memory. The AI ​​model analyzes the input data in real time and provides insights into the characteristics and habitat of the new species. This allows researchers to decide on the next research direction on the spot and advance their research more efficiently.

[1358] The above is a detailed description of one embodiment of the present invention, which enables advanced data analysis and rapid decision-making even in an unstable communication environment.

[1359] The processing flow will be explained below.

[1360] Step 1:

[1361] The server collects the latest training data.

[1362] The server acquires weather data, earthquake data, and other related data and stores it in a database.

[1363] Step 2:

[1364] The server preprocesses the acquired training data.

[1365] It removes noise, normalizes, and fills missing values, converting the data into a format suitable for training AI models.

[1366] Step 3:

[1367] The server trains the artificial intelligence model.

[1368] Apply machine learning algorithms to build AI models and train them using the acquired training data.

[1369] Step 4:

[1370] The server converts the artificial intelligence model after training into a portable AI memory device.

[1371] The model is optimized, compressed, converted to a memory-compatible format, and saved.

[1372] Step 5:

[1373] The server sends the latest AI model to the device via the network.

[1374] The trained AI model is encoded and data transmission is performed using a secure communication method.

[1375] Step 6:

[1376] The device detects an update notification for the AI ​​model received from the server.

[1377] It periodically checks for update notifications from the server and starts receiving them if a new model is available.

[1378] Step 7:

[1379] The device downloads the new AI model.

[1380] The AI ​​model is downloaded via the network and quickly saved to the device.

[1381] Step 8:

[1382] The device will install the new AI model.

[1383] It unpacks the saved AI model into memory, deletes the previous model, and installs the new one.

[1384] Step 9:

[1385] Users collect data at disaster sites and remote locations.

[1386] The necessary data (images, audio, sensor data, etc.) is collected using cameras, sensors, etc. and input into the terminal.

[1387] Step 10:

[1388] The terminal receives and pre-processes data provided by the user.

[1389] Check the format of the input data and remove outliers and noise to make it analyzable.

[1390] Step 11:

[1391] The device inputs the preprocessed data into an AI model for real-time analysis.

[1392] The AI ​​model begins the analysis and generates analytical results.

[1393] Step 12:

[1394] The terminal obtains the analysis results and provides them to the user through a user interface.

[1395] The results of the analysis are then formatted appropriately and presented to the user via a display or other output device.

[1396] Step 13:

[1397] Users make decisions based on the analysis results.

[1398] Based on the analysis results received, we will formulate and implement research directions and rescue operation plans.

[1399] Example 1

[1400] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1401] Real-time data analysis is required in situations where data processing is required at multiple locations, such as in medical care, disaster relief, or remote research. However, in these situations, communication environments are often unstable or unavailable, resulting in delays in data analysis and processing. In addition, because the amount of information transmitted is large, efficient processing on the terminal side is also important. Conventional methods have made it difficult to make quick decisions under such circumstances.

[1402] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1403] In this invention, the server includes means for using high-performance computing resources to train an AI model based on the latest training data, converting it for portable use, and transmitting it to the terminal, means for the terminal to receive update notifications from the server and download and install new AI models, means for the user to input various data collected using a data collection tool into the terminal and store it in a storage medium, means for the terminal to preprocess the data and analyze it in real time using the AI ​​model stored in the storage medium, and provide the user with the analysis results, and means for the user to make decisions based on the analysis results. This enables fast and accurate data analysis and efficient decision-making even in situations where the communication environment is unstable or unavailable.

[1404] "High-performance computing resources" refers to hardware and software for efficiently and quickly processing data and performing calculations, and specifically includes high-performance GPUs and servers.

[1405] "Training data" refers to various types of data used to train artificial intelligence models, including, for example, image data and text data.

[1406] "Artificial intelligence model" refers to a model designed using machine learning algorithms and trained to perform specific tasks automatically.

[1407] "Converting for portable use" refers to converting a trained artificial intelligence model into a format that can run efficiently on a device that processes data.

[1408] "Update Notification" refers to a message sent from a server to a device informing the device that a newer version of an artificial intelligence model is available.

[1409] "Downloading and installing" refers to the device receiving (downloading) a new artificial intelligence model from the server and setting it up for use (installing).

[1410] "Data collection tools" refers to various devices and sensors that users use to collect data, including cameras and thermometers.

[1411] "Real-time analysis" refers to analyzing data as soon as it is entered and generating results.

[1412] "Providing the analysis results to the user" refers to the terminal presenting the information obtained by the analysis to the user through a user interface or display device.

[1413] "Making a decision" refers to a user deciding on a future course of action or strategy based on the information provided.

[1414] "Unstable or unavailable network environment" refers to a situation where the network connection is of poor quality or completely unavailable.

[1415] This invention relates to a system that stores a miniaturized artificial intelligence model on a storage medium and performs real-time data analysis even in situations where communication is unstable or unavailable. This system is realized through the cooperation of a server, terminals, and users.

[1416] server

[1417] The server uses high-performance computing resources (e.g., NVIDIA Tesla V100 GPU) to train an artificial intelligence model based on the latest training data. The server collects the necessary training data from databases and cloud storage and optimizes the model using machine learning algorithms. After training is complete, the server uses specific tools such as TensorFlow Lite Converter to convert the model for portable use. The converted model is then transmitted to the device via a network (e.g., 5G communication). A secure communication protocol (e.g., SSL / TLS) is used for this transmission.

[1418] Terminal

[1419] The device receives update notifications from the server and downloads and installs the new AI model. Update notifications are sent using protocols such as HTTP / 2. After downloading, the device stores the new model in its internal memory (e.g., an SD card) and installs it in a usable state. The device uses the downloaded model to analyze data provided by the user in real time.

[1420] User

[1421] Users collect the necessary data using data collection tools (e.g., cameras, sensors). For example, at disaster sites, they collect photos of the damage and sensor data. The collected data is entered into the device and stored in the internal memory. Users use data collection tools to efficiently collect data.

[1422] Data analysis

[1423] The device performs preprocessing such as noise removal and normalization on the data provided by the user. This preprocessing improves the accuracy of the analysis. The device then analyzes the preprocessed data in real time using an artificial intelligence model. The analysis results are provided to the user as formatted data (e.g., JSON format), and the user can view the results on the device's display or through a dedicated application.

[1424] Specific examples

[1425] Example 1: Disaster Relief Scenario

[1426] When communication networks are unavailable at disaster sites, disaster relief teams collect images and audio to assess the damage situation. The collected data is input into a terminal, and an artificial intelligence model immediately analyzes it. For example, a prompt might be used, such as, "Please analyze the damage situation using this image data." The analysis results include the scale of damage, high-risk areas, and optimal rescue routes. Rescue teams use this information to carry out prompt and effective rescue operations.

[1427] Example 2: Remote research scenario

[1428] Researchers studying new species in remote locations input data collected on-site into a terminal, using a prompt such as, "Analyze the collected biological data and determine whether it is a new species." The AI ​​model analyzes the input data in real time and provides insights into the characteristics and habitat of the new species. This allows researchers to decide on the next research direction on the spot and advance their research more efficiently.

[1429] This system enables rapid and accurate data analysis and efficient decision-making even in situations where communication is unstable or unavailable, and is expected to have a wide range of applications, including disaster relief and remote research.

[1430] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1431] The flow of this system's program processing

[1432] Step 1: Server collects training data

[1433] The server collects training data from various data sources (e.g., databases, cloud storage). The input is raw data (e.g., image data, text data) obtained from multiple data sources and stores this data in local storage. Specific operations include executing API calls and database queries.

[1434] Step 2: The server trains the AI ​​model

[1435] The server uses a high-performance GPU (e.g., NVIDIA Tesla V100) to train an AI model based on the collected training data. The input is the stored training data, and the output is an optimized AI model. Specifically, the training process is performed using a machine learning library (e.g., TensorFlow, PyTorch).

[1436] Step 3: The server converts the AI ​​model for portable use

[1437] The server converts the trained AI model into a format that can be used on portable devices (e.g., ONNX format). The input is the trained AI model, and the output is an AI model converted for portable devices. Specific operations involve using tools such as TensorFlow Lite Converter.

[1438] Step 4: The server sends the model to the device

[1439] The server sends the converted AI model to the device via a network (e.g., 5G communication). The input is the AI ​​model converted for the portable device, and the output is the model data to be sent. Specifically, data is transmitted using a secure communication protocol (e.g., SSL / TLS).

[1440] Step 5: Your device receives an update notification

[1441] The terminal receives an update notification from the server. The input is a notification message from the server, and the output is that the terminal recognizes the model update. The specific operation is to notify using the HTTP / 2 protocol.

[1442] Step 6: Your device will download the new model

[1443] After the device confirms the update notification, it downloads the new model file from the server. The input is the model data from the server, and the output is the downloaded model file. The specific operation is to execute the file download process.

[1444] Step 7: The device installs the model

[1445] The device stores the downloaded model in its internal memory (e.g., SD card) and installs it. The input is the downloaded model file, and the output is the installed model. Specific operations include unpacking and deploying the model file.

[1446] Step 8: User prepares data collection tools

[1447] The user prepares a data collection tool, such as a camera (e.g., a high-resolution camera) or a sensor (e.g., an environmental sensor). The input is the readiness of the tool to be used, and the output is that the data collection tool is ready for use. Specific actions include calibrating the device and checking the battery.

[1448] Step 9: Collect the required data

[1449] The user uses the prepared tools to collect images, videos, audio, sensor data, etc. The input is the raw data obtained from the collection tool, and the output is the collected data. Specific actions include taking photos and recording audio.

[1450] Step 10: User enters data into terminal

[1451] The collected data is connected to a terminal and transferred to the internal memory. The input is the collected data, and the output is the data stored in the terminal. Specific operations include USB connection and wireless communication (e.g., Bluetooth).

[1452] Step 11: The device preprocesses the data

[1453] The device receives data provided by the user and performs preprocessing such as noise removal and normalization. The input is unprocessed data and the output is preprocessed data. Specific operations include adjusting the image resolution and filtering audio data.

[1454] Step 12: The device analyzes the data using the AI ​​model

[1455] The device uses an AI model to perform real-time analysis based on the preprocessed data. The input is the preprocessed data, and the output is the analysis results. Specific operations include image recognition and voice analysis.

[1456] Step 13: The device generates the analysis results

[1457] Once the analysis is complete, the terminal generates the results and outputs them as formatted data (e.g., JSON format). The input is the analysis result, and the output is the formatted result data. The specific operation is to convert the results into a data format.

[1458] Step 14: User receives analysis results

[1459] The user receives the analysis results from the terminal's display device or a dedicated application. The input is the formatted result data, and the output is the result displayed to the user. The specific operation is to display the results on the display device.

[1460] Step 15: User makes decision based on results

[1461] The user makes a decision based on the analysis results. The input is the displayed analysis results, and the output is the decision content. The specific action is to determine the optimal response depending on the situation.

[1462] The above is the specific flow of the program processing of this system.

[1463] (Application example 1)

[1464] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1465] In recent years, advances in autonomous driving technology have generated much expectation for autonomous vehicles, such as reducing traffic accidents and easing traffic congestion. However, when communication networks are unstable, necessary data analysis can be delayed, making it impossible to ensure safety. To address this issue, a new system capable of performing data analysis with high accuracy and in real time is required.

[1466] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1467] In this invention, the server includes means for storing a miniaturized artificial intelligence model in a storage medium, means for transmitting the latest artificial intelligence model via a network to periodically update the artificial intelligence model stored in the storage medium, means for analyzing collected data in real time using the artificial intelligence model stored in the storage medium and providing the analysis results to a user, and means for processing video data acquired from a camera mounted on the vehicle in real time even in an unstable communication environment to detect the presence or absence of obstacles and lane deviations, thereby enabling autonomous vehicles to operate safely and effectively even in situations where the communication network is unstable or unavailable.

[1468] A "miniaturized artificial intelligence model" is an artificial intelligence model that has been optimized and scaled down so that it can operate in environments with limited computing resources and memory capacity.

[1469] "Storage media" means any device or material for storing data electronically or magnetically.

[1470] A "network" is a system or infrastructure that connects multiple devices and enables data communication.

[1471] "Data" means any form of information, such as information, signals, or measurements, that is collected, stored, or analyzed.

[1472] "Real-time" means that input data is processed and analyzed almost as soon as it is generated.

[1473] A "user" is someone who uses or benefits from the use of a system or device.

[1474] A "vehicle-mounted camera" is a device installed in a vehicle to capture images of its surroundings.

[1475] "Video data" is digital data containing visual information captured by a camera.

[1476] "Processing" is a series of operations performed on data to manipulate and analyze it and obtain useful information or results.

[1477] An "obstacle" is any object or obstacle that may impede the vehicle's progress.

[1478] "Lane deviation" refers to a state in which the vehicle's current direction of travel deviates from the lane of the road.

[1479] "Discovery" is the act of finding data or situations that meet specified conditions or characteristics.

[1480] The server collects the latest training data and optimizes the AI ​​model. It uses high-performance computing resources to train a new AI model and converts it for portable AI memory. It then sends it to the device over the network, allowing the device to always have the latest AI model. The main software used is Keras and TensorFlow, and a high-performance server (e.g., a server with a GPU) is required for hardware.

[1481] The device receives update notifications from the server, downloads new AI models, and stores them in its internal memory. All the device requires is a network connection and local storage, which can be found on common devices such as smartphones and tablets.

[1482] The user collects the necessary data (video data) using a camera mounted on the vehicle. This video data is input into the terminal and saved in a storage medium. The collected video data is preprocessed using software such as Keras and PIL.

[1483] The device then analyzes the data in real time using an artificial intelligence model stored on the storage media. Preprocessing of the video data includes standardizing and normalizing the image size, converting the data into a format that the model can use to achieve optimal results. Analysis is performed using a high-performance processor (e.g., Apple A14 Bionic) and software such as Keras and TensorFlow.

[1484] The analysis results include information such as the presence or absence of obstacles and lane deviations. These results are provided to the user via the device's user interface. Based on this information, the user can make decisions in real time to ensure safe autonomous driving.

[1485] Examples:

[1486] This autonomous driving support AI assistant analyzes images from a camera in front of the vehicle in real time while driving on the highway. For example, even if communication is lost, it can detect pedestrians or obstacles around the vehicle and issue a warning or make an emergency stop. The autonomous vehicle uses this information to decide its next course of action.

[1487] Example prompts to input to a generative AI model:

[1488] "Detect pedestrians in real time from the forward camera image and display the results."

[1489] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1490] Step 1:

[1491] The server trains the latest artificial intelligence models and converts them for portable AI memory.

[1492] Input: High-performance computing resources (e.g., GPU-equipped servers), up-to-date training data

[1493] Output: Optimized artificial intelligence model

[1494] How it works: The server uses Keras and TensorFlow to extract features from the collected dataset, train the model, and optimize the weights. It then converts the optimized model into a portable format (e.g., .h5 file).

[1495] Step 2:

[1496] The server sends the artificial intelligence model to the terminal.

[1497] Input: Optimized artificial intelligence model, network connection

[1498] Output: AI model downloaded to device

[1499] Specific operation: The server sends the optimized model to the device using the HTTP protocol, etc. The device downloads the model via the network and stores it in its internal memory.

[1500] Step 3:

[1501] A user collects video data using a camera mounted on a vehicle.

[1502] Input: Vehicle-mounted camera

[1503] Output: Camera video data

[1504] Specific operation: The camera starts up at the timing and conditions specified by the user and continuously captures images of the area in front of the vehicle. This image data is collected frame by frame and transferred to the terminal.

[1505] Step 4:

[1506] The device preprocesses the video data and converts it into an analyzable data format.

[1507] Input: Camera video data

[1508] Output: Pre-processed video data

[1509] Specific operation: The device uses the Python Image Library (PIL) and Keras to perform preprocessing such as standardizing the image size (e.g., resizing to 224x224 pixels) and normalizing (scaling pixel values ​​to the 0-1 range).

[1510] Step 5:

[1511] The device uses artificial intelligence models to analyze the pre-processed data in real time.

[1512] Input: Preprocessed video data, artificial intelligence model

[1513] Output: Analysis results (presence or absence of obstacles, lane deviation, etc.)

[1514] Specific operation: The device uses Keras and TensorFlow to input preprocessed data into an artificial intelligence model and make predictions to detect the presence or absence of obstacles and lane deviations. The model's output includes specific classification results and location information.

[1515] Step 6:

[1516] The terminal provides the analysis results to a user interface.

[1517] Input: Analysis results

[1518] Output: A visual or audio notification provided to the user

[1519] How it works: The device provides the user with real-time analysis results via a user interface (e.g., on-screen alerts, voice guidance system), allowing the user to make driving decisions based on this information.

[1520] Example prompts based on concrete examples:

[1521] "Detect pedestrians in real time from the forward camera image and display the results."

[1522] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1523] The present invention relates to a system that stores a miniaturized artificial intelligence model on a storage medium and performs real-time data analysis even in situations where communication is unstable or unavailable. Another feature of the present invention is that it provides feedback and suggestions based on the user's state by combining it with an emotion engine that recognizes the user's emotions. A specific example of a system based on the present invention is described in detail below.

[1524] 1. The server updates the AI ​​model

[1525] The server collects the latest training data and optimizes the AI ​​model. After training a new AI model using the training data, the server converts it into a portable AI memory format and transmits it to the device via the network. Therefore, the server uses its high-performance computing resources to efficiently generate and distribute AI models.

[1526] 2. The device receives the AI ​​model

[1527] The device receives the latest AI model update notification from the server and downloads the model via the network. After downloading, the device stores the new model in its internal memory and installs it. This ensures that the device always has the latest AI model.

[1528] 3. Users collect data

[1529] Users collect necessary data in remote locations or disaster sites. The collected data is diverse, including images, videos, audio, and sensor data. This data is input into a terminal and stored on a storage medium. Users use data collection tools (e.g., cameras, sensors) to efficiently collect data.

[1530] 4. The device analyzes the data

[1531] The device uses the AI ​​model stored on the storage medium to perform real-time data analysis based on data provided by the user. As data is input, the device preprocesses the data and converts it into a suitable format for analysis. From there, the AI ​​model begins its analysis and generates results. The analysis results are provided to the user via a user interface or display device.

[1532] 5. Emotion Recognition by Emotion Engine

[1533] The device is equipped with an emotion engine that recognizes the user's emotions. This engine uses data acquired from sensors such as cameras and microphones to analyze the user's facial expressions, voice, and behavioral patterns. Based on this data, the emotion engine identifies the user's emotional state in real time and provides the user with the analysis results.

[1534] 6. Users consume the results and receive emotional feedback

[1535] Users can then make real-time decisions based on the analysis results. The emotion engine provides appropriate feedback and suggestions based on the user's emotional state. For example, if the stress level is high, it will suggest relaxation techniques and rest. If the user is in a positive emotional state, it will immediately display encouraging messages to take the next step.

[1536] Specific examples

[1537] Example 1: Disaster Relief Scenario

[1538] When communication networks are unavailable at a disaster site, disaster relief teams collect images and audio to assess the damage. The collected data is input into a portable AI memory and instantly analyzed by an artificial intelligence model. The analysis results include the scale of damage, high-risk areas, and optimal rescue routes. At the same time, an emotion engine analyzes the emotional state of the rescue team and suggests appropriate rest if stress levels are high. The rescue team then uses this information to carry out prompt and effective rescue operations.

[1539] Example 2: Remote research scenario

[1540] Researchers studying new species in remote locations input data collected on-site into a portable AI memory. The AI ​​model analyzes the input data in real time and provides insights into the characteristics and habitat of the new species. The emotion engine assesses the researcher's emotional state from their facial expressions and voice, and sends encouraging messages or suggests rest as needed. This allows researchers to decide on the next research direction on the spot and advance their research more efficiently.

[1541] This concludes the detailed description of one embodiment of the present invention, which enables advanced data analysis and appropriate feedback based on the user's emotions even in unstable communication environments, resulting in more effective and efficient decision-making.

[1542] The processing flow will be explained below.

[1543] Step 1:

[1544] The server collects the latest training data.

[1545] The server acquires weather data, earthquake data, and other related data and stores it in a database.

[1546] Step 2:

[1547] The server preprocesses the acquired training data.

[1548] It removes noise, normalizes, and fills missing values, converting the data into a format suitable for training AI models.

[1549] Step 3:

[1550] The server trains the artificial intelligence model.

[1551] Apply machine learning algorithms to build AI models and train them using the acquired training data.

[1552] Step 4:

[1553] The server converts the artificial intelligence model after training into a portable AI memory device.

[1554] The model is optimized, compressed, converted to a memory-compatible format, and saved.

[1555] Step 5:

[1556] The server sends the latest AI model to the device via the network.

[1557] The trained AI model is encoded and data transmission is performed using a secure communication method.

[1558] Step 6:

[1559] The device detects an update notification for the AI ​​model received from the server.

[1560] It periodically checks for update notifications from the server and starts receiving them if a new model is available.

[1561] Step 7:

[1562] The device downloads the new AI model.

[1563] The AI ​​model is downloaded via the network and quickly saved to the device.

[1564] Step 8:

[1565] The device will install the new AI model.

[1566] It unpacks the saved AI model into memory, deletes the previous model, and installs the new one.

[1567] Step 9:

[1568] Users collect data at disaster sites and remote locations.

[1569] The necessary data (images, audio, sensor data, etc.) is collected using cameras, sensors, etc. and input into the terminal.

[1570] Step 10:

[1571] The terminal receives and pre-processes data provided by the user.

[1572] Check the format of the input data and remove outliers and noise to make it analyzable.

[1573] Step 11:

[1574] The device inputs the preprocessed data into an AI model for real-time analysis.

[1575] The AI ​​model begins the analysis and generates analytical results.

[1576] Step 12:

[1577] The terminal obtains the analysis results and provides them to the user through a user interface.

[1578] The results of the analysis are then formatted appropriately and presented to the user via a display or other output device.

[1579] Step 13:

[1580] The device analyzes the user's facial expressions and voice using an emotion engine.

[1581] The device uses data from the camera and microphone to determine the user's emotional state.

[1582] Step 14:

[1583] The device provides feedback and suggestions based on the emotional state it identifies.

[1584] If the user is feeling stressed, it will suggest ways to relax, and if they are feeling positive, it will display an encouraging message.

[1585] Step 15:

[1586] Users use emotion-based feedback to make decisions.

[1587] Choose the appropriate action based on your mental state and move on to the next step.

[1588] Example 2

[1589] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1590] Conventional systems have difficulty analyzing data in unstable communication environments, which can delay real-time decision-making based on collected data. Furthermore, they do not provide feedback or suggestions that take into account the user's emotional state, hindering efficient activities.

[1591] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for storing a miniaturized artificial intelligence model in a storage medium for multi-site data processing, means for transmitting the latest artificial intelligence model via a network to periodically update the artificial intelligence model stored in the storage medium, means for analyzing collected data in real time using the artificial intelligence model stored in the storage medium and providing the analysis results to the user, means for performing real-time emotion analysis using data acquired from a sensor to recognize the user's emotional state, and means for providing appropriate feedback and suggestions based on the user's emotional state. This enables advanced data analysis and appropriate feedback based on the user's emotions even in an unstable communication environment, thereby realizing more effective and efficient decision-making.

[1592] "Miniaturized AI models" refer to AI algorithms or neural networks that are reduced in size and designed to run efficiently on portable devices.

[1593] "Storage media" refers to computer hardware components used for long-term data storage, including HDDs, SSDs, and USB memory sticks.

[1594] "Network" refers to the infrastructure for data communication between multiple computers and devices, and specifically includes Wi-Fi, 4G / 5G, Ethernet, etc.

[1595] "Data analytics" refers to the set of processes that are undertaken to process collected data and extract meaningful information and insights.

[1596] "Means for providing to the user" refers to an interface or mechanism that visually or audibly conveys analysis results and feedback information to the user.

[1597] "Emotion analysis" refers to a set of algorithms and methods for identifying a user's emotional or psychological state based on data obtained from sensors.

[1598] "Means for providing feedback and suggestions" refers to a system or interface that provides information to encourage or motivate a user to take appropriate action based on the user's emotional state or the results of data analysis.

[1599] The present invention relates to a system that stores a miniaturized artificial intelligence model on a storage medium and performs real-time data analysis even in unstable or unavailable communication environments. Another feature of the present invention is that it combines an emotion engine that recognizes the user's emotions to provide feedback and suggestions based on the user's state.

[1600] The system is implemented using the following hardware and software.

[1601] Update of artificial intelligence models by the server

[1602] The server optimizes the AI ​​model using high-performance computing resources. Specifically, it utilizes deep learning frameworks such as TensorFlow and PyTorch using GPUs on cloud services (e.g., Google Cloud Platform and Amazon Web Services). The server periodically collects the latest training data (e.g., image and audio data) and uses this data to train the AI ​​model. The server then converts the model into an optimal format for portable AI memory (e.g., ONNX) and transmits it to the device via a network (e.g., Wi-Fi, 4G / 5G).

[1603] Receiving artificial intelligence models and analyzing data on the device

[1604] The terminal receives the latest AI model update notification from the server and downloads the new model via the network. After downloading, the terminal stores the model in its internal memory and installs it. The terminal is a portable device such as a smartphone or laptop, and performs real-time data analysis based on the received model. Based on the data collected by the user (e.g., images, audio, sensor data), the data is preprocessed and converted into an appropriate format for analysis, and the AI ​​model analyzes the data. The analysis results are provided to the user via a user interface (e.g., a display).

[1605] Emotion recognition and feedback by emotion engine

[1606] The device is equipped with sensors such as a camera and microphone, which are used to collect the user's emotional data. The emotion engine uses this data to analyze the user's facial expressions, voice, and behavioral patterns to identify the user's emotional state in real time. Based on the analysis results, it provides appropriate feedback to the user. For example, if the user's stress level is high, it suggests taking a break, and if the user is in a positive emotional state, it displays an encouraging message to take the next step.

[1607] Prompt Sentence Examples

[1608] Example 1: Disaster Relief Scenario

[1609] A disaster relief team took 100 images at the scene and entered them into a mobile device. Explain how the device's AI model analyzed the extent of the damage, identified high-risk areas, and suggested rest based on stress levels.

[1610] Example 2: Remote research scenario

[1611] A researcher remotely collects photos of 30 new plant species and inputs them into a device. The device's AI model analyzes the data, provides characteristics of the new species, and an emotion engine detects when the researcher is fatigued and provides appropriate feedback. Explain the process.

[1612] This concludes the detailed description of one embodiment of the present invention, which enables advanced data analysis and appropriate feedback based on the user's emotions even in unstable communication environments, resulting in more effective and efficient decision-making.

[1613] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1614] Step 1:

[1615] A server collects training data.

[1616] Input: Large amounts of image and audio data stored in cloud storage (e.g., Amazon S3).

[1617] Specific operation: The server downloads the training data via the cloud API.

[1618] Output: Training data saved in local storage.

[1619] Step 2:

[1620] The server trains the artificial intelligence model.

[1621] Input: The training data collected in step 1.

[1622] What it does: The server trains a neural network using a deep learning framework such as TensorFlow or PyTorch.

[1623] Output: A fully trained artificial intelligence model.

[1624] Step 3:

[1625] The server converts the model into a portable format.

[1626] Input: The artificial intelligence model trained in step 2.

[1627] Specific operation: The server converts the model into a compatible format such as ONNX.

[1628] Output: An artificial intelligence model in a portable format.

[1629] Step 4:

[1630] The server sends the model to the device.

[1631] Input: The artificial intelligence model in portable format converted in step 3.

[1632] Specific operation: The server uploads the model to the device via the network (e.g., Wi-Fi, 5G) using the HTTP or FTP protocol.

[1633] Output: The artificial intelligence model downloaded to the device.

[1634] Step 5:

[1635] Your device will receive an update notification.

[1636] Input: Update notifications sent by the server.

[1637] Specific behavior: The device receives a push notification and checks for the existence of a new model.

[1638] Output: Update notification reception status.

[1639] Step 6:

[1640] The device will download and install the new model.

[1641] Input: The artificial intelligence model sent from the server in step 4.

[1642] Specific operation: The device sends an HTTP request to the server to download the model file. After the download is complete, the device stores the model in its internal memory, replacing the old model.

[1643] Output: The latest installed artificial intelligence model.

[1644] Step 7:

[1645] The user collects the data.

[1646] Input: Real-time data from cameras and sensors (e.g., images, video, audio, sensor data).

[1647] Specific operation: The user takes photos of the disaster site with a camera and collects environmental data from sensors.

[1648] Output: The dataset entered on the terminal.

[1649] Step 8:

[1650] The device preprocesses the data.

[1651] Input: Data collected in step 7.

[1652] Specific operation: The device performs preprocessing such as noise removal and resolution adjustment.

[1653] Output: Preprocessed data.

[1654] Step 9:

[1655] The device analyzes the preprocessed data.

[1656] Input: The data preprocessed in step 8 and the latest artificial intelligence model installed on the device.

[1657] Specific operation: The device makes an API call, inputs the preprocessed data into the model, and begins analysis.

[1658] Output: Analysis results (e.g., identification of high-risk areas, assessment of damage scale).

[1659] Step 10:

[1660] The terminal provides the analysis results to the user.

[1661] Input: Analysis results obtained in step 9.

[1662] Specific operation: The device visualizes and displays the analysis results on a display or other user interface.

[1663] Output: The analytical information that is presented visually to the user.

[1664] Step 11:

[1665] The device uses an emotion engine to collect and analyze emotion data.

[1666] Input: User facial and voice data obtained from sensors such as cameras and microphones.

[1667] Specific operation: The device runs an emotion analysis algorithm based on sensor data to determine the user's emotional state.

[1668] Output: Data indicating the user's emotional state (e.g., stress level, joy, anger).

[1669] Step 12:

[1670] The user receives emotional feedback.

[1671] Input: Emotion data obtained in step 11 and analysis results provided in step 10.

[1672] Specific operation: The device displays appropriate feedback (e.g., suggestions for rest, encouraging messages) based on the user's emotional state.

[1673] Output: The feedback message provided to the user.

[1674] This is the specific processing flow of the system's program. This enables advanced data analysis and appropriate feedback based on the user's emotions even in unstable communication environments, resulting in more effective and efficient decision-making.

[1675] (Application example 2)

[1676] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1677] In many brick-and-mortar stores today, it is difficult to understand customers' emotions and states in real time and provide appropriate customer service. In particular, in environments with unstable communication lines or when no connection is available, it is difficult to provide advanced support using artificial intelligence technology. Therefore, while there is a demand for improved customer satisfaction and effective customer support, current systems are unable to adequately address these challenges.

[1678] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1679] In this invention, the server includes means for storing a miniaturized artificial intelligence model in a storage medium, means for transmitting the latest artificial intelligence model via a communication line, means for analyzing collected information in real time using the artificial intelligence model stored in the storage medium and providing the analysis results to the user, and means for using an emotion engine that analyzes the user's emotional state to provide emotion-based feedback and suggestions. This makes it possible to analyze the emotional state of customers in real time and provide appropriate customer service support even in an environment where communication lines are unstable, such as in a physical store.

[1680] "Multi-location information processing" refers to the collection of information at multiple physical or virtual locations and the integration and processing of that information.

[1681] A "miniaturized artificial intelligence model" refers to an artificial intelligence model that is smaller than a general artificial intelligence model and is designed to be able to operate with limited resources.

[1682] "Storage medium" refers to a device or medium for permanently or temporarily storing data.

[1683] "Communication Line" refers to the physical or wireless network infrastructure for transmitting and receiving data.

[1684] "Means for transmitting the latest artificial intelligence model" refers to the mechanism by which the updated AI model is transferred to other devices over the network.

[1685] "Means for analyzing information in real time" refers to mechanisms for immediately processing collected data and providing the results to users immediately.

[1686] An "emotion engine that analyzes the user's emotional state" refers to a system that uses emotion recognition technology to identify emotions from the user's facial expressions and voice.

[1687] "Means for providing emotion-based feedback and suggestions" refers to mechanisms that provide appropriate advice or instructions for action based on the user's emotional state.

[1688] A "physical store" refers to a physical sales or service location that customers can visit in person.

[1689] "Customer service support" refers to systems and means that assist sales staff and service providers in the customer service activities they provide to customers.

[1690] MODE FOR CARRYING OUT THE INVENTION

[1691] To implement this invention, the following system configuration and procedures are required.

[1692] System Configuration

[1693] Hardware

[1694] 1. Device: Smart glasses (with built-in camera and display).

[1695] 2. Server: A server with high-performance computing resources.

[1696] software

[1697] 1. Artificial intelligence model: The miniaturized AI model is stored on the device's storage media.

[1698] 2. Data analysis engine: Uses OpenCV and DeepFace to perform real-time analysis of collected data.

[1699] 3. Emotion Engine: Uses emotion recognition technology based on DeepFace.

[1700] 4. Communication Protocol: The communication protocol to support data communication between the terminal and the server.

[1701] Feedback of results

[1702] 1. Analysis results: The device analyzes the collected data and displays the analysis results on the screen in real time.

[1703] 2. Emotional Feedback: The emotion engine identifies the user's emotional state and provides contextual feedback and suggestions.

[1704] 3. Model update from the server: The server sends the latest artificial intelligence model to the terminal, and the terminal stores the model in its storage medium.

[1705] Operation explanation

[1706] The device uses the smart glasses' camera to capture the facial expressions and behavior of customers visiting a physical store. The captured data is processed in real time using the device's data analysis engine (OpenCV, DeepFace). Based on the analysis results, the customer's emotional state is identified. For example, DeepFace can recognize emotions such as "happiness," "sadness," and "surprise" from the customer's facial image.

[1707] The emotion recognition results from the emotion engine are fed back to the user (store clerk), and appropriate customer service methods (e.g., "If the customer seems anxious, explain gently") are suggested.

[1708] The server also periodically trains the latest AI model and sends it to the terminal, which then stores the received model in a storage medium, contributing to improving analysis accuracy.

[1709] Specific examples

[1710] For example, consider a scenario in which a customer is being served by a sales clerk wearing smart glasses in a brick-and-mortar store. The emotion of "anxiety" is recognized based on the customer's facial expression captured by the camera. In this case, the sales clerk's display will recommend "speaking kindly and explaining the product." This will enable the sales clerk to provide appropriate service to the customer, improving customer satisfaction.

[1711] Prompt Sentence Examples

[1712] Predefine the user's emotional state and enter it in text format, such as:

[1713] user_emotion: "anxiety"

[1714] customer_behavior: "looking at products"

[1715] Based on this, the generative AI model provides the following feedback:

[1716] Suggestion: "They may ask you to explain this product, so be gentle."

[1717] This system makes it possible to analyze customers' emotional states in real time and provide appropriate customer support, even in physical stores where communication lines are unstable.

[1718] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1719] Step 1:

[1720] The server trains the latest artificial intelligence models and sends model updates to the devices.

[1721] Input: Training data collected by the server.

[1722] Data processing and computation: The server uses high-performance computing resources to train new artificial intelligence models, then converts the trained models into portable AI memory.

[1723] Output: The latest AI model sent to the device.

[1724] Specific operation: The server trains the latest model according to a regular schedule and sends update data to the terminal via the network.

[1725] Step 2:

[1726] The terminal receives the latest artificial intelligence model from the server and stores it in a storage medium.

[1727] Input: The latest AI model data sent from the server.

[1728] Data processing and calculation: The device downloads the received model data to its internal memory and installs it automatically.

[1729] Output: The latest AI model stored on a storage medium.

[1730] Specific operation: The device monitors the network connection status, and when it receives a model update notification from the server, it automatically downloads and installs the data.

[1731] Step 3:

[1732] A user (store clerk) collects facial expression data of customers using the camera in the smart glasses.

[1733] Input: Customer face image.

[1734] Data processing and computation: Collecting image data captured by the smart glasses camera and converting it into a format suitable for analysis.

[1735] Output: Facial image data in a format suitable for analysis.

[1736] How it works: A store clerk wears smart glasses, and a camera automatically captures facial expression data while interacting with customers.

[1737] Step 4:

[1738] The device analyzes the collected facial expression data to identify the emotional state.

[1739] Input: Collected customer facial image data.

[1740] Data processing and computation: Using OpenCV and DeepFace, image data is analyzed in real time to identify the customer's emotional state (e.g., joy, anxiety, surprise, etc.).

[1741] Output: Customer emotional state data.

[1742] How it works: Facial image data is input into the analysis engine, DeepFace analyzes and identifies the emotional state, and displays the results on the device.

[1743] Step 5:

[1744] The emotion engine generates appropriate feedback and suggestions based on the emotional state identified.

[1745] Input: Customer emotional state data.

[1746] Data processing and computation: The emotion engine generates pre-defined feedback and suggestions based on the captured emotional state.

[1747] Output: Feedback regarding specific ways to respond to store staff and customer support.

[1748] Specific behavior: If the emotion engine identifies "anxiety," it will display recommended actions, such as "explain things gently" to the store clerk, on the smart glasses display.

[1749] Step 6:

[1750] The user (store clerk) provides appropriate customer service based on the analysis results and feedback.

[1751] Input: Analysis results and feedback provided by the device.

[1752] Data processing and calculation: The store clerk uses the analysis results and feedback to provide appropriate support to the customer.

[1753] Output: Hospitality and product suggestions to customers.

[1754] Specific actions: The store clerk follows the instructions of the smart glasses and provides customer service by speaking to them in a friendly manner and explaining the products in detail.

[1755] Step 7:

[1756] The server periodically collects new training data and continuously trains the model to improve its accuracy.

[1757] Input: Emotion recognition data and customer service outcome data collected in physical stores.

[1758] Data processing and calculation: The server uses the collected data to retrain the AI ​​model to improve its accuracy.

[1759] Output: Improved AI model data.

[1760] Specific operation: The server periodically analyzes the data collected from the physical store and improves the model. As a result, a new AI model is distributed to the device again.

[1761] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1762] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1763] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1764] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1765] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1766] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1767] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1768] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1769] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1770] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1771] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1772] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1773] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1774] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1775] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1776] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1777] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1778] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1779] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1780] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1781] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1782] The following is further disclosed regarding the above embodiment.

[1783] (Claim 1)

[1784] A means for storing the miniaturized artificial intelligence model in a storage medium for multi-point data processing;

[1785] means for transmitting the latest artificial intelligence model via a network in order to periodically update the artificial intelligence model stored in the storage medium;

[1786] means for analyzing the collected data in real time using an artificial intelligence model stored in a storage medium and providing the analysis results to a user;

[1787] A system including:

[1788] (Claim 2)

[1789] 2. The system according to claim 1, further comprising means for performing data analysis using an artificial intelligence model stored in a storage medium even under conditions where the network is unstable or unavailable.

[1790] (Claim 3)

[1791] 2. The system according to claim 1, further comprising means for optimizing rescue operations at a disaster site using an artificial intelligence model stored in a storage medium even when a communication network is unavailable.

[1792] (Claim 4)

[1793] 2. The system according to claim 1, further comprising a means for analyzing data collected by a researcher in a remote location at a research site in real time and determining the direction of the research.

[1794] "Example 1"

[1795] (Claim 1)

[1796] A means for training an artificial intelligence model based on the latest training data using high-performance computing resources, converting it for portable use, and transmitting it to a terminal;

[1797] A means for the terminal to receive update notifications from the server and download and install new artificial intelligence models;

[1798] A means for inputting various data collected by a user using the data collection tool into a terminal and storing the data in a storage medium;

[1799] a means for the terminal to preprocess the data, analyze the data in real time using an artificial intelligence model stored in the storage medium, and provide the analysis result to the user;

[1800] a means for a user to make decisions based on the analysis results;

[1801] A system including:

[1802] (Claim 2)

[1803] 2. The system according to claim 1, further comprising means for performing data analysis using an artificial intelligence model stored in a storage medium even under conditions where the network is unstable or unavailable.

[1804] (Claim 3)

[1805] 2. The system according to claim 1, further comprising means for optimizing rescue operations at a disaster site using an artificial intelligence model stored in a storage medium even when a communication network is unavailable.

[1806] "Application Example 1"

[1807] (Claim 1)

[1808] A means for storing the miniaturized artificial intelligence model in a storage medium for multi-point data processing;

[1809] means for transmitting the latest artificial intelligence model via a network in order to periodically update the artificial intelligence model stored in the storage medium;

[1810] means for analyzing the collected data in real time using an artificial intelligence model stored in a storage medium and providing the analysis results to a user;

[1811] Even in an unstable communication environment, the system processes video data acquired from a camera mounted on the vehicle in real time to detect the presence or absence of obstacles and lane deviations.

[1812] A system including:

[1813] (Claim 2)

[1814] 2. The system according to claim 1, further comprising means for performing data analysis using an artificial intelligence model stored in a storage medium even under conditions where the network is unstable or unavailable.

[1815] (Claim 3)

[1816] The system according to claim 1, which is equipped with a means for detecting obstacles and pedestrians in real time from camera images ahead of an autonomous vehicle, even when the communication environment is unstable, to ensure the safety of operation.

[1817] "Example 2: Combining Emotion Engines"

[1818] (Claim 1)

[1819] A means for storing the miniaturized artificial intelligence model in a storage medium for multi-point data processing;

[1820] means for transmitting the latest artificial intelligence model via a network in order to periodically update the artificial intelligence model stored in the storage medium;

[1821] means for analyzing the collected data in real time using an artificial intelligence model stored in a storage medium and providing the analysis results to a user;

[1822] means for performing real-time emotion analysis using data obtained from the sensors to recognize the emotional state of a user;

[1823] means for providing appropriate feedback and suggestions based on the user's emotional state;

[1824] A system including:

[1825] (Claim 2)

[1826] 2. The system according to claim 1, further comprising means for performing data analysis using an artificial intelligence model stored in a storage medium even under conditions where the network is unstable or unavailable.

[1827] (Claim 3)

[1828] 2. The system according to claim 1, further comprising means for optimizing rescue operations at a disaster site using an artificial intelligence model stored in a storage medium even when a communication network is unavailable.

[1829] "Application example 2 when combining emotion engines"

[1830] (Claim 1)

[1831] A means for storing the miniaturized artificial intelligence model in a storage medium for multi-point information processing;

[1832] a means for transmitting the latest artificial intelligence model via a communication line in order to periodically update the artificial intelligence model stored in the storage medium;

[1833] a means for analyzing the collected information in real time using an artificial intelligence model stored in a storage medium and providing the analysis results to a user;

[1834] a means for providing emotion-based feedback and suggestions using an emotion engine that analyzes the user's emotional state;

[1835] A system including:

[1836] (Claim 2)

[1837] 2. The system according to claim 1, further comprising means for performing information analysis using an artificial intelligence model stored in a storage medium even under circumstances where communication lines are unstable or unavailable.

[1838] (Claim 3)

[1839] The system according to claim 1, further comprising a means for analyzing the emotional state of customers in real time using an artificial intelligence model stored in a storage medium and providing appropriate customer support, even when a communication line is unavailable in a physical store. [Explanation of symbols]

[1840] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for storing the miniaturized artificial intelligence model in a storage medium for multi-point data processing; means for transmitting the latest artificial intelligence model via a network in order to periodically update the artificial intelligence model stored in the storage medium; means for analyzing the collected data in real time using an artificial intelligence model stored in a storage medium and providing the analysis results to a user; A system including:

2. 2. The system according to claim 1, further comprising means for performing data analysis using an artificial intelligence model stored in a storage medium even under conditions where the network is unstable or unavailable.

3. 2. The system according to claim 1, further comprising means for optimizing rescue operations at a disaster site using an artificial intelligence model stored in a storage medium even when a communication network is unavailable.

4. 2. The system according to claim 1, further comprising a means for analyzing data collected by a researcher at a remote location in real time at the research site and determining the direction of the research.

Images