system

Abstract

Provide a system. The present invention includes: a generative model means for analyzing video data acquired from a monitoring device; a means for generating a work log based on the results of analysis by the generative model means; means for generating the work log as a non-fungible token; A means for providing the non-fungible tokens to users; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] It is difficult to secure consumer trust in the safety and quality transparency of food manufacturing processes, and many consumers are skeptical because they cannot verify the authenticity or quality of products. Rapid detection of errors and abnormalities in the manufacturing process and improvement of quality are also challenges. In light of these challenges, there is a need to improve the reliability and transparency of food manufacturing processes, provide consumers with peace of mind, and improve manufacturing quality through error detection. [Means for solving the problem]

[0005] The present invention provides a system including a generative model means for analyzing video data acquired from a monitoring device, a means for generating a work log based on the results of analysis by the generative model means, a means for generating the work log as a non-fungible token, and a means for providing the non-fungible token to a user. This system ensures the transparency and reliability of the food manufacturing process by detecting errors and abnormalities in the manufacturing process and providing a record of such errors and abnormalities to consumers as a non-fungible token, thereby providing consumers with a sense of security.

[0006] A "monitoring device" is a device for acquiring real-time video data from the food manufacturing process.

[0007] The "generative model means" is a means that uses an artificial intelligence model to analyze acquired video data and detect errors and abnormalities in the work process.

[0008] The "work log" is data that records the details of each work process in chronological order based on the results of analysis by the generative model means.

[0009] A "non-fungible token" is a token that uses blockchain technology to represent unique digital ownership, and is a digital certificate that includes information such as work logs and product origin information.

[0010] "User" refers to a consumer who purchases a product and receives a non-fungible token corresponding to that product.

[0011] The "food manufacturing process" refers to the series of steps by which food is produced, from the procurement of raw materials to the shipping of the final product.

[0012] "Blockchain" is a system that uses distributed ledger technology to store and manage data in a secure and reliable manner. [Brief explanation of the drawings]

[0013] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] This invention is a system that utilizes video data acquired from a monitoring device to visualize food manufacturing processes and improve reliability. Below, we will explain the processing of a program that is centered around a server, terminals, and users.

[0035] Server Processing

[0036] The server acquires video data in real time from monitoring devices installed in the factory. The video data sent from each camera is temporarily stored in the server's storage. This video data is pre-processed and subjected to necessary processing such as noise removal before being input into the generative model.

[0037] The generative model analyzes video data using deep learning and other artificial intelligence techniques. Specifically, it monitors worker movements and product status in real time during the manufacturing process to detect errors and anomalies. Based on the analysis results, the server generates a work log, recording the details of the work process in chronological order.

[0038] The generated work logs are then compiled and generated as a non-fungible token (NFT), which includes information on the product's origin, manufacturing process, quality inspection results, etc. Finally, the server registers the NFT on the blockchain, and ownership is transferred to the consumer when the product is shipped.

[0039] Terminal handling

[0040] Terminals in the factory display the video data sent from the server in real time. Workers and managers can use the terminals to check whether their work is being carried out correctly. Notifications of errors and abnormalities sent from the server are also displayed in real time, providing immediate feedback to workers and managers.

[0041] For example, if a worker accidentally performs an inappropriate operation on a product, the generative model will detect the anomaly and display an error notification on the terminal, allowing the manager to quickly resolve the problem and maintain quality.

[0042] User Action

[0043] When users purchase a product, they receive a corresponding NFT, which records the product's detailed manufacturing process, origin information, quality inspection results, etc., allowing users to verify the product's authenticity.

[0044] For example, if a user purchases a high-quality melon, they can view the corresponding NFT through a smartphone app. By viewing the NFT, the user can obtain a series of information about the farm where the melon was grown, harvested, and shipped. This ensures the authenticity of the product, allowing users to make a purchase with confidence.

[0045] This system will enable food manufacturers to provide consumers with reliable information, and its error detection function in the manufacturing process is expected to improve product quality.

[0046] The processing flow will be explained below.

[0047] Server Processing

[0048] Step 1:

[0049] The server acquires video data in real time from the monitoring devices installed in the factory. Each monitoring camera transmits the video data to the server via the network. The received video data is first temporarily stored in the server's storage.

[0050] Step 2:

[0051] The server pre-processes the stored video data, performing noise removal and frame division to optimize the data for analysis. This pre-processed data is then input into the generative model.

[0052] Step 3:

[0053] The server analyzes the pre-processed video data using a generative model (a model that uses artificial intelligence techniques such as deep learning). The generative model monitors the movements of workers and the status of products in real time to detect errors and anomalies.

[0054] Step 4:

[0055] The server generates a work log based on the results of the analysis by the generative model. The work log includes information such as what operations were performed in which process, and details of any detected errors or anomalies. This work log is recorded in chronological order.

[0056] Step 5:

[0057] The server generates a non-fungible token (NFT) based on the generated work log. The NFT is configured to include information on the product's origin, manufacturing process, quality inspection results, etc.

[0058] Step 6:

[0059] The server registers the generated NFT on the blockchain. The registered NFT is irreversibly stored on the blockchain, and the risk of tampering is low.

[0060] Step 7:

[0061] The server transfers ownership of the corresponding NFT to the consumer when the product is shipped, and this ownership transfer information is also recorded on the blockchain.

[0062] Terminal handling

[0063] Step 1:

[0064] The terminal receives real-time video data sent from the server, and the received video data is displayed on a screen, allowing workers or managers to monitor the status of the manufacturing site in real time.

[0065] Step 2:

[0066] The terminal receives notifications of errors and abnormality detection sent from the server. If an error is detected, the details are displayed on the terminal display in real time.

[0067] Step 3:

[0068] The terminal notifies the operator and manager of the error using sound and light, allowing them to quickly identify and address the problem.

[0069] User Action

[0070] Step 1:

[0071] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[0072] Step 2:

[0073] Users can access the blockchain using a smartphone app or PC, view the NFTs recorded on the blockchain, and check the details of the NFTs they own.

[0074] Step 3:

[0075] Users can view product information written on the NFT, including detailed information such as the product's place of origin, manufacturing process, and quality inspection results. This allows users to confirm the authenticity of the product and consume it with confidence.

[0076] Example 1

[0077] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0078] In conventional food manufacturing processes, there is a lack of visualization of the product manufacturing process and an insufficient improvement in reliability. This makes it difficult to guarantee product quality and gain consumer trust. In addition, detecting errors and abnormalities in the manufacturing process is often done manually, requiring quick and accurate response.

[0079] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0080] In this invention, the server includes means for preprocessing video data acquired from a monitoring device, deep learning model means for analyzing the video data, means for generating a work log based on the analysis results, means for generating the work log as a non-fungible token, and means for providing the non-fungible token to a user. This enables visualization and improvement of reliability of the product manufacturing process, as well as rapid and accurate detection of errors and abnormalities in the manufacturing process.

[0081] "Monitoring equipment" refers to equipment used to monitor and record manufacturing processes in a factory in real time.

[0082] "Video data" refers to visual information acquired by a monitoring device expressed in digital form.

[0083] "Preprocessing" refers to processing of acquired video data, such as noise removal and brightness adjustment.

[0084] A "deep learning model" is an algorithm that uses artificial intelligence technology to analyze large amounts of data and perform pattern recognition and prediction.

[0085] A "work log" is data that records details such as work content and abnormality detection results in the manufacturing process.

[0086] A "non-fungible token" is a digital certificate that uniquely records detailed information about a product or digital asset based on blockchain technology.

[0087] "User" means a person who uses the information and services provided through this system.

[0088] "Error or Anomaly" refers to any condition or behavior in the manufacturing process that deviates from standard procedures or quality standards.

[0089] This invention is a system that utilizes video data acquired from a monitoring device to visualize food manufacturing processes and improve reliability. Below, we will explain in detail the processing of a program that is centered around a server, terminals, and users.

[0090] Server Processing

[0091] The server acquires video data in real time from monitoring devices installed in the factory. The video data sent from each camera is temporarily stored in the server's storage. This video data is pre-processed. The pre-processing process includes noise removal and brightness adjustment for each frame, and image processing filters are applied using the OpenCV library.

[0092] After preprocessing, the video data is input into a generative model using the deep learning libraries TENSORFLOW (registered trademark) and PyTorch. This generative model analyzes the video data frame by frame and monitors the movements of workers and the status of products in the manufacturing process. The generative model detects product defects and inconsistencies in worker behavior in real time.

[0093] Based on the model's analysis results, the server generates an operation log, which records details such as the time the anomaly was detected, the nature of the anomaly, and countermeasures. The operation log is created in JSON format and stored in a database. The server then compiles the operation logs and generates a single non-fungible token (NFT). The NFT contains information about the product's origin, manufacturing process, and quality inspection results. Specifically, a smart contract is created using a blockchain platform such as Ethereum, and the NFT is registered.

[0094] Terminal handling

[0095] Terminals in the factory display the video data sent from the server in real time. The software used is VLC Media Player or other streaming playback software. Workers and managers can use the terminals to check whether their work is being carried out correctly. Notifications of errors and abnormalities sent from the server are also displayed in real time, providing immediate feedback to workers and managers.

[0096] For example, if a worker accidentally performs an inappropriate operation on a product, the generative model will detect the anomaly and display an error notification on the terminal, allowing the manager to quickly resolve the problem and maintain product quality.

[0097] User Action

[0098] After purchasing a product, users will receive a corresponding NFT, which records the product's detailed manufacturing process, country of origin, and quality inspection results. Users can check the NFT's detailed information by scanning the QR code (registered trademark) with a smartphone app.

[0099] For example, if a user purchases a high-quality melon, they can view the corresponding NFT in a smartphone app. By viewing the NFT, the user can obtain a series of information about the farm where the melon was grown, how it passed quality inspections, and how it was shipped.

[0100] Examples and prompts

[0101] A specific example is a process that collects video data from surveillance cameras, uses AI to detect anomalies, records the results in a JSON-formatted log, and generates it as an NFT.

[0102] When processing the device, the video is displayed in real time using VLC media player, and if an abnormality is detected, a pop-up alert is displayed.

[0103] The user process involves scanning the NFT of the purchased product with the app to check the manufacturing process and quality inspection results.

[0104] An example prompt might look like this:

[0105] "I want to check the manufacturing process of the melon I purchased."

[0106] "Show me the quality inspection results recorded on the NFT."

[0107] "Display surveillance camera video data in real time."

[0108] This system will enable food manufacturers to provide consumers with reliable information, and its error detection function in the manufacturing process is expected to improve product quality.

[0109] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0110] Server Processing

[0111] Step 1:

[0112] The server acquires video data in real time from surveillance cameras installed in the factory. Specifically, it receives streaming data generated by IP cameras and high-resolution cameras and temporarily stores it in the server's storage. Input: Video streaming data from surveillance cameras. Output: Raw video data stored in the server's storage.

[0113] Step 2:

[0114] The server preprocesses the acquired video data. During this process, noise is removed and brightness is adjusted for each frame. Specifically, image processing filters are applied using the OpenCV library. Input: Raw video data stored in the server's storage. Output: Preprocessed, clean video data.

[0115] Step 3:

[0116] After preprocessing, the video data is input into a generative model. The generative model uses deep learning libraries such as TensorFlow and PyTorch. The model analyzes the video data frame by frame and monitors the movements of workers and the state of products in real time. Input: Preprocessed, clean video data. Output: Analysis results from the generative model.

[0117] Step 4:

[0118] The generative model analyzes the data and detects errors and anomalies. Specifically, it detects product defects and inconsistencies in worker behavior as anomalies. In this process, pre-set thresholds and pattern recognition algorithms are used. Input: Analysis results by the generative model. Output: Detected data for errors and anomalies.

[0119] Step 5:

[0120] The server generates a work log based on the error or anomaly detection data. The work log records details such as the time the anomaly was detected, the nature of the anomaly, and countermeasures. Specifically, it creates log data in JSON format and saves it in a database. Input: Error or anomaly detection data. Output: JSON format work log.

[0121] Step 6:

[0122] The server compiles the work logs and generates an NFT. The NFT contains information about the product's origin, manufacturing process, and quality inspection results. Specifically, a smart contract is created using a blockchain platform such as Ethereum to register the NFT. Input: Work logs in JSON format. Output: NFT recorded on the blockchain.

[0123] Terminal handling

[0124] Step 1:

[0125] The terminals in the factory display the video data sent from the server in real time. Specifically, VLC Media Player or other streaming playback software is used. Input: Real-time video data sent from the server. Output: Video displayed on the terminal.

[0126] Step 2:

[0127] The terminal displays error and abnormality notifications sent from the server in real time. Specifically, when an abnormality is detected, a pop-up window is displayed and an alert is sent to the worker or administrator. Input: Error notification sent from the server. Output: Error pop-up displayed on the terminal.

[0128] Step 3:

[0129] Workers and managers check the notifications on their devices and take the necessary action quickly. For example, if an abnormality is displayed, the work is stopped immediately and the problem is corrected. Input: Error pop-up displayed on the device. Output: Workers and managers quickly correct the problem.

[0130] User Action

[0131] Step 1:

[0132] After purchasing a product, users receive the corresponding NFT. Specifically, they can check detailed information about the NFT by scanning the QR code with a smartphone app. Input: QR code provided with the product. Output: Detailed information about the NFT displayed on the smartphone app.

[0133] Step 2:

[0134] Through the NFT, users can check the product's detailed manufacturing process, country of origin, and quality inspection results. For example, if they purchase a high-quality melon, they can find out which farm it was grown on and what quality inspections it passed. Input: NFT information read by a smartphone app. Output: Displayed product manufacturing process and quality inspection results.

[0135] Step 3:

[0136] Users will recognize the reliability of the product through NFT and use it with peace of mind. Specifically, they will have access to the product's manufacturing process and inspection results, and will feel secure by obtaining reliable information. Input: Displayed product manufacturing process and quality inspection results. Output: Recognition of product reliability and a sense of security.

[0137] (Application example 1)

[0138] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0139] In modern factories and production lines, improving the efficiency of product quality control and security management is a major challenge. However, existing systems make it difficult to detect anomalies in real time and take immediate action. Furthermore, to increase product reliability, detailed records of the manufacturing process and the provision of that information to consumers are necessary, but systems for doing so are often not in place. As such, integrating security and quality control within factories remains a major challenge.

[0140] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0141] In this invention, the server includes generative model means for analyzing video data acquired from a monitoring device, means for generating a work log based on the results of analysis by the generative model means, means for generating the work log as a non-fungible token, means for providing the non-fungible token to a user, notification means for acquiring video data from the monitoring device in real time and detecting errors or abnormalities, and means for displaying a notification of abnormality detection by the notification means in real time. This not only makes it possible to detect abnormalities in real time and take immediate action, but also makes it possible to provide consumers with detailed information to increase product reliability.

[0142] A "monitoring device" is a device that is installed in a factory and acquires video data in real time.

[0143] A "generative model" is a system that uses deep learning and artificial intelligence techniques to analyze video data and detect errors and anomalies.

[0144] A "work log" is data that records the details of the manufacturing process in chronological order based on the results of analysis by the generative model.

[0145] A "non-fungible token" is digital data that uses blockchain technology and cannot be altered or copied once issued.

[0146] "Users" refer to consumers who purchase the factory's products and receive non-fungible tokens, as well as factory managers.

[0147] "Notification means" is a mechanism that presents information in real time when an abnormality or error is detected.

[0148] The "real-time display means" is a device that instantly displays information from the notification means on a terminal and quickly provides information to the relevant parties.

[0149] This invention is a system that utilizes video data acquired from monitoring devices in factories to improve the efficiency of factory security and quality control, and to increase the reliability of products for consumers. The following describes the processing of a program that is centered around a server, terminals, and users.

[0150] Server Action:

[0151] The server first acquires video data in real time from monitoring devices installed in the factory. The acquired video data is temporarily stored in the server's storage. After undergoing preprocessing such as noise removal, the video data is input into a generative model such as a deep learning model. The generative model analyzes the video data using deep learning and other artificial intelligence technologies, monitoring the movements of workers and the status of products in real time during the manufacturing process and detecting abnormalities and errors. Based on the results of this analysis, the server generates a work log, recording details of the work process in chronological order. The generated work log is generated as an NFT (non-fungible token), which includes information such as the product's place of origin, manufacturing process, and quality inspection results. Finally, the server registers this NFT on the blockchain, and ownership is transferred to the consumer when the product is shipped.

[0152] Terminal handling:

[0153] Video data sent from the server can be displayed in real time on terminals within the factory. Workers and managers can use the terminals to check whether their work is being carried out correctly, and notifications of errors and abnormalities detected from the server are also displayed in real time. For example, if a worker mistakenly performs an inappropriate operation on a product, the generative model will detect the abnormality and an error notification will immediately appear on the terminal. As a result, managers can quickly resolve the problem and maintain product quality.

[0154] User Action:

[0155] When users purchase a product, they receive a corresponding NFT. The NFT contains detailed information about the product's manufacturing process, country of origin, and quality inspection results, allowing users to verify the product's authenticity. For example, when a user purchases a luxury food ingredient, they can view the corresponding NFT through a smartphone app. By viewing the NFT, users can obtain a series of information about how the product was manufactured and how quality inspections were carried out. This ensures the authenticity of the product, allowing users to purchase with confidence.

[0156] Prompt Sentence Examples

[0157] Examples of prompts to input to a generative AI model include:

[0158] "We set up IP addresses for cameras in factories and automatically preprocess the video data captured by the cameras. We then input that data into a TensorFlow model to detect anomalies in real time. If an anomaly is detected, we send a push notification to the administrator and register the details of the anomaly as an NFT on the blockchain."

[0159] This system not only strengthens factory security and enables immediate detection and response to abnormalities and errors, but also provides consumers with detailed product information and increases their reliability.

[0160] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0161] Step 1:

[0162] The server acquires video data in real time from monitoring devices installed in the factory. The video data sent from the camera is first sent to the server and temporarily saved in storage. This saved data undergoes pre-processing such as noise removal and image sharpening.

[0163] Input: Real-time video data from the camera

[0164] Output: Pre-processed video data

[0165] Specific operations: noise removal, grayscale conversion, edge enhancement

[0166] Step 2:

[0167] The preprocessed video data is input into a generative AI model. The server uses this model to analyze the video data and monitor the movements of workers and the status of products during the manufacturing process. If an error or abnormality is detected, the server proceeds to the next step based on the analysis results.

[0168] Input: Preprocessed video data

[0169] Output: Analysis results (presence or absence of errors or abnormalities)

[0170] Specific operation: Detecting anomalies using deep learning models and generating analysis results

[0171] Step 3:

[0172] The server generates a work log based on the analysis results. The work log records the timestamp of the anomaly or error, the product information in question, and other details. This work log is then generated as a non-fungible token (NFT) that includes further related information.

[0173] Input: Analysis results

[0174] Output: Work log (detailed in chronological order) and NFT

[0175] Specific operations: generating work logs, creating NFTs

[0176] Step 4:

[0177] The server registers the generated NFT on the blockchain, which makes it impossible to tamper with work logs and product information, ensuring authenticity. When the product is shipped, ownership of the corresponding NFT is transferred to the consumer.

[0178] Input: NFT

[0179] Output: NFT registered on the blockchain

[0180] Specific operations: NFT blockchain registration, ownership management

[0181] Step 5:

[0182] Terminals in the factory display the video data and analysis results sent from the server in real time. When an abnormality is detected, the terminal displays a notification on the screen and sends feedback to workers and managers, allowing workers to quickly respond to the problem.

[0183] Input: Video data and analysis results from the server

[0184] Output: Real-time video display, abnormality notification display

[0185] Specific operations: Display of video data, real-time display of abnormality notifications

[0186] Step 6:

[0187] When users purchase a product, they receive a corresponding NFT, which records the product's detailed manufacturing process, country of origin, and quality inspection results, and users can access this information on their smartphones.

[0188] Input: NFT

[0189] Output: User-visible product information

[0190] Specific operation: Displaying NFT and checking product information through a smartphone app

[0191] These steps will ensure efficient security and quality control at the factory and enable reliable information to be provided to consumers.

[0192] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0193] This invention combines a system that utilizes video data acquired from a monitoring device to visualize and improve the reliability of food manufacturing processes with an emotion engine that recognizes user emotions. Below, we will explain the processing of the program, which is centered on the server, terminal, and user.

[0194] Server Processing

[0195] The server acquires video data in real time from the monitoring devices installed in the factory. Each monitoring camera transmits the video data to the server via the network. The received video data is first temporarily stored in the server's storage.

[0196] The server performs preprocessing on the stored video data. After preprocessing, such as noise removal and frame division, the data is input into a generative model. The generative model analyzes the video data using artificial intelligence techniques such as deep learning, monitors the movements of workers and the status of products in real time, and detects errors and abnormalities.

[0197] Based on the results of the analysis by the generative model, the server generates a work log, which includes details of which operations were performed in which process, as well as any detected errors or anomalies. This work log is recorded in chronological order.

[0198] The server then generates a non-fungible token (NFT) based on the generated work log. The NFT contains information about the product's origin, manufacturing process, and quality inspection results. The server registers the generated NFT on the blockchain and transfers ownership of the corresponding NFT to the consumer when the product is shipped.

[0199] Furthermore, the system incorporates an emotion engine that recognizes users' emotions. The server uses the emotion engine to analyze the emotional data of users who receive NFTs and evaluate their satisfaction.

[0200] Terminal handling

[0201] Terminals in the factory receive real-time video data sent from the server, which is then displayed on a screen, allowing workers and managers to monitor the status of the manufacturing site in real time.

[0202] The terminal receives notifications of errors and abnormalities sent from the server and displays them in real time. The details of the error are notified to the worker or manager using sound or light, allowing the problem to be identified and dealt with quickly.

[0203] User Action

[0204] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[0205] Users can access the blockchain using a smartphone app or PC and view the recorded NFT. The NFT contains detailed information about the product's manufacturing process, country of origin, quality inspection results, etc., allowing users to verify the authenticity of the product.

[0206] For example, if a user purchases a high-quality melon, they receive an NFT. The emotion engine analyzes the emotional data when the user views the NFT through a smartphone app. If the user's satisfaction is rated high, additional product information or special offers can be provided based on that information. Conversely, if the user's satisfaction is low, feedback is collected and areas for improvement are identified.

[0207] This system will enable food manufacturers to provide consumers with reliable information, and its error detection function in the manufacturing process is expected to improve product quality. Furthermore, by incorporating an emotion engine, it will be possible to grasp user satisfaction in real time, leading to improved services.

[0208] The processing flow will be explained below.

[0209] Server Processing

[0210] Step 1:

[0211] The server acquires video data in real time from monitoring devices installed in the factory. Each camera transmits the video data to the server via the network, and the server temporarily stores it in storage.

[0212] Step 2:

[0213] The server performs preprocessing on the stored video data, such as noise removal, frame division, and resolution adjustment, to prepare the data in a format suitable for the generative model.

[0214] Step 3:

[0215] The server then inputs the pre-processed video data into a generative model, which then uses deep learning and other artificial intelligence techniques to analyze the data and monitor the manufacturing process in real time.

[0216] Step 4:

[0217] Based on the results of the analysis by the generative model, the server generates a work log that records the details of each work step, including operation details, time information, errors and abnormalities, etc.

[0218] Step 5:

[0219] The server generates a non-fungible token (NFT) based on the generated work log, which includes information such as the product's origin and quality inspection results, in addition to the work log.

[0220] Step 6:

[0221] The server registers the generated NFT on the blockchain, where it is recorded and stored in an immutable form.

[0222] Step 7:

[0223] The server transfers ownership of the corresponding NFT to the consumer when the product is shipped, and this ownership transfer information is also recorded on the blockchain.

[0224] Step 8:

[0225] The server launches an emotion engine that analyzes the emotional data of users who receive NFTs, receives emotional data from the user's smartphone app, and evaluates their level of satisfaction.

[0226] Terminal handling

[0227] Step 1:

[0228] The terminal receives real-time video data sent from the server and displays it on a screen, allowing workers and managers to monitor the status of the manufacturing site in real time.

[0229] Step 2:

[0230] The terminal receives notifications of errors and abnormalities sent from the server, and the notification contents are displayed on the screen in real time.

[0231] Step 3:

[0232] The terminal notifies the operator and manager of any errors with sound and light, allowing them to quickly identify and address the problem.

[0233] User Action

[0234] Step 1:

[0235] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[0236] Step 2:

[0237] Users can access the blockchain using a smartphone app or PC and view the recorded NFT, which contains detailed information about the product's manufacturing process, country of origin, and quality inspection results, allowing users to verify the product's authenticity.

[0238] Step 3:

[0239] When a user browses an NFT, the emotion engine collects emotional data from the user, for example, by analyzing facial expressions and voice using a smartphone's camera and sensors.

[0240] Step 4:

[0241] The emotion engine analyzes the collected emotion data and evaluates the user's satisfaction. The evaluation results are sent to the server and used to improve user satisfaction.

[0242] For example, when a user purchases a high-quality melon and views the corresponding NFT, the emotion engine analyzes the emotional data. If the user's satisfaction is rated high, additional product information and special offers will be provided. If the user's satisfaction is low, feedback will be collected and used to improve the service next time.

[0243] Example 2

[0244] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0245] While conventional food manufacturing process monitoring systems have the ability to visualize the manufacturing process and detect errors, they lack the means to provide information to increase product reliability or to grasp consumer satisfaction in real time. Furthermore, consumers today are increasingly demanding transparency and reliability in products, and want detailed information about the product's manufacturing process and quality. To solve these issues, a new system is needed that combines real-time monitoring data analysis with detailed information provided by non-fungible tokens (NFTs), and also incorporates an emotion engine to evaluate consumer satisfaction.

[0246] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0247] In this invention, the server includes generative model means for analyzing video data acquired from a monitoring device, means for generating a work log based on the results of analysis by the generative model means, means for generating the work log as a non-fungible token, means for providing the non-fungible token to a user, and emotion engine means for analyzing user emotion data. This enables visualization of the manufacturing process and real-time detection of errors, and allows detailed information on product reliability to be provided to consumers, as well as real-time evaluation of consumer satisfaction.

[0248] A "monitoring device" is a device that captures images of the manufacturing process in a factory in real time and acquires video data.

[0249] The "generative model means" is a means for analyzing video data acquired from a surveillance device using deep learning and artificial intelligence techniques.

[0250] The "work log" is a log that records details of operations and abnormality detection in the manufacturing process, and is generated based on the results of analysis by the generative model means.

[0251] A non-fungible token (NFT) is a token that records detailed product information and has uniqueness as a digital asset.

[0252] The "emotion engine means" is a means for analyzing the user's emotion data and evaluating the degree of satisfaction.

[0253] The "function to detect errors and abnormalities" is a function for detecting defects and abnormal operations in the manufacturing process in real time.

[0254] "Product origin information" is detailed information about the place of origin of a product.

[0255] "Manufacturing process information" is detailed information about each step in the manufacturing process of a product.

[0256] "User" refers to a consumer who purchases a product and receives non-fungible tokens.

[0257] This invention combines a system that utilizes video data acquired from a monitoring device to visualize food manufacturing processes and improve reliability, with an emotion engine that recognizes user emotions. Specific embodiments of the system, which is centered around a server, terminals, and users, are described below.

[0258] Server Processing

[0259] The server acquires video data in real time from monitoring devices installed in the factory. Specifically, the monitoring cameras use RTSP (Real-Time Streaming Protocol) to stream video data to the server. The received video data is first temporarily stored in cloud storage such as Amazon S3.

[0260] The server preprocesses the stored video data using Python's OpenCV library. This preprocessing includes noise removal and frame segmentation, which improves the accuracy of the analysis. The preprocessed video data is then input into a generative model. The generative model is built using frameworks such as TensorFlow and PyTorch and uses deep learning to analyze the video data. The analysis results include worker movements and product status, and also detect errors and abnormalities in the manufacturing process.

[0261] Based on the results of the analysis by the generative model, the server generates and records a work log in a NoSQL database such as MongoDB. The work log includes details of operations performed at each stage and any abnormalities.

[0262] The server then generates a non-fungible token (NFT) based on the transaction log and registers it on the Ethereum blockchain using Python's Web3 library. The NFT contains information about the product's origin, manufacturing process, and quality inspection results, which can be verified by consumers.

[0263] The emotion engine analyzes the emotional data when a user receives an NFT. It uses an NLP library for emotion analysis and evaluates the user's satisfaction. This allows us to understand how users feel about the product information in real time.

[0264] Terminal handling

[0265] Terminals in the factory receive real-time video data sent from the server. The video data is received via WebSocket or similar and displayed on the terminal's display. Workers and managers can monitor the video using software such as VLC media player.

[0266] The terminal receives notifications of errors and abnormalities sent from the server and notifies workers and managers in real time with sound and light, enabling them to quickly identify and address problems.

[0267] User Action

[0268] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[0269] Users can access a blockchain viewer such as EtherScan to view the recorded NFT, which contains detailed information about the product's manufacturing process, origin, and quality inspection results, allowing users to verify the product's authenticity.

[0270] For example, if a user purchases a luxury melon, they can view the NFT and its detailed information on their smartphone app. The emotion engine then analyzes the user's emotional data and evaluates their level of satisfaction. If satisfaction is high, additional product information and special offers are provided. Conversely, if satisfaction is low, feedback is collected and areas for improvement are identified.

[0271] Examples of prompt statements

[0272] Here are some example prompts to input to a generative AI model:

[0273] "Design a program to analyze video data acquired from factory monitoring equipment, visualize the manufacturing process, and detect errors."

[0274] This system will enable food manufacturers to provide consumers with reliable information, and the error detection function in the manufacturing process is expected to improve product quality. In addition, by incorporating an emotion engine, it will be possible to grasp user satisfaction in real time, leading to improved services.

[0275] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0276] Step 1: Acquire video data

[0277] The server acquires video data in real time from the monitoring devices installed in the factory. Each monitoring camera transmits the video data to the server via streaming using RTSP (Real-Time Streaming Protocol).

[0278] Input: Real-time video data sent from a surveillance device

[0279] Output: Raw data stored on the server

[0280] Specific operation: Save video data to cloud storage such as Amazon S3.

[0281] Step 2: Preprocessing the video data

[0282] The server performs preprocessing on the stored video data, using the Python OpenCV library to remove noise and split the video data into frames.

[0283] Input: Raw data stored on the server

[0284] Output: Pre-processed video data

[0285] Specific operation: Apply an average filter to reduce noise and extract a specific frame.

[0286] Step 3: Analysis with a generative model

[0287] The pre-processed video data is then fed into a generative model, which is built using the TensorFlow and PyTorch frameworks to analyze the data and detect errors and anomalies in the manufacturing process.

[0288] Input: Preprocessed video data

[0289] Output: Analysis results for each process and abnormality detection information

[0290] Specific operation: Data is input into the generative AI model, which analyzes the worker's actions and the product's condition. If an abnormality is detected, an alert is generated.

[0291] Step 4: Generate a work log

[0292] Based on the analysis results, the server generates a work log, using a NoSQL database such as MongoDB to record information about each process and any anomalies in chronological order.

[0293] Input: Analysis results from the generative model

[0294] Output: Work log

[0295] Specific operation: A text log is generated based on the analysis results and saved in a database.

[0296] Step 5: Generate and register your NFT

[0297] The server generates a non-fungible token (NFT) based on the generated work log and registers the NFT on the Ethereum blockchain using Python's Web3 library.

[0298] Input: Work Log

[0299] Output: NFT registered on the blockchain

[0300] Specific operation: Generates an NFT containing information on the product's origin, manufacturing process, and quality inspection results, and manages it using a smart contract.

[0301] Step 6: Receive real-time video and error reports

[0302] The terminal receives real-time video data and error notifications sent from the server, allowing workers and managers to monitor the status of the manufacturing site in real time.

[0303] Input: Video data and error notifications from the server

[0304] Output: Real-time video and notifications displayed on the display

[0305] Specific operation: Receives video data via WebSocket and displays it in VLC media player. If an abnormality is detected, it notifies you with sound and light.

[0306] Step 7: Receive and view your NFT

[0307] Users receive the corresponding NFT when they purchase a product, transfer ownership of the NFT through a smartphone app or online platform, and then access a blockchain viewer such as EtherScan to view the recorded NFT.

[0308] Input: NFT data from the server

[0309] Output: NFT information viewed on the user's smartphone or PC

[0310] How it works: Use an app like MetaMask to receive your NFT and then use EtherScan to view detailed manufacturing and origin information.

[0311] Step 8: Analyze the sentiment data

[0312] The server analyzes the emotional data when users receive NFTs, and uses an NLP library to analyze user feedback and comments to evaluate user satisfaction.

[0313] Input: User feedback and comments

[0314] Output: User satisfaction evaluation results

[0315] How it works: Feedback data is fed into an NLP library, and satisfaction is scored based on sentiment analysis. Additional product information and special offers are provided based on the evaluation results.

[0316] (Application example 2)

[0317] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0318] In conventional manufacturing processes, there are systems for product quality control and improving the reliability of data related to the manufacturing process, but there are limited means of providing reliable information directly to consumers. Furthermore, it is difficult to grasp consumer satisfaction in real time, making it difficult to quickly implement specific countermeasures. This has prevented companies from properly understanding consumer needs and emotions and improving the quality of their services.

[0319] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0320] In this invention, the server includes generative model means for analyzing video data acquired from a monitoring device, means for generating a work log based on the results of analysis by the generative model means, means for generating the work log as a non-fungible token, means for providing the non-fungible token to a user, emotion analysis means for recognizing user emotions, and means for evaluating user satisfaction based on emotion data analyzed by the emotion analysis means. This makes it possible to provide consumers with reliable product information, grasp consumer purchasing behavior and satisfaction in real time, and improve service quality.

[0321] A "monitoring device" is a device used to monitor a specific area, such as a factory or a brick-and-mortar store, in real time and acquire video data.

[0322] "Video data" refers to real-time visual information acquired from a monitoring device, and is the data to be analyzed.

[0323] A "generative model means" is a model incorporating deep learning and artificial intelligence techniques that is used to analyze captured video data and extract specific information.

[0324] The "work log" is data that records which work was performed in which process based on the results of analysis by the generative model means.

[0325] A "non-fungible token" is a unique digital asset created based on blockchain technology that contains information about the product's origin and manufacturing process.

[0326] "User" refers to a consumer or customer who uses the system and receives information about products and services.

[0327] "Emotion analysis means" is a technology for analyzing a user's facial expressions and behavior to determine their emotional state at that time.

[0328] "Satisfaction" refers to the degree of satisfaction a user feels with a product or service, and is evaluated by sentiment analysis means.

[0329] The present invention provides a system that combines a monitoring device, a generative model, work log generation, non-fungible token generation and provision, emotion analysis means, and satisfaction assessment means.

[0330] Server Processing

[0331] The server acquires video data in real time from monitoring devices installed in factories and brick-and-mortar stores. Each monitoring camera transmits video data to the server via the network. The received video data is first temporarily stored in the server's storage.

[0332] The server performs preprocessing on the stored video data, including noise removal and frame segmentation. The preprocessed data is then input into a generative model (e.g., TensorFlow, Keras). This generative model uses deep learning technology to analyze the video data, monitor the actions of workers and the status of products in real time, and detect errors and anomalies.

[0333] Based on the analysis results, the server generates a work log, which includes details of what operations were performed at which stage and any errors or anomalies detected. This work log is recorded in chronological order. A non-fungible token (NFT) is then generated based on the generated work log. The NFT includes information about the product's origin, manufacturing process, quality inspection results, etc. The server registers the generated NFT on a blockchain (e.g., Ethereum, Hyperledger Fabric), and transfers ownership of the corresponding NFT to the consumer when the product is shipped.

[0334] Furthermore, this system incorporates an emotion analysis mechanism to recognize users' emotions. The server uses an emotion analysis engine (e.g., Affectiva, Microsoft® Emotion API) to analyze the emotional data of users who receive NFTs and evaluate their satisfaction.

[0335] Terminal handling

[0336] Terminals in factories or physical stores receive real-time video data sent from the server. The received video data is displayed on a screen, allowing workers and managers to monitor the status of the manufacturing site or physical store in real time. The terminals receive notifications of errors and abnormality detection sent from the server and display them in real time. The details of the error are notified to workers and managers with sound or light, allowing them to quickly identify and address the problem.

[0337] User Action

[0338] When a user purchases a product, they receive a corresponding NFT. Purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user. Users can access the blockchain using a smartphone app or PC to view the recorded NFT. The NFT contains detailed information about the product's manufacturing process, country of origin, and quality inspection results, allowing users to verify the product's authenticity.

[0339] For example, if a user purchases a smartphone, they can receive a corresponding NFT. The sentiment analysis engine analyzes emotional data when the user browses the NFT through a smartphone app. For example, if the user expresses excitement or satisfaction while browsing, that information can be used to provide additional product information or special offers. Conversely, if the user expresses dissatisfaction, feedback can be collected and areas for improvement identified.

[0340] Example prompt for a generative AI model:

[0341] / / Customer sentiment analysis prompt

[0342] Input data: Video data captured by a camera

[0343] Goal: Identify customer emotions (happiness, anger, confusion, excitement, etc.) from video data and display the results in real time.

[0344] This system will enable manufacturers and brick-and-mortar store operators to provide consumers with reliable information, and is expected to improve product quality through error detection functions in the manufacturing process and sales environment.Furthermore, by using sentiment analysis tools, it will be possible to grasp user satisfaction in real time, leading to improved services.

[0345] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0346] Step 1:

[0347] The server acquires video data in real time from monitoring devices installed in factories and brick-and-mortar stores. Each monitoring camera sends video data to the server via the network, and this video data is temporarily stored in the server's storage. The input is video data from the monitoring device, and the output is the temporarily stored video data. Specifically, the server checks the connection of the monitoring camera and transfers the video data to the server in a stream format.

[0348] Step 2:

[0349] The server performs preprocessing on the stored video data, such as noise removal and frame division. This preprocessing improves data quality and increases analysis accuracy. The input is temporarily stored video data, and the output is preprocessed video data. Specifically, OpenCV is used to perform noise filtering and divide continuous video into individual frames.

[0350] Step 3:

[0351] The server inputs the preprocessed video data into a generative model to analyze the worker's actions and the product's status. Generative AI models (e.g., TensorFlow, Keras) are used to detect errors and anomalies in real time. The input is the preprocessed video data, and the output is the error and anomaly detection results. Specifically, features are extracted for each frame, and abnormal actions are identified using a deep learning model.

[0352] Step 4:

[0353] The server generates a work log based on the analysis results obtained from the generative model and records it in chronological order. The input is the error and anomaly detection results, and the output is the work log. Specifically, the analysis results are recorded in a log file in text format, and a timestamp is added to each log.

[0354] Step 5:

[0355] The server generates a non-fungible token (NFT) based on the generated work log and registers it on the blockchain. The input is the work log, and the output is the NFT registered on the blockchain. Specifically, the work log is converted to JSON format and registered on the blockchain via a smart contract.

[0356] Step 6:

[0357] The terminal receives real-time video data and error notifications sent from the server and displays them on the display. The input is video data and error notifications from the server, and the output is displayed on the display. Specifically, the video data is streamed, and if an error occurs, the worker is notified with sound and light.

[0358] Step 7:

[0359] When a user purchases a product, they receive the corresponding NFT and view it on a smartphone app or PC. The input is purchase information, and the output is the display of the NFT on the user's smartphone or PC. Specifically, the user enters their purchase information and transfers NFT ownership on the blockchain.

[0360] Step 8:

[0361] The server uses an emotion analysis engine to analyze the user's emotional data when viewing NFTs and evaluate their satisfaction. The input is the user's facial expression data, and the output is the emotion analysis results and satisfaction rating. Specifically, it uses Affectiva and the Microsoft Emotion API to analyze emotions from the user's facial expressions and provides feedback to the user along with the results.

[0362] Example prompt sentence:

[0363] / / Customer sentiment analysis prompt

[0364] Input data: Video data captured by a camera

[0365] Goal: Identify customer emotions (happiness, anger, confusion, excitement, etc.) from video data and display the results in real time.

[0366] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0367] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0368] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0369] [Second embodiment]

[0370] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0371] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0372] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0373] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0374] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0375] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0376] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0377] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0378] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0379] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0380] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0381] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0382] This invention is a system that utilizes video data acquired from a monitoring device to visualize food manufacturing processes and improve reliability. Below, we will explain the processing of a program that is centered around a server, terminals, and users.

[0383] Server Processing

[0384] The server acquires video data in real time from monitoring devices installed in the factory. The video data sent from each camera is temporarily stored in the server's storage. This video data is pre-processed and subjected to necessary processing such as noise removal before being input into the generative model.

[0385] The generative model analyzes video data using deep learning and other artificial intelligence techniques. Specifically, it monitors worker movements and product status in real time during the manufacturing process to detect errors and anomalies. Based on the analysis results, the server generates a work log, recording the details of the work process in chronological order.

[0386] The generated work logs are then compiled and generated as a non-fungible token (NFT), which includes information on the product's origin, manufacturing process, quality inspection results, etc. Finally, the server registers the NFT on the blockchain, and ownership is transferred to the consumer when the product is shipped.

[0387] Terminal handling

[0388] Terminals in the factory display the video data sent from the server in real time. Workers and managers can use the terminals to check whether their work is being carried out correctly. Notifications of errors and abnormalities sent from the server are also displayed in real time, providing immediate feedback to workers and managers.

[0389] For example, if a worker accidentally performs an inappropriate operation on a product, the generative model will detect the anomaly and display an error notification on the terminal, allowing the manager to quickly resolve the problem and maintain quality.

[0390] User Action

[0391] When users purchase a product, they receive a corresponding NFT, which records the product's detailed manufacturing process, origin information, quality inspection results, etc., allowing users to verify the product's authenticity.

[0392] For example, if a user purchases a high-quality melon, they can view the corresponding NFT through a smartphone app. By viewing the NFT, the user can obtain a series of information about the farm where the melon was grown, harvested, and shipped. This ensures the authenticity of the product, allowing users to make a purchase with confidence.

[0393] This system will enable food manufacturers to provide consumers with reliable information, and its error detection function in the manufacturing process is expected to improve product quality.

[0394] The processing flow will be explained below.

[0395] Server Processing

[0396] Step 1:

[0397] The server acquires video data in real time from the monitoring devices installed in the factory. Each monitoring camera transmits the video data to the server via the network. The received video data is first temporarily stored in the server's storage.

[0398] Step 2:

[0399] The server pre-processes the stored video data, performing noise removal and frame division to optimize the data for analysis. This pre-processed data is then input into the generative model.

[0400] Step 3:

[0401] The server analyzes the pre-processed video data using a generative model (a model that uses artificial intelligence techniques such as deep learning). The generative model monitors the movements of workers and the status of products in real time to detect errors and anomalies.

[0402] Step 4:

[0403] The server generates a work log based on the results of the analysis by the generative model. The work log includes information such as what operations were performed in which process, and details of any detected errors or anomalies. This work log is recorded in chronological order.

[0404] Step 5:

[0405] The server generates a non-fungible token (NFT) based on the generated work log. The NFT is configured to include information on the product's origin, manufacturing process, quality inspection results, etc.

[0406] Step 6:

[0407] The server registers the generated NFT on the blockchain. The registered NFT is irreversibly stored on the blockchain, and the risk of tampering is low.

[0408] Step 7:

[0409] The server transfers ownership of the corresponding NFT to the consumer when the product is shipped, and this ownership transfer information is also recorded on the blockchain.

[0410] Terminal handling

[0411] Step 1:

[0412] The terminal receives real-time video data sent from the server, and the received video data is displayed on a screen, allowing workers or managers to monitor the status of the manufacturing site in real time.

[0413] Step 2:

[0414] The terminal receives notifications of errors and abnormality detection sent from the server. If an error is detected, the details are displayed on the terminal display in real time.

[0415] Step 3:

[0416] The terminal notifies the operator and manager of the error using sound and light, allowing them to quickly identify and address the problem.

[0417] User Action

[0418] Step 1:

[0419] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[0420] Step 2:

[0421] Users can access the blockchain using a smartphone app or PC, view the NFTs recorded on the blockchain, and check the details of the NFTs they own.

[0422] Step 3:

[0423] Users can view product information written on the NFT, including detailed information such as the product's place of origin, manufacturing process, and quality inspection results. This allows users to confirm the authenticity of the product and consume it with confidence.

[0424] Example 1

[0425] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0426] In conventional food manufacturing processes, there is a lack of visualization of the product manufacturing process and an insufficient improvement in reliability. This makes it difficult to guarantee product quality and gain consumer trust. In addition, detecting errors and abnormalities in the manufacturing process is often done manually, requiring quick and accurate response.

[0427] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0428] In this invention, the server includes means for preprocessing video data acquired from a monitoring device, deep learning model means for analyzing the video data, means for generating a work log based on the analysis results, means for generating the work log as a non-fungible token, and means for providing the non-fungible token to a user. This enables visualization and improvement of reliability of the product manufacturing process, as well as rapid and accurate detection of errors and abnormalities in the manufacturing process.

[0429] "Monitoring equipment" refers to equipment used to monitor and record manufacturing processes in a factory in real time.

[0430] "Video data" refers to visual information acquired by a monitoring device expressed in digital form.

[0431] "Preprocessing" refers to processing of acquired video data, such as noise removal and brightness adjustment.

[0432] A "deep learning model" is an algorithm that uses artificial intelligence technology to analyze large amounts of data and perform pattern recognition and prediction.

[0433] A "work log" is data that records details such as work content and abnormality detection results in the manufacturing process.

[0434] A "non-fungible token" is a digital certificate that uniquely records detailed information about a product or digital asset based on blockchain technology.

[0435] "User" means a person who uses the information and services provided through this system.

[0436] "Error or Anomaly" refers to any condition or behavior in the manufacturing process that deviates from standard procedures or quality standards.

[0437] This invention is a system that utilizes video data acquired from a monitoring device to visualize food manufacturing processes and improve reliability. Below, we will explain in detail the processing of a program that is centered around a server, terminals, and users.

[0438] Server Processing

[0439] The server acquires video data in real time from monitoring devices installed in the factory. The video data sent from each camera is temporarily stored in the server's storage. This video data is pre-processed. The pre-processing process includes noise removal and brightness adjustment for each frame, and image processing filters are applied using the OpenCV library.

[0440] After preprocessing, the video data is input into a generative model using deep learning libraries such as TensorFlow and PyTorch. This generative model analyzes the video data frame by frame to monitor the movements of workers and the status of products in the manufacturing process. The generative model detects product defects and inconsistencies in worker behavior in real time.

[0441] Based on the model's analysis results, the server generates an operation log, which records details such as the time the anomaly was detected, the nature of the anomaly, and countermeasures. The operation log is created in JSON format and stored in a database. The server then compiles the operation logs and generates a single non-fungible token (NFT). The NFT contains information about the product's origin, manufacturing process, and quality inspection results. Specifically, a smart contract is created using a blockchain platform such as Ethereum, and the NFT is registered.

[0442] Terminal handling

[0443] Terminals in the factory display the video data sent from the server in real time. The software used is VLC Media Player or other streaming playback software. Workers and managers can use the terminals to check whether their work is being carried out correctly. Notifications of errors and abnormalities sent from the server are also displayed in real time, providing immediate feedback to workers and managers.

[0444] For example, if a worker accidentally performs an inappropriate operation on a product, the generative model will detect the anomaly and display an error notification on the terminal, allowing the manager to quickly resolve the problem and maintain product quality.

[0445] User Action

[0446] After purchasing a product, users will receive a corresponding NFT, which records the product's detailed manufacturing process, country of origin, and quality inspection results. Users can check the NFT's detailed information by scanning the QR code with a smartphone app.

[0447] For example, if a user purchases a high-quality melon, they can view the corresponding NFT in a smartphone app. By viewing the NFT, the user can obtain a series of information about the farm where the melon was grown, how it passed quality inspections, and how it was shipped.

[0448] Examples and prompts

[0449] A specific example is a process that collects video data from surveillance cameras, uses AI to detect anomalies, records the results in a JSON-formatted log, and generates it as an NFT.

[0450] When processing the device, the video is displayed in real time using VLC media player, and if an abnormality is detected, a pop-up alert is displayed.

[0451] The user process involves scanning the NFT of the purchased product with the app to check the manufacturing process and quality inspection results.

[0452] An example prompt might look like this:

[0453] "I want to check the manufacturing process of the melon I purchased."

[0454] "Show me the quality inspection results recorded on the NFT."

[0455] "Display surveillance camera video data in real time."

[0456] This system will enable food manufacturers to provide consumers with reliable information, and its error detection function in the manufacturing process is expected to improve product quality.

[0457] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0458] Server Processing

[0459] Step 1:

[0460] The server acquires video data in real time from surveillance cameras installed in the factory. Specifically, it receives streaming data generated by IP cameras and high-resolution cameras and temporarily stores it in the server's storage. Input: Video streaming data from surveillance cameras. Output: Raw video data stored in the server's storage.

[0461] Step 2:

[0462] The server preprocesses the acquired video data. During this process, noise is removed and brightness is adjusted for each frame. Specifically, image processing filters are applied using the OpenCV library. Input: Raw video data stored in the server's storage. Output: Preprocessed, clean video data.

[0463] Step 3:

[0464] After preprocessing, the video data is input into a generative model. The generative model uses deep learning libraries such as TensorFlow and PyTorch. The model analyzes the video data frame by frame and monitors the movements of workers and the state of products in real time. Input: Preprocessed, clean video data. Output: Analysis results from the generative model.

[0465] Step 4:

[0466] The generative model analyzes the data and detects errors and anomalies. Specifically, it detects product defects and inconsistencies in worker behavior as anomalies. In this process, pre-set thresholds and pattern recognition algorithms are used. Input: Analysis results by the generative model. Output: Detected data for errors and anomalies.

[0467] Step 5:

[0468] The server generates a work log based on the error or anomaly detection data. The work log records details such as the time the anomaly was detected, the nature of the anomaly, and countermeasures. Specifically, it creates log data in JSON format and saves it in a database. Input: Error or anomaly detection data. Output: JSON format work log.

[0469] Step 6:

[0470] The server compiles the work logs and generates an NFT. The NFT contains information about the product's origin, manufacturing process, and quality inspection results. Specifically, a smart contract is created using a blockchain platform such as Ethereum to register the NFT. Input: Work logs in JSON format. Output: NFT recorded on the blockchain.

[0471] Terminal handling

[0472] Step 1:

[0473] The terminals in the factory display the video data sent from the server in real time. Specifically, VLC Media Player or other streaming playback software is used. Input: Real-time video data sent from the server. Output: Video displayed on the terminal.

[0474] Step 2:

[0475] The terminal displays error and abnormality notifications sent from the server in real time. Specifically, when an abnormality is detected, a pop-up window is displayed and an alert is sent to the worker or administrator. Input: Error notification sent from the server. Output: Error pop-up displayed on the terminal.

[0476] Step 3:

[0477] Workers and managers check the notifications on their devices and take the necessary action quickly. For example, if an abnormality is displayed, the work is stopped immediately and the problem is corrected. Input: Error pop-up displayed on the device. Output: Workers and managers quickly correct the problem.

[0478] User Action

[0479] Step 1:

[0480] After purchasing a product, users receive the corresponding NFT. Specifically, they can check detailed information about the NFT by scanning the QR code with a smartphone app. Input: QR code provided with the product. Output: Detailed information about the NFT displayed on the smartphone app.

[0481] Step 2:

[0482] Through the NFT, users can check the product's detailed manufacturing process, country of origin, and quality inspection results. For example, if they purchase a high-quality melon, they can find out which farm it was grown on and what quality inspections it passed. Input: NFT information read by a smartphone app. Output: Displayed product manufacturing process and quality inspection results.

[0483] Step 3:

[0484] Users will recognize the reliability of the product through NFT and use it with peace of mind. Specifically, they will have access to the product's manufacturing process and inspection results, and will feel secure by obtaining reliable information. Input: Displayed product manufacturing process and quality inspection results. Output: Recognition of product reliability and a sense of security.

[0485] (Application example 1)

[0486] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0487] In modern factories and production lines, improving the efficiency of product quality control and security management is a major challenge. However, existing systems make it difficult to detect anomalies in real time and take immediate action. Furthermore, to increase product reliability, detailed records of the manufacturing process and the provision of that information to consumers are necessary, but systems for doing so are often not in place. As such, integrating security and quality control within factories remains a major challenge.

[0488] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0489] In this invention, the server includes generative model means for analyzing video data acquired from a monitoring device, means for generating a work log based on the results of analysis by the generative model means, means for generating the work log as a non-fungible token, means for providing the non-fungible token to a user, notification means for acquiring video data from the monitoring device in real time and detecting errors or abnormalities, and means for displaying a notification of abnormality detection by the notification means in real time. This not only makes it possible to detect abnormalities in real time and take immediate action, but also makes it possible to provide consumers with detailed information to increase product reliability.

[0490] A "monitoring device" is a device that is installed in a factory and acquires video data in real time.

[0491] A "generative model" is a system that uses deep learning and artificial intelligence techniques to analyze video data and detect errors and anomalies.

[0492] A "work log" is data that records the details of the manufacturing process in chronological order based on the results of analysis by the generative model.

[0493] A "non-fungible token" is digital data that uses blockchain technology and cannot be altered or copied once issued.

[0494] "Users" refer to consumers who purchase the factory's products and receive non-fungible tokens, as well as factory managers.

[0495] "Notification means" is a mechanism that presents information in real time when an abnormality or error is detected.

[0496] The "real-time display means" is a device that instantly displays information from the notification means on a terminal and quickly provides information to the relevant parties.

[0497] This invention is a system that utilizes video data acquired from monitoring devices in factories to improve the efficiency of factory security and quality control, and to increase the reliability of products for consumers. The following describes the processing of a program that is centered around a server, terminals, and users.

[0498] Server Action:

[0499] The server first acquires video data in real time from monitoring devices installed in the factory. The acquired video data is temporarily stored in the server's storage. After undergoing preprocessing such as noise removal, the video data is input into a generative model such as a deep learning model. The generative model analyzes the video data using deep learning and other artificial intelligence technologies, monitoring the movements of workers and the status of products in real time during the manufacturing process and detecting abnormalities and errors. Based on the results of this analysis, the server generates a work log, recording details of the work process in chronological order. The generated work log is generated as an NFT (non-fungible token), which includes information such as the product's place of origin, manufacturing process, and quality inspection results. Finally, the server registers this NFT on the blockchain, and ownership is transferred to the consumer when the product is shipped.

[0500] Terminal handling:

[0501] Video data sent from the server can be displayed in real time on terminals within the factory. Workers and managers can use the terminals to check whether their work is being carried out correctly, and notifications of errors and abnormalities detected from the server are also displayed in real time. For example, if a worker mistakenly performs an inappropriate operation on a product, the generative model will detect the abnormality and an error notification will immediately appear on the terminal. As a result, managers can quickly resolve the problem and maintain product quality.

[0502] User Action:

[0503] When users purchase a product, they receive a corresponding NFT. The NFT contains detailed information about the product's manufacturing process, country of origin, and quality inspection results, allowing users to verify the product's authenticity. For example, when a user purchases a luxury food ingredient, they can view the corresponding NFT through a smartphone app. By viewing the NFT, users can obtain a series of information about how the product was manufactured and how quality inspections were carried out. This ensures the authenticity of the product, allowing users to purchase with confidence.

[0504] Prompt Sentence Examples

[0505] Examples of prompts to input to a generative AI model include:

[0506] "We set up IP addresses for cameras in factories and automatically preprocess the video data captured by the cameras. We then input that data into a TensorFlow model to detect anomalies in real time. If an anomaly is detected, we send a push notification to the administrator and register the details of the anomaly as an NFT on the blockchain."

[0507] This system not only strengthens factory security and enables immediate detection and response to abnormalities and errors, but also provides consumers with detailed product information and increases their reliability.

[0508] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0509] Step 1:

[0510] The server acquires video data in real time from monitoring devices installed in the factory. The video data sent from the camera is first sent to the server and temporarily saved in storage. This saved data undergoes pre-processing such as noise removal and image sharpening.

[0511] Input: Real-time video data from the camera

[0512] Output: Pre-processed video data

[0513] Specific operations: noise removal, grayscale conversion, edge enhancement

[0514] Step 2:

[0515] The preprocessed video data is input into a generative AI model. The server uses this model to analyze the video data and monitor the movements of workers and the status of products during the manufacturing process. If an error or abnormality is detected, the server proceeds to the next step based on the analysis results.

[0516] Input: Preprocessed video data

[0517] Output: Analysis results (presence or absence of errors or abnormalities)

[0518] Specific operation: Detecting anomalies using deep learning models and generating analysis results

[0519] Step 3:

[0520] The server generates a work log based on the analysis results. The work log records the timestamp of the anomaly or error, the product information in question, and other details. This work log is then generated as a non-fungible token (NFT) that includes further related information.

[0521] Input: Analysis results

[0522] Output: Work log (detailed in chronological order) and NFT

[0523] Specific operations: generating work logs, creating NFTs

[0524] Step 4:

[0525] The server registers the generated NFT on the blockchain, which makes it impossible to tamper with work logs and product information, ensuring authenticity. When the product is shipped, ownership of the corresponding NFT is transferred to the consumer.

[0526] Input: NFT

[0527] Output: NFT registered on the blockchain

[0528] Specific operations: NFT blockchain registration, ownership management

[0529] Step 5:

[0530] Terminals in the factory display the video data and analysis results sent from the server in real time. When an abnormality is detected, the terminal displays a notification on the screen and sends feedback to workers and managers, allowing workers to quickly respond to the problem.

[0531] Input: Video data and analysis results from the server

[0532] Output: Real-time video display, abnormality notification display

[0533] Specific operations: Display of video data, real-time display of abnormality notifications

[0534] Step 6:

[0535] When users purchase a product, they receive a corresponding NFT, which records the product's detailed manufacturing process, country of origin, and quality inspection results, and users can access this information on their smartphones.

[0536] Input: NFT

[0537] Output: User-visible product information

[0538] Specific operation: Displaying NFT and checking product information through a smartphone app

[0539] These steps will ensure efficient security and quality control at the factory and enable reliable information to be provided to consumers.

[0540] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0541] This invention combines a system that utilizes video data acquired from a monitoring device to visualize and improve the reliability of food manufacturing processes with an emotion engine that recognizes user emotions. Below, we will explain the processing of the program, which is centered on the server, terminal, and user.

[0542] Server Processing

[0543] The server acquires video data in real time from the monitoring devices installed in the factory. Each monitoring camera transmits the video data to the server via the network. The received video data is first temporarily stored in the server's storage.

[0544] The server performs preprocessing on the stored video data. After preprocessing, such as noise removal and frame division, the data is input into a generative model. The generative model analyzes the video data using artificial intelligence techniques such as deep learning, monitors the movements of workers and the status of products in real time, and detects errors and abnormalities.

[0545] Based on the results of the analysis by the generative model, the server generates a work log, which includes details of which operations were performed in which process, as well as any detected errors or anomalies. This work log is recorded in chronological order.

[0546] The server then generates a non-fungible token (NFT) based on the generated work log. The NFT contains information about the product's origin, manufacturing process, and quality inspection results. The server registers the generated NFT on the blockchain and transfers ownership of the corresponding NFT to the consumer when the product is shipped.

[0547] Furthermore, the system incorporates an emotion engine that recognizes users' emotions. The server uses the emotion engine to analyze the emotional data of users who receive NFTs and evaluate their satisfaction.

[0548] Terminal handling

[0549] Terminals in the factory receive real-time video data sent from the server, which is then displayed on a screen, allowing workers and managers to monitor the status of the manufacturing site in real time.

[0550] The terminal receives notifications of errors and abnormalities sent from the server and displays them in real time. The details of the error are notified to the worker or manager using sound or light, allowing the problem to be identified and dealt with quickly.

[0551] User Action

[0552] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[0553] Users can access the blockchain using a smartphone app or PC and view the recorded NFT. The NFT contains detailed information about the product's manufacturing process, country of origin, quality inspection results, etc., allowing users to verify the authenticity of the product.

[0554] For example, if a user purchases a high-quality melon, they receive an NFT. The emotion engine analyzes the emotional data when the user views the NFT through a smartphone app. If the user's satisfaction is rated high, additional product information or special offers can be provided based on that information. Conversely, if the user's satisfaction is low, feedback is collected and areas for improvement are identified.

[0555] This system will enable food manufacturers to provide consumers with reliable information, and its error detection function in the manufacturing process is expected to improve product quality. Furthermore, by incorporating an emotion engine, it will be possible to grasp user satisfaction in real time, leading to improved services.

[0556] The processing flow will be explained below.

[0557] Server Processing

[0558] Step 1:

[0559] The server acquires video data in real time from monitoring devices installed in the factory. Each camera transmits the video data to the server via the network, and the server temporarily stores it in storage.

[0560] Step 2:

[0561] The server performs preprocessing on the stored video data, such as noise removal, frame division, and resolution adjustment, to prepare the data in a format suitable for the generative model.

[0562] Step 3:

[0563] The server then inputs the pre-processed video data into a generative model, which then uses deep learning and other artificial intelligence techniques to analyze the data and monitor the manufacturing process in real time.

[0564] Step 4:

[0565] Based on the results of the analysis by the generative model, the server generates a work log that records the details of each work step, including operation details, time information, errors and abnormalities, etc.

[0566] Step 5:

[0567] The server generates a non-fungible token (NFT) based on the generated work log, which includes information such as the product's origin and quality inspection results, in addition to the work log.

[0568] Step 6:

[0569] The server registers the generated NFT on the blockchain, where it is recorded and stored in an immutable form.

[0570] Step 7:

[0571] The server transfers ownership of the corresponding NFT to the consumer when the product is shipped, and this ownership transfer information is also recorded on the blockchain.

[0572] Step 8:

[0573] The server launches an emotion engine that analyzes the emotional data of users who receive NFTs, receives emotional data from the user's smartphone app, and evaluates their level of satisfaction.

[0574] Terminal handling

[0575] Step 1:

[0576] The terminal receives real-time video data sent from the server and displays it on a screen, allowing workers and managers to monitor the status of the manufacturing site in real time.

[0577] Step 2:

[0578] The terminal receives notifications of errors and abnormalities sent from the server, and the notification contents are displayed on the screen in real time.

[0579] Step 3:

[0580] The terminal notifies the operator and manager of any errors with sound and light, allowing them to quickly identify and address the problem.

[0581] User Action

[0582] Step 1:

[0583] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[0584] Step 2:

[0585] Users can access the blockchain using a smartphone app or PC and view the recorded NFT, which contains detailed information about the product's manufacturing process, country of origin, and quality inspection results, allowing users to verify the product's authenticity.

[0586] Step 3:

[0587] When a user browses an NFT, the emotion engine collects emotional data from the user, for example, by analyzing facial expressions and voice using a smartphone's camera and sensors.

[0588] Step 4:

[0589] The emotion engine analyzes the collected emotion data and evaluates the user's satisfaction. The evaluation results are sent to the server and used to improve user satisfaction.

[0590] For example, when a user purchases a high-quality melon and views the corresponding NFT, the emotion engine analyzes the emotional data. If the user's satisfaction is rated high, additional product information and special offers will be provided. If the user's satisfaction is low, feedback will be collected and used to improve the service next time.

[0591] Example 2

[0592] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0593] While conventional food manufacturing process monitoring systems have the ability to visualize the manufacturing process and detect errors, they lack the means to provide information to increase product reliability or to grasp consumer satisfaction in real time. Furthermore, consumers today are increasingly demanding transparency and reliability in products, and want detailed information about the product's manufacturing process and quality. To solve these issues, a new system is needed that combines real-time monitoring data analysis with detailed information provided by non-fungible tokens (NFTs), and also incorporates an emotion engine to evaluate consumer satisfaction.

[0594] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0595] In this invention, the server includes generative model means for analyzing video data acquired from a monitoring device, means for generating a work log based on the results of analysis by the generative model means, means for generating the work log as a non-fungible token, means for providing the non-fungible token to a user, and emotion engine means for analyzing user emotion data. This enables visualization of the manufacturing process and real-time detection of errors, and allows detailed information on product reliability to be provided to consumers, as well as real-time evaluation of consumer satisfaction.

[0596] A "monitoring device" is a device that captures images of the manufacturing process in a factory in real time and acquires video data.

[0597] The "generative model means" is a means for analyzing video data acquired from a surveillance device using deep learning and artificial intelligence techniques.

[0598] The "work log" is a log that records details of operations and abnormality detection in the manufacturing process, and is generated based on the results of analysis by the generative model means.

[0599] A non-fungible token (NFT) is a token that records detailed product information and has uniqueness as a digital asset.

[0600] The "emotion engine means" is a means for analyzing the user's emotion data and evaluating the degree of satisfaction.

[0601] The "function to detect errors and abnormalities" is a function for detecting defects and abnormal operations in the manufacturing process in real time.

[0602] "Product origin information" is detailed information about the place of origin of a product.

[0603] "Manufacturing process information" is detailed information about each step in the manufacturing process of a product.

[0604] "User" refers to a consumer who purchases a product and receives non-fungible tokens.

[0605] This invention combines a system that utilizes video data acquired from a monitoring device to visualize food manufacturing processes and improve reliability, with an emotion engine that recognizes user emotions. Specific embodiments of the system, which is centered around a server, terminals, and users, are described below.

[0606] Server Processing

[0607] The server acquires video data in real time from monitoring devices installed in the factory. Specifically, the monitoring cameras use RTSP (Real-Time Streaming Protocol) to stream video data to the server. The received video data is first temporarily stored in cloud storage such as Amazon S3.

[0608] The server preprocesses the stored video data using Python's OpenCV library. This preprocessing includes noise removal and frame segmentation, which improves the accuracy of the analysis. The preprocessed video data is then input into a generative model. The generative model is built using frameworks such as TensorFlow and PyTorch and uses deep learning to analyze the video data. The analysis results include worker movements and product status, and also detect errors and abnormalities in the manufacturing process.

[0609] Based on the results of the analysis by the generative model, the server generates and records a work log in a NoSQL database such as MongoDB. The work log includes details of operations performed at each stage and any abnormalities.

[0610] The server then generates a non-fungible token (NFT) based on the transaction log and registers it on the Ethereum blockchain using Python's Web3 library. The NFT contains information about the product's origin, manufacturing process, and quality inspection results, which can be verified by consumers.

[0611] The emotion engine analyzes the emotional data when a user receives an NFT. It uses an NLP library for emotion analysis and evaluates the user's satisfaction. This allows us to understand how users feel about the product information in real time.

[0612] Terminal handling

[0613] Terminals in the factory receive real-time video data sent from the server. The video data is received via WebSocket or similar and displayed on the terminal's display. Workers and managers can monitor the video using software such as VLC media player.

[0614] The terminal receives notifications of errors and abnormalities sent from the server and notifies workers and managers in real time with sound and light, enabling them to quickly identify and address problems.

[0615] User Action

[0616] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[0617] Users can access a blockchain viewer such as EtherScan to view the recorded NFT, which contains detailed information about the product's manufacturing process, origin, and quality inspection results, allowing users to verify the product's authenticity.

[0618] For example, if a user purchases a luxury melon, they can view the NFT and its detailed information on their smartphone app. The emotion engine then analyzes the user's emotional data and evaluates their level of satisfaction. If satisfaction is high, additional product information and special offers are provided. Conversely, if satisfaction is low, feedback is collected and areas for improvement are identified.

[0619] Examples of prompt statements

[0620] Here are some example prompts to input to a generative AI model:

[0621] "Design a program to analyze video data acquired from factory monitoring equipment, visualize the manufacturing process, and detect errors."

[0622] This system will enable food manufacturers to provide consumers with reliable information, and the error detection function in the manufacturing process is expected to improve product quality. In addition, by incorporating an emotion engine, it will be possible to grasp user satisfaction in real time, leading to improved services.

[0623] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0624] Step 1: Acquire video data

[0625] The server acquires video data in real time from the monitoring devices installed in the factory. Each monitoring camera transmits the video data to the server via streaming using RTSP (Real-Time Streaming Protocol).

[0626] Input: Real-time video data sent from a surveillance device

[0627] Output: Raw data stored on the server

[0628] Specific operation: Save video data to cloud storage such as Amazon S3.

[0629] Step 2: Preprocessing the video data

[0630] The server performs preprocessing on the stored video data, using the Python OpenCV library to remove noise and split the video data into frames.

[0631] Input: Raw data stored on the server

[0632] Output: Pre-processed video data

[0633] Specific operation: Apply an average filter to reduce noise and extract a specific frame.

[0634] Step 3: Analysis with a generative model

[0635] The pre-processed video data is then fed into a generative model, which is built using the TensorFlow and PyTorch frameworks to analyze the data and detect errors and anomalies in the manufacturing process.

[0636] Input: Preprocessed video data

[0637] Output: Analysis results for each process and abnormality detection information

[0638] Specific operation: Data is input into the generative AI model, which analyzes the worker's actions and the product's condition. If an abnormality is detected, an alert is generated.

[0639] Step 4: Generate a work log

[0640] Based on the analysis results, the server generates a work log, using a NoSQL database such as MongoDB to record information about each process and any anomalies in chronological order.

[0641] Input: Analysis results from the generative model

[0642] Output: Work log

[0643] Specific operation: A text log is generated based on the analysis results and saved in a database.

[0644] Step 5: Generate and register your NFT

[0645] The server generates a non-fungible token (NFT) based on the generated work log and registers the NFT on the Ethereum blockchain using Python's Web3 library.

[0646] Input: Work Log

[0647] Output: NFT registered on the blockchain

[0648] Specific operation: Generates an NFT containing information on the product's origin, manufacturing process, and quality inspection results, and manages it using a smart contract.

[0649] Step 6: Receive real-time video and error reports

[0650] The terminal receives real-time video data and error notifications sent from the server, allowing workers and managers to monitor the status of the manufacturing site in real time.

[0651] Input: Video data and error notifications from the server

[0652] Output: Real-time video and notifications displayed on the display

[0653] Specific operation: Receives video data via WebSocket and displays it in VLC media player. If an abnormality is detected, it notifies you with sound and light.

[0654] Step 7: Receive and view your NFT

[0655] Users receive the corresponding NFT when they purchase a product, transfer ownership of the NFT through a smartphone app or online platform, and then access a blockchain viewer such as EtherScan to view the recorded NFT.

[0656] Input: NFT data from the server

[0657] Output: NFT information viewed on the user's smartphone or PC

[0658] How it works: Use an app like MetaMask to receive your NFT and then use EtherScan to view detailed manufacturing and origin information.

[0659] Step 8: Analyze the sentiment data

[0660] The server analyzes the emotional data when users receive NFTs, and uses an NLP library to analyze user feedback and comments to evaluate user satisfaction.

[0661] Input: User feedback and comments

[0662] Output: User satisfaction evaluation results

[0663] How it works: Feedback data is fed into an NLP library, and satisfaction is scored based on sentiment analysis. Additional product information and special offers are provided based on the evaluation results.

[0664] (Application example 2)

[0665] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0666] In conventional manufacturing processes, there are systems for product quality control and improving the reliability of data related to the manufacturing process, but there are limited means of providing reliable information directly to consumers. Furthermore, it is difficult to grasp consumer satisfaction in real time, making it difficult to quickly implement specific countermeasures. This has prevented companies from properly understanding consumer needs and emotions and improving the quality of their services.

[0667] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0668] In this invention, the server includes generative model means for analyzing video data acquired from a monitoring device, means for generating a work log based on the results of analysis by the generative model means, means for generating the work log as a non-fungible token, means for providing the non-fungible token to a user, emotion analysis means for recognizing user emotions, and means for evaluating user satisfaction based on emotion data analyzed by the emotion analysis means. This makes it possible to provide consumers with reliable product information, grasp consumer purchasing behavior and satisfaction in real time, and improve service quality.

[0669] A "monitoring device" is a device used to monitor a specific area, such as a factory or a brick-and-mortar store, in real time and acquire video data.

[0670] "Video data" refers to real-time visual information acquired from a monitoring device, and is the data to be analyzed.

[0671] A "generative model means" is a model incorporating deep learning and artificial intelligence techniques that is used to analyze captured video data and extract specific information.

[0672] The "work log" is data that records which work was performed in which process based on the results of analysis by the generative model means.

[0673] A "non-fungible token" is a unique digital asset created based on blockchain technology that contains information about the product's origin and manufacturing process.

[0674] "User" refers to a consumer or customer who uses the system and receives information about products and services.

[0675] "Emotion analysis means" is a technology for analyzing a user's facial expressions and behavior to determine their emotional state at that time.

[0676] "Satisfaction" refers to the degree of satisfaction a user feels with a product or service, and is evaluated by sentiment analysis means.

[0677] The present invention provides a system that combines a monitoring device, a generative model, work log generation, non-fungible token generation and provision, emotion analysis means, and satisfaction assessment means.

[0678] Server Processing

[0679] The server acquires video data in real time from monitoring devices installed in factories and brick-and-mortar stores. Each monitoring camera transmits video data to the server via the network. The received video data is first temporarily stored in the server's storage.

[0680] The server performs preprocessing on the stored video data, including noise removal and frame segmentation. The preprocessed data is then input into a generative model (e.g., TensorFlow, Keras). This generative model uses deep learning technology to analyze the video data, monitor the actions of workers and the status of products in real time, and detect errors and anomalies.

[0681] Based on the analysis results, the server generates a work log, which includes details of what operations were performed at which stage and any errors or anomalies detected. This work log is recorded in chronological order. A non-fungible token (NFT) is then generated based on the generated work log. The NFT includes information about the product's origin, manufacturing process, quality inspection results, etc. The server registers the generated NFT on a blockchain (e.g., Ethereum, Hyperledger Fabric), and transfers ownership of the corresponding NFT to the consumer when the product is shipped.

[0682] Furthermore, this system incorporates an emotion analysis mechanism to recognize users' emotions. The server uses an emotion analysis engine (e.g., Affectiva, Microsoft Emotion API) to analyze the emotional data of users who receive NFTs and evaluate their satisfaction.

[0683] Terminal handling

[0684] Terminals in factories or physical stores receive real-time video data sent from the server. The received video data is displayed on a screen, allowing workers and managers to monitor the status of the manufacturing site or physical store in real time. The terminals receive notifications of errors and abnormality detection sent from the server and display them in real time. The details of the error are notified to workers and managers with sound or light, allowing them to quickly identify and address the problem.

[0685] User Action

[0686] When a user purchases a product, they receive a corresponding NFT. Purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user. Users can access the blockchain using a smartphone app or PC to view the recorded NFT. The NFT contains detailed information about the product's manufacturing process, country of origin, and quality inspection results, allowing users to verify the product's authenticity.

[0687] For example, if a user purchases a smartphone, they can receive a corresponding NFT. The sentiment analysis engine analyzes emotional data when the user browses the NFT through a smartphone app. For example, if the user expresses excitement or satisfaction while browsing, that information can be used to provide additional product information or special offers. Conversely, if the user expresses dissatisfaction, feedback can be collected and areas for improvement identified.

[0688] Example prompt for a generative AI model:

[0689] / / Customer sentiment analysis prompt

[0690] Input data: Video data captured by a camera

[0691] Goal: Identify customer emotions (happiness, anger, confusion, excitement, etc.) from video data and display the results in real time.

[0692] This system will enable manufacturers and brick-and-mortar store operators to provide consumers with reliable information, and is expected to improve product quality through error detection functions in the manufacturing process and sales environment.Furthermore, by using sentiment analysis tools, it will be possible to grasp user satisfaction in real time, leading to improved services.

[0693] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0694] Step 1:

[0695] The server acquires video data in real time from monitoring devices installed in factories and brick-and-mortar stores. Each monitoring camera sends video data to the server via the network, and this video data is temporarily stored in the server's storage. The input is video data from the monitoring device, and the output is the temporarily stored video data. Specifically, the server checks the connection of the monitoring camera and transfers the video data to the server in a stream format.

[0696] Step 2:

[0697] The server performs preprocessing on the stored video data, such as noise removal and frame division. This preprocessing improves data quality and increases analysis accuracy. The input is temporarily stored video data, and the output is preprocessed video data. Specifically, OpenCV is used to perform noise filtering and divide continuous video into individual frames.

[0698] Step 3:

[0699] The server inputs the preprocessed video data into a generative model to analyze the worker's actions and the product's status. Generative AI models (e.g., TensorFlow, Keras) are used to detect errors and anomalies in real time. The input is the preprocessed video data, and the output is the error and anomaly detection results. Specifically, features are extracted for each frame, and abnormal actions are identified using a deep learning model.

[0700] Step 4:

[0701] The server generates a work log based on the analysis results obtained from the generative model and records it in chronological order. The input is the error and anomaly detection results, and the output is the work log. Specifically, the analysis results are recorded in a log file in text format, and a timestamp is added to each log.

[0702] Step 5:

[0703] The server generates a non-fungible token (NFT) based on the generated work log and registers it on the blockchain. The input is the work log, and the output is the NFT registered on the blockchain. Specifically, the work log is converted to JSON format and registered on the blockchain via a smart contract.

[0704] Step 6:

[0705] The terminal receives real-time video data and error notifications sent from the server and displays them on the display. The input is video data and error notifications from the server, and the output is displayed on the display. Specifically, the video data is streamed, and if an error occurs, the worker is notified with sound and light.

[0706] Step 7:

[0707] When a user purchases a product, they receive the corresponding NFT and view it on a smartphone app or PC. The input is purchase information, and the output is the display of the NFT on the user's smartphone or PC. Specifically, the user enters their purchase information and transfers NFT ownership on the blockchain.

[0708] Step 8:

[0709] The server uses an emotion analysis engine to analyze the user's emotional data when viewing NFTs and evaluate their satisfaction. The input is the user's facial expression data, and the output is the emotion analysis results and satisfaction rating. Specifically, it uses Affectiva and the Microsoft Emotion API to analyze emotions from the user's facial expressions and provides feedback to the user along with the results.

[0710] Example prompt sentence:

[0711] / / Customer sentiment analysis prompt

[0712] Input data: Video data captured by a camera

[0713] Goal: Identify customer emotions (happiness, anger, confusion, excitement, etc.) from video data and display the results in real time.

[0714] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0715] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0716] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0717] [Third embodiment]

[0718] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0719] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0720] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0721] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0722] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0723] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0724] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0725] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0726] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0727] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0728] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0729] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0730] This invention is a system that utilizes video data acquired from a monitoring device to visualize food manufacturing processes and improve reliability. Below, we will explain the processing of a program that is centered around a server, terminals, and users.

[0731] Server Processing

[0732] The server acquires video data in real time from monitoring devices installed in the factory. The video data sent from each camera is temporarily stored in the server's storage. This video data is pre-processed and subjected to necessary processing such as noise removal before being input into the generative model.

[0733] The generative model analyzes video data using deep learning and other artificial intelligence techniques. Specifically, it monitors worker movements and product status in real time during the manufacturing process to detect errors and anomalies. Based on the analysis results, the server generates a work log, recording the details of the work process in chronological order.

[0734] The generated work logs are then compiled and generated as a non-fungible token (NFT), which includes information on the product's origin, manufacturing process, quality inspection results, etc. Finally, the server registers the NFT on the blockchain, and ownership is transferred to the consumer when the product is shipped.

[0735] Terminal handling

[0736] Terminals in the factory display the video data sent from the server in real time. Workers and managers can use the terminals to check whether their work is being carried out correctly. Notifications of errors and abnormalities sent from the server are also displayed in real time, providing immediate feedback to workers and managers.

[0737] For example, if a worker accidentally performs an inappropriate operation on a product, the generative model will detect the anomaly and display an error notification on the terminal, allowing the manager to quickly resolve the problem and maintain quality.

[0738] User Action

[0739] When users purchase a product, they receive a corresponding NFT, which records the product's detailed manufacturing process, origin information, quality inspection results, etc., allowing users to verify the product's authenticity.

[0740] For example, if a user purchases a high-quality melon, they can view the corresponding NFT through a smartphone app. By viewing the NFT, the user can obtain a series of information about the farm where the melon was grown, harvested, and shipped. This ensures the authenticity of the product, allowing users to make a purchase with confidence.

[0741] This system will enable food manufacturers to provide consumers with reliable information, and its error detection function in the manufacturing process is expected to improve product quality.

[0742] The processing flow will be explained below.

[0743] Server Processing

[0744] Step 1:

[0745] The server acquires video data in real time from the monitoring devices installed in the factory. Each monitoring camera transmits the video data to the server via the network. The received video data is first temporarily stored in the server's storage.

[0746] Step 2:

[0747] The server pre-processes the stored video data, performing noise removal and frame division to optimize the data for analysis. This pre-processed data is then input into the generative model.

[0748] Step 3:

[0749] The server analyzes the pre-processed video data using a generative model (a model that uses artificial intelligence techniques such as deep learning). The generative model monitors the movements of workers and the status of products in real time to detect errors and anomalies.

[0750] Step 4:

[0751] The server generates a work log based on the results of the analysis by the generative model. The work log includes information such as what operations were performed in which process, and details of any detected errors or anomalies. This work log is recorded in chronological order.

[0752] Step 5:

[0753] The server generates a non-fungible token (NFT) based on the generated work log. The NFT is configured to include information on the product's origin, manufacturing process, quality inspection results, etc.

[0754] Step 6:

[0755] The server registers the generated NFT on the blockchain. The registered NFT is irreversibly stored on the blockchain, and the risk of tampering is low.

[0756] Step 7:

[0757] The server transfers ownership of the corresponding NFT to the consumer when the product is shipped, and this ownership transfer information is also recorded on the blockchain.

[0758] Terminal handling

[0759] Step 1:

[0760] The terminal receives real-time video data sent from the server, and the received video data is displayed on a screen, allowing workers or managers to monitor the status of the manufacturing site in real time.

[0761] Step 2:

[0762] The terminal receives notifications of errors and abnormality detection sent from the server. If an error is detected, the details are displayed on the terminal display in real time.

[0763] Step 3:

[0764] The terminal notifies the operator and manager of the error using sound and light, allowing them to quickly identify and address the problem.

[0765] User Action

[0766] Step 1:

[0767] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[0768] Step 2:

[0769] Users can access the blockchain using a smartphone app or PC, view the NFTs recorded on the blockchain, and check the details of the NFTs they own.

[0770] Step 3:

[0771] Users can view product information written on the NFT, including detailed information such as the product's place of origin, manufacturing process, and quality inspection results. This allows users to confirm the authenticity of the product and consume it with confidence.

[0772] Example 1

[0773] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0774] In conventional food manufacturing processes, there is a lack of visualization of the product manufacturing process and an insufficient improvement in reliability. This makes it difficult to guarantee product quality and gain consumer trust. In addition, detecting errors and abnormalities in the manufacturing process is often done manually, requiring quick and accurate response.

[0775] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0776] In this invention, the server includes means for preprocessing video data acquired from a monitoring device, deep learning model means for analyzing the video data, means for generating a work log based on the analysis results, means for generating the work log as a non-fungible token, and means for providing the non-fungible token to a user. This enables visualization and improvement of reliability of the product manufacturing process, as well as rapid and accurate detection of errors and abnormalities in the manufacturing process.

[0777] "Monitoring equipment" refers to equipment used to monitor and record manufacturing processes in a factory in real time.

[0778] "Video data" refers to visual information acquired by a monitoring device expressed in digital form.

[0779] "Preprocessing" refers to processing of acquired video data, such as noise removal and brightness adjustment.

[0780] A "deep learning model" is an algorithm that uses artificial intelligence technology to analyze large amounts of data and perform pattern recognition and prediction.

[0781] A "work log" is data that records details such as work content and abnormality detection results in the manufacturing process.

[0782] A "non-fungible token" is a digital certificate that uniquely records detailed information about a product or digital asset based on blockchain technology.

[0783] "User" means a person who uses the information and services provided through this system.

[0784] "Error or Anomaly" refers to any condition or behavior in the manufacturing process that deviates from standard procedures or quality standards.

[0785] This invention is a system that utilizes video data acquired from a monitoring device to visualize food manufacturing processes and improve reliability. Below, we will explain in detail the processing of a program that is centered around a server, terminals, and users.

[0786] Server Processing

[0787] The server acquires video data in real time from monitoring devices installed in the factory. The video data sent from each camera is temporarily stored in the server's storage. This video data is pre-processed. The pre-processing process includes noise removal and brightness adjustment for each frame, and image processing filters are applied using the OpenCV library.

[0788] After preprocessing, the video data is input into a generative model using deep learning libraries such as TensorFlow and PyTorch. This generative model analyzes the video data frame by frame to monitor the movements of workers and the status of products in the manufacturing process. The generative model detects product defects and inconsistencies in worker behavior in real time.

[0789] Based on the model's analysis results, the server generates an operation log, which records details such as the time the anomaly was detected, the nature of the anomaly, and countermeasures. The operation log is created in JSON format and stored in a database. The server then compiles the operation logs and generates a single non-fungible token (NFT). The NFT contains information about the product's origin, manufacturing process, and quality inspection results. Specifically, a smart contract is created using a blockchain platform such as Ethereum, and the NFT is registered.

[0790] Terminal handling

[0791] Terminals in the factory display the video data sent from the server in real time. The software used is VLC Media Player or other streaming playback software. Workers and managers can use the terminals to check whether their work is being carried out correctly. Notifications of errors and abnormalities sent from the server are also displayed in real time, providing immediate feedback to workers and managers.

[0792] For example, if a worker accidentally performs an inappropriate operation on a product, the generative model will detect the anomaly and display an error notification on the terminal, allowing the manager to quickly resolve the problem and maintain product quality.

[0793] User Action

[0794] After purchasing a product, users will receive a corresponding NFT, which records the product's detailed manufacturing process, country of origin, and quality inspection results. Users can check the NFT's detailed information by scanning the QR code with a smartphone app.

[0795] For example, if a user purchases a high-quality melon, they can view the corresponding NFT in a smartphone app. By viewing the NFT, the user can obtain a series of information about the farm where the melon was grown, how it passed quality inspections, and how it was shipped.

[0796] Examples and prompts

[0797] A specific example is a process that collects video data from surveillance cameras, uses AI to detect anomalies, records the results in a JSON-formatted log, and generates it as an NFT.

[0798] When processing the device, the video is displayed in real time using VLC media player, and if an abnormality is detected, a pop-up alert is displayed.

[0799] The user process involves scanning the NFT of the purchased product with the app to check the manufacturing process and quality inspection results.

[0800] An example prompt might look like this:

[0801] "I want to check the manufacturing process of the melon I purchased."

[0802] "Show me the quality inspection results recorded on the NFT."

[0803] "Display surveillance camera video data in real time."

[0804] This system will enable food manufacturers to provide consumers with reliable information, and its error detection function in the manufacturing process is expected to improve product quality.

[0805] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0806] Server Processing

[0807] Step 1:

[0808] The server acquires video data in real time from surveillance cameras installed in the factory. Specifically, it receives streaming data generated by IP cameras and high-resolution cameras and temporarily stores it in the server's storage. Input: Video streaming data from surveillance cameras. Output: Raw video data stored in the server's storage.

[0809] Step 2:

[0810] The server preprocesses the acquired video data. During this process, noise is removed and brightness is adjusted for each frame. Specifically, image processing filters are applied using the OpenCV library. Input: Raw video data stored in the server's storage. Output: Preprocessed, clean video data.

[0811] Step 3:

[0812] After preprocessing, the video data is input into a generative model. The generative model uses deep learning libraries such as TensorFlow and PyTorch. The model analyzes the video data frame by frame and monitors the movements of workers and the state of products in real time. Input: Preprocessed, clean video data. Output: Analysis results from the generative model.

[0813] Step 4:

[0814] The generative model analyzes the data and detects errors and anomalies. Specifically, it detects product defects and inconsistencies in worker behavior as anomalies. In this process, pre-set thresholds and pattern recognition algorithms are used. Input: Analysis results by the generative model. Output: Detected data for errors and anomalies.

[0815] Step 5:

[0816] The server generates a work log based on the error or anomaly detection data. The work log records details such as the time the anomaly was detected, the nature of the anomaly, and countermeasures. Specifically, it creates log data in JSON format and saves it in a database. Input: Error or anomaly detection data. Output: JSON format work log.

[0817] Step 6:

[0818] The server compiles the work logs and generates an NFT. The NFT contains information about the product's origin, manufacturing process, and quality inspection results. Specifically, a smart contract is created using a blockchain platform such as Ethereum to register the NFT. Input: Work logs in JSON format. Output: NFT recorded on the blockchain.

[0819] Terminal handling

[0820] Step 1:

[0821] The terminals in the factory display the video data sent from the server in real time. Specifically, VLC Media Player or other streaming playback software is used. Input: Real-time video data sent from the server. Output: Video displayed on the terminal.

[0822] Step 2:

[0823] The terminal displays error and abnormality notifications sent from the server in real time. Specifically, when an abnormality is detected, a pop-up window is displayed and an alert is sent to the worker or administrator. Input: Error notification sent from the server. Output: Error pop-up displayed on the terminal.

[0824] Step 3:

[0825] Workers and managers check the notifications on their devices and take the necessary action quickly. For example, if an abnormality is displayed, the work is stopped immediately and the problem is corrected. Input: Error pop-up displayed on the device. Output: Workers and managers quickly correct the problem.

[0826] User Action

[0827] Step 1:

[0828] After purchasing a product, users receive the corresponding NFT. Specifically, they can check detailed information about the NFT by scanning the QR code with a smartphone app. Input: QR code provided with the product. Output: Detailed information about the NFT displayed on the smartphone app.

[0829] Step 2:

[0830] Through the NFT, users can check the product's detailed manufacturing process, country of origin, and quality inspection results. For example, if they purchase a high-quality melon, they can find out which farm it was grown on and what quality inspections it passed. Input: NFT information read by a smartphone app. Output: Displayed product manufacturing process and quality inspection results.

[0831] Step 3:

[0832] Users will recognize the reliability of the product through NFT and use it with peace of mind. Specifically, they will have access to the product's manufacturing process and inspection results, and will feel secure by obtaining reliable information. Input: Displayed product manufacturing process and quality inspection results. Output: Recognition of product reliability and a sense of security.

[0833] (Application example 1)

[0834] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0835] In modern factories and production lines, improving the efficiency of product quality control and security management is a major challenge. However, existing systems make it difficult to detect anomalies in real time and take immediate action. Furthermore, to increase product reliability, detailed records of the manufacturing process and the provision of that information to consumers are necessary, but systems for doing so are often not in place. As such, integrating security and quality control within factories remains a major challenge.

[0836] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0837] In this invention, the server includes generative model means for analyzing video data acquired from a monitoring device, means for generating a work log based on the results of analysis by the generative model means, means for generating the work log as a non-fungible token, means for providing the non-fungible token to a user, notification means for acquiring video data from the monitoring device in real time and detecting errors or abnormalities, and means for displaying a notification of abnormality detection by the notification means in real time. This not only makes it possible to detect abnormalities in real time and take immediate action, but also makes it possible to provide consumers with detailed information to increase product reliability.

[0838] A "monitoring device" is a device that is installed in a factory and acquires video data in real time.

[0839] A "generative model" is a system that uses deep learning and artificial intelligence techniques to analyze video data and detect errors and anomalies.

[0840] A "work log" is data that records the details of the manufacturing process in chronological order based on the results of analysis by the generative model.

[0841] A "non-fungible token" is digital data that uses blockchain technology and cannot be altered or copied once issued.

[0842] "Users" refer to consumers who purchase the factory's products and receive non-fungible tokens, as well as factory managers.

[0843] "Notification means" is a mechanism that presents information in real time when an abnormality or error is detected.

[0844] The "real-time display means" is a device that instantly displays information from the notification means on a terminal and quickly provides information to the relevant parties.

[0845] This invention is a system that utilizes video data acquired from monitoring devices in factories to improve the efficiency of factory security and quality control, and to increase the reliability of products for consumers. The following describes the processing of a program that is centered around a server, terminals, and users.

[0846] Server Action:

[0847] The server first acquires video data in real time from monitoring devices installed in the factory. The acquired video data is temporarily stored in the server's storage. After undergoing preprocessing such as noise removal, the video data is input into a generative model such as a deep learning model. The generative model analyzes the video data using deep learning and other artificial intelligence technologies, monitoring the movements of workers and the status of products in real time during the manufacturing process and detecting abnormalities and errors. Based on the results of this analysis, the server generates a work log, recording details of the work process in chronological order. The generated work log is generated as an NFT (non-fungible token), which includes information such as the product's place of origin, manufacturing process, and quality inspection results. Finally, the server registers this NFT on the blockchain, and ownership is transferred to the consumer when the product is shipped.

[0848] Terminal handling:

[0849] Video data sent from the server can be displayed in real time on terminals within the factory. Workers and managers can use the terminals to check whether their work is being carried out correctly, and notifications of errors and abnormalities detected from the server are also displayed in real time. For example, if a worker mistakenly performs an inappropriate operation on a product, the generative model will detect the abnormality and an error notification will immediately appear on the terminal. As a result, managers can quickly resolve the problem and maintain product quality.

[0850] User Action:

[0851] When users purchase a product, they receive a corresponding NFT. The NFT contains detailed information about the product's manufacturing process, country of origin, and quality inspection results, allowing users to verify the product's authenticity. For example, when a user purchases a luxury food ingredient, they can view the corresponding NFT through a smartphone app. By viewing the NFT, users can obtain a series of information about how the product was manufactured and how quality inspections were carried out. This ensures the authenticity of the product, allowing users to purchase with confidence.

[0852] Prompt Sentence Examples

[0853] Examples of prompts to input to a generative AI model include:

[0854] "We set up IP addresses for cameras in factories and automatically preprocess the video data captured by the cameras. We then input that data into a TensorFlow model to detect anomalies in real time. If an anomaly is detected, we send a push notification to the administrator and register the details of the anomaly as an NFT on the blockchain."

[0855] This system not only strengthens factory security and enables immediate detection and response to abnormalities and errors, but also provides consumers with detailed product information and increases their reliability.

[0856] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0857] Step 1:

[0858] The server acquires video data in real time from monitoring devices installed in the factory. The video data sent from the camera is first sent to the server and temporarily saved in storage. This saved data undergoes pre-processing such as noise removal and image sharpening.

[0859] Input: Real-time video data from the camera

[0860] Output: Pre-processed video data

[0861] Specific operations: noise removal, grayscale conversion, edge enhancement

[0862] Step 2:

[0863] The preprocessed video data is input into a generative AI model. The server uses this model to analyze the video data and monitor the movements of workers and the status of products during the manufacturing process. If an error or abnormality is detected, the server proceeds to the next step based on the analysis results.

[0864] Input: Preprocessed video data

[0865] Output: Analysis results (presence or absence of errors or abnormalities)

[0866] Specific operation: Detecting anomalies using deep learning models and generating analysis results

[0867] Step 3:

[0868] The server generates a work log based on the analysis results. The work log records the timestamp of the anomaly or error, the product information in question, and other details. This work log is then generated as a non-fungible token (NFT) that includes further related information.

[0869] Input: Analysis results

[0870] Output: Work log (detailed in chronological order) and NFT

[0871] Specific operations: generating work logs, creating NFTs

[0872] Step 4:

[0873] The server registers the generated NFT on the blockchain, which makes it impossible to tamper with work logs and product information, ensuring authenticity. When the product is shipped, ownership of the corresponding NFT is transferred to the consumer.

[0874] Input: NFT

[0875] Output: NFT registered on the blockchain

[0876] Specific operations: NFT blockchain registration, ownership management

[0877] Step 5:

[0878] Terminals in the factory display the video data and analysis results sent from the server in real time. When an abnormality is detected, the terminal displays a notification on the screen and sends feedback to workers and managers, allowing workers to quickly respond to the problem.

[0879] Input: Video data and analysis results from the server

[0880] Output: Real-time video display, abnormality notification display

[0881] Specific operations: Display of video data, real-time display of abnormality notifications

[0882] Step 6:

[0883] When users purchase a product, they receive a corresponding NFT, which records the product's detailed manufacturing process, country of origin, and quality inspection results, and users can access this information on their smartphones.

[0884] Input: NFT

[0885] Output: User-visible product information

[0886] Specific operation: Displaying NFT and checking product information through a smartphone app

[0887] These steps will ensure efficient security and quality control at the factory and enable reliable information to be provided to consumers.

[0888] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0889] This invention combines a system that utilizes video data acquired from a monitoring device to visualize and improve the reliability of food manufacturing processes with an emotion engine that recognizes user emotions. Below, we will explain the processing of the program, which is centered on the server, terminal, and user.

[0890] Server Processing

[0891] The server acquires video data in real time from the monitoring devices installed in the factory. Each monitoring camera transmits the video data to the server via the network. The received video data is first temporarily stored in the server's storage.

[0892] The server performs preprocessing on the stored video data. After preprocessing, such as noise removal and frame division, the data is input into a generative model. The generative model analyzes the video data using artificial intelligence techniques such as deep learning, monitors the movements of workers and the status of products in real time, and detects errors and abnormalities.

[0893] Based on the results of the analysis by the generative model, the server generates a work log, which includes details of which operations were performed in which process, as well as any detected errors or anomalies. This work log is recorded in chronological order.

[0894] The server then generates a non-fungible token (NFT) based on the generated work log. The NFT contains information about the product's origin, manufacturing process, and quality inspection results. The server registers the generated NFT on the blockchain and transfers ownership of the corresponding NFT to the consumer when the product is shipped.

[0895] Furthermore, the system incorporates an emotion engine that recognizes users' emotions. The server uses the emotion engine to analyze the emotional data of users who receive NFTs and evaluate their satisfaction.

[0896] Terminal handling

[0897] Terminals in the factory receive real-time video data sent from the server, which is then displayed on a screen, allowing workers and managers to monitor the status of the manufacturing site in real time.

[0898] The terminal receives notifications of errors and abnormalities sent from the server and displays them in real time. The details of the error are notified to the worker or manager using sound or light, allowing the problem to be identified and dealt with quickly.

[0899] User Action

[0900] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[0901] Users can access the blockchain using a smartphone app or PC and view the recorded NFT. The NFT contains detailed information about the product's manufacturing process, country of origin, quality inspection results, etc., allowing users to verify the authenticity of the product.

[0902] For example, if a user purchases a high-quality melon, they receive an NFT. The emotion engine analyzes the emotional data when the user views the NFT through a smartphone app. If the user's satisfaction is rated high, additional product information or special offers can be provided based on that information. Conversely, if the user's satisfaction is low, feedback is collected and areas for improvement are identified.

[0903] This system will enable food manufacturers to provide consumers with reliable information, and its error detection function in the manufacturing process is expected to improve product quality. Furthermore, by incorporating an emotion engine, it will be possible to grasp user satisfaction in real time, leading to improved services.

[0904] The processing flow will be explained below.

[0905] Server Processing

[0906] Step 1:

[0907] The server acquires video data in real time from monitoring devices installed in the factory. Each camera transmits the video data to the server via the network, and the server temporarily stores it in storage.

[0908] Step 2:

[0909] The server performs preprocessing on the stored video data, such as noise removal, frame division, and resolution adjustment, to prepare the data in a format suitable for the generative model.

[0910] Step 3:

[0911] The server then inputs the pre-processed video data into a generative model, which then uses deep learning and other artificial intelligence techniques to analyze the data and monitor the manufacturing process in real time.

[0912] Step 4:

[0913] Based on the results of the analysis by the generative model, the server generates a work log that records the details of each work step, including operation details, time information, errors and abnormalities, etc.

[0914] Step 5:

[0915] The server generates a non-fungible token (NFT) based on the generated work log, which includes information such as the product's origin and quality inspection results, in addition to the work log.

[0916] Step 6:

[0917] The server registers the generated NFT on the blockchain, where it is recorded and stored in an immutable form.

[0918] Step 7:

[0919] The server transfers ownership of the corresponding NFT to the consumer when the product is shipped, and this ownership transfer information is also recorded on the blockchain.

[0920] Step 8:

[0921] The server launches an emotion engine that analyzes the emotional data of users who receive NFTs, receives emotional data from the user's smartphone app, and evaluates their level of satisfaction.

[0922] Terminal handling

[0923] Step 1:

[0924] The terminal receives real-time video data sent from the server and displays it on a screen, allowing workers and managers to monitor the status of the manufacturing site in real time.

[0925] Step 2:

[0926] The terminal receives notifications of errors and abnormalities sent from the server, and the notification contents are displayed on the screen in real time.

[0927] Step 3:

[0928] The terminal notifies the operator and manager of any errors with sound and light, allowing them to quickly identify and address the problem.

[0929] User Action

[0930] Step 1:

[0931] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[0932] Step 2:

[0933] Users can access the blockchain using a smartphone app or PC and view the recorded NFT, which contains detailed information about the product's manufacturing process, country of origin, and quality inspection results, allowing users to verify the product's authenticity.

[0934] Step 3:

[0935] When a user browses an NFT, the emotion engine collects emotional data from the user, for example, by analyzing facial expressions and voice using a smartphone's camera and sensors.

[0936] Step 4:

[0937] The emotion engine analyzes the collected emotion data and evaluates the user's satisfaction. The evaluation results are sent to the server and used to improve user satisfaction.

[0938] For example, when a user purchases a high-quality melon and views the corresponding NFT, the emotion engine analyzes the emotional data. If the user's satisfaction is rated high, additional product information and special offers will be provided. If the user's satisfaction is low, feedback will be collected and used to improve the service next time.

[0939] Example 2

[0940] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0941] While conventional food manufacturing process monitoring systems have the ability to visualize the manufacturing process and detect errors, they lack the means to provide information to increase product reliability or to grasp consumer satisfaction in real time. Furthermore, consumers today are increasingly demanding transparency and reliability in products, and want detailed information about the product's manufacturing process and quality. To solve these issues, a new system is needed that combines real-time monitoring data analysis with detailed information provided by non-fungible tokens (NFTs), and also incorporates an emotion engine to evaluate consumer satisfaction.

[0942] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0943] In this invention, the server includes generative model means for analyzing video data acquired from a monitoring device, means for generating a work log based on the results of analysis by the generative model means, means for generating the work log as a non-fungible token, means for providing the non-fungible token to a user, and emotion engine means for analyzing user emotion data. This enables visualization of the manufacturing process and real-time detection of errors, and allows detailed information on product reliability to be provided to consumers, as well as real-time evaluation of consumer satisfaction.

[0944] A "monitoring device" is a device that captures images of the manufacturing process in a factory in real time and acquires video data.

[0945] The "generative model means" is a means for analyzing video data acquired from a surveillance device using deep learning and artificial intelligence techniques.

[0946] The "work log" is a log that records details of operations and abnormality detection in the manufacturing process, and is generated based on the results of analysis by the generative model means.

[0947] A non-fungible token (NFT) is a token that records detailed product information and has uniqueness as a digital asset.

[0948] The "emotion engine means" is a means for analyzing the user's emotion data and evaluating the degree of satisfaction.

[0949] The "function to detect errors and abnormalities" is a function for detecting defects and abnormal operations in the manufacturing process in real time.

[0950] "Product origin information" is detailed information about the place of origin of a product.

[0951] "Manufacturing process information" is detailed information about each step in the manufacturing process of a product.

[0952] "User" refers to a consumer who purchases a product and receives non-fungible tokens.

[0953] This invention combines a system that utilizes video data acquired from a monitoring device to visualize food manufacturing processes and improve reliability, with an emotion engine that recognizes user emotions. Specific embodiments of the system, which is centered around a server, terminals, and users, are described below.

[0954] Server Processing

[0955] The server acquires video data in real time from monitoring devices installed in the factory. Specifically, the monitoring cameras use RTSP (Real-Time Streaming Protocol) to stream video data to the server. The received video data is first temporarily stored in cloud storage such as Amazon S3.

[0956] The server preprocesses the stored video data using Python's OpenCV library. This preprocessing includes noise removal and frame segmentation, which improves the accuracy of the analysis. The preprocessed video data is then input into a generative model. The generative model is built using frameworks such as TensorFlow and PyTorch and uses deep learning to analyze the video data. The analysis results include worker movements and product status, and also detect errors and abnormalities in the manufacturing process.

[0957] Based on the results of the analysis by the generative model, the server generates and records a work log in a NoSQL database such as MongoDB. The work log includes details of operations performed at each stage and any abnormalities.

[0958] The server then generates a non-fungible token (NFT) based on the transaction log and registers it on the Ethereum blockchain using Python's Web3 library. The NFT contains information about the product's origin, manufacturing process, and quality inspection results, which can be verified by consumers.

[0959] The emotion engine analyzes the emotional data when a user receives an NFT. It uses an NLP library for emotion analysis and evaluates the user's satisfaction. This allows us to understand how users feel about the product information in real time.

[0960] Terminal handling

[0961] Terminals in the factory receive real-time video data sent from the server. The video data is received via WebSocket or similar and displayed on the terminal's display. Workers and managers can monitor the video using software such as VLC media player.

[0962] The terminal receives notifications of errors and abnormalities sent from the server and notifies workers and managers in real time with sound and light, enabling them to quickly identify and address problems.

[0963] User Action

[0964] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[0965] Users can access a blockchain viewer such as EtherScan to view the recorded NFT, which contains detailed information about the product's manufacturing process, origin, and quality inspection results, allowing users to verify the product's authenticity.

[0966] For example, if a user purchases a luxury melon, they can view the NFT and its detailed information on their smartphone app. The emotion engine then analyzes the user's emotional data and evaluates their level of satisfaction. If satisfaction is high, additional product information and special offers are provided. Conversely, if satisfaction is low, feedback is collected and areas for improvement are identified.

[0967] Examples of prompt statements

[0968] Here are some example prompts to input to a generative AI model:

[0969] "Design a program to analyze video data acquired from factory monitoring equipment, visualize the manufacturing process, and detect errors."

[0970] This system will enable food manufacturers to provide consumers with reliable information, and the error detection function in the manufacturing process is expected to improve product quality. In addition, by incorporating an emotion engine, it will be possible to grasp user satisfaction in real time, leading to improved services.

[0971] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0972] Step 1: Acquire video data

[0973] The server acquires video data in real time from the monitoring devices installed in the factory. Each monitoring camera transmits the video data to the server via streaming using RTSP (Real-Time Streaming Protocol).

[0974] Input: Real-time video data sent from a surveillance device

[0975] Output: Raw data stored on the server

[0976] Specific operation: Save video data to cloud storage such as Amazon S3.

[0977] Step 2: Preprocessing the video data

[0978] The server performs preprocessing on the stored video data, using the Python OpenCV library to remove noise and split the video data into frames.

[0979] Input: Raw data stored on the server

[0980] Output: Pre-processed video data

[0981] Specific operation: Apply an average filter to reduce noise and extract a specific frame.

[0982] Step 3: Analysis with a generative model

[0983] The pre-processed video data is then fed into a generative model, which is built using the TensorFlow and PyTorch frameworks to analyze the data and detect errors and anomalies in the manufacturing process.

[0984] Input: Preprocessed video data

[0985] Output: Analysis results for each process and abnormality detection information

[0986] Specific operation: Data is input into the generative AI model, which analyzes the worker's actions and the product's condition. If an abnormality is detected, an alert is generated.

[0987] Step 4: Generate a work log

[0988] Based on the analysis results, the server generates a work log, using a NoSQL database such as MongoDB to record information about each process and any anomalies in chronological order.

[0989] Input: Analysis results from the generative model

[0990] Output: Work log

[0991] Specific operation: A text log is generated based on the analysis results and saved in a database.

[0992] Step 5: Generate and register your NFT

[0993] The server generates a non-fungible token (NFT) based on the generated work log and registers the NFT on the Ethereum blockchain using Python's Web3 library.

[0994] Input: Work Log

[0995] Output: NFT registered on the blockchain

[0996] Specific operation: Generates an NFT containing information on the product's origin, manufacturing process, and quality inspection results, and manages it using a smart contract.

[0997] Step 6: Receive real-time video and error reports

[0998] The terminal receives real-time video data and error notifications sent from the server, allowing workers and managers to monitor the status of the manufacturing site in real time.

[0999] Input: Video data and error notifications from the server

[1000] Output: Real-time video and notifications displayed on the display

[1001] Specific operation: Receives video data via WebSocket and displays it in VLC media player. If an abnormality is detected, it notifies you with sound and light.

[1002] Step 7: Receive and view your NFT

[1003] Users receive the corresponding NFT when they purchase a product, transfer ownership of the NFT through a smartphone app or online platform, and then access a blockchain viewer such as EtherScan to view the recorded NFT.

[1004] Input: NFT data from the server

[1005] Output: NFT information viewed on the user's smartphone or PC

[1006] How it works: Use an app like MetaMask to receive your NFT and then use EtherScan to view detailed manufacturing and origin information.

[1007] Step 8: Analyze the sentiment data

[1008] The server analyzes the emotional data when users receive NFTs, and uses an NLP library to analyze user feedback and comments to evaluate user satisfaction.

[1009] Input: User feedback and comments

[1010] Output: User satisfaction evaluation results

[1011] How it works: Feedback data is fed into an NLP library, and satisfaction is scored based on sentiment analysis. Additional product information and special offers are provided based on the evaluation results.

[1012] (Application example 2)

[1013] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1014] In conventional manufacturing processes, there are systems for product quality control and improving the reliability of data related to the manufacturing process, but there are limited means of providing reliable information directly to consumers. Furthermore, it is difficult to grasp consumer satisfaction in real time, making it difficult to quickly implement specific countermeasures. This has prevented companies from properly understanding consumer needs and emotions and improving the quality of their services.

[1015] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1016] In this invention, the server includes generative model means for analyzing video data acquired from a monitoring device, means for generating a work log based on the results of analysis by the generative model means, means for generating the work log as a non-fungible token, means for providing the non-fungible token to a user, emotion analysis means for recognizing user emotions, and means for evaluating user satisfaction based on emotion data analyzed by the emotion analysis means. This makes it possible to provide consumers with reliable product information, grasp consumer purchasing behavior and satisfaction in real time, and improve service quality.

[1017] A "monitoring device" is a device used to monitor a specific area, such as a factory or a brick-and-mortar store, in real time and acquire video data.

[1018] "Video data" refers to real-time visual information acquired from a monitoring device, and is the data to be analyzed.

[1019] A "generative model means" is a model incorporating deep learning and artificial intelligence techniques that is used to analyze captured video data and extract specific information.

[1020] The "work log" is data that records which work was performed in which process based on the results of analysis by the generative model means.

[1021] A "non-fungible token" is a unique digital asset created based on blockchain technology that contains information about the product's origin and manufacturing process.

[1022] "User" refers to a consumer or customer who uses the system and receives information about products and services.

[1023] "Emotion analysis means" is a technology for analyzing a user's facial expressions and behavior to determine their emotional state at that time.

[1024] "Satisfaction" refers to the degree of satisfaction a user feels with a product or service, and is evaluated by sentiment analysis means.

[1025] The present invention provides a system that combines a monitoring device, a generative model, work log generation, non-fungible token generation and provision, emotion analysis means, and satisfaction assessment means.

[1026] Server Processing

[1027] The server acquires video data in real time from monitoring devices installed in factories and brick-and-mortar stores. Each monitoring camera transmits video data to the server via the network. The received video data is first temporarily stored in the server's storage.

[1028] The server performs preprocessing on the stored video data, including noise removal and frame segmentation. The preprocessed data is then input into a generative model (e.g., TensorFlow, Keras). This generative model uses deep learning technology to analyze the video data, monitor the actions of workers and the status of products in real time, and detect errors and anomalies.

[1029] Based on the analysis results, the server generates a work log, which includes details of what operations were performed at which stage and any errors or anomalies detected. This work log is recorded in chronological order. A non-fungible token (NFT) is then generated based on the generated work log. The NFT includes information about the product's origin, manufacturing process, quality inspection results, etc. The server registers the generated NFT on a blockchain (e.g., Ethereum, Hyperledger Fabric), and transfers ownership of the corresponding NFT to the consumer when the product is shipped.

[1030] Furthermore, this system incorporates an emotion analysis mechanism to recognize users' emotions. The server uses an emotion analysis engine (e.g., Affectiva, Microsoft Emotion API) to analyze the emotional data of users who receive NFTs and evaluate their satisfaction.

[1031] Terminal handling

[1032] Terminals in factories or physical stores receive real-time video data sent from the server. The received video data is displayed on a screen, allowing workers and managers to monitor the status of the manufacturing site or physical store in real time. The terminals receive notifications of errors and abnormality detection sent from the server and display them in real time. The details of the error are notified to workers and managers with sound or light, allowing them to quickly identify and address the problem.

[1033] User Action

[1034] When a user purchases a product, they receive a corresponding NFT. Purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user. Users can access the blockchain using a smartphone app or PC to view the recorded NFT. The NFT contains detailed information about the product's manufacturing process, country of origin, and quality inspection results, allowing users to verify the product's authenticity.

[1035] For example, if a user purchases a smartphone, they can receive a corresponding NFT. The sentiment analysis engine analyzes emotional data when the user browses the NFT through a smartphone app. For example, if the user expresses excitement or satisfaction while browsing, that information can be used to provide additional product information or special offers. Conversely, if the user expresses dissatisfaction, feedback can be collected and areas for improvement identified.

[1036] Example prompt for a generative AI model:

[1037] / / Customer sentiment analysis prompt

[1038] Input data: Video data captured by a camera

[1039] Goal: Identify customer emotions (happiness, anger, confusion, excitement, etc.) from video data and display the results in real time.

[1040] This system will enable manufacturers and brick-and-mortar store operators to provide consumers with reliable information, and is expected to improve product quality through error detection functions in the manufacturing process and sales environment.Furthermore, by using sentiment analysis tools, it will be possible to grasp user satisfaction in real time, leading to improved services.

[1041] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1042] Step 1:

[1043] The server acquires video data in real time from monitoring devices installed in factories and brick-and-mortar stores. Each monitoring camera sends video data to the server via the network, and this video data is temporarily stored in the server's storage. The input is video data from the monitoring device, and the output is the temporarily stored video data. Specifically, the server checks the connection of the monitoring camera and transfers the video data to the server in a stream format.

[1044] Step 2:

[1045] The server performs preprocessing on the stored video data, such as noise removal and frame division. This preprocessing improves data quality and increases analysis accuracy. The input is temporarily stored video data, and the output is preprocessed video data. Specifically, OpenCV is used to perform noise filtering and divide continuous video into individual frames.

[1046] Step 3:

[1047] The server inputs the preprocessed video data into a generative model to analyze the worker's actions and the product's status. Generative AI models (e.g., TensorFlow, Keras) are used to detect errors and anomalies in real time. The input is the preprocessed video data, and the output is the error and anomaly detection results. Specifically, features are extracted for each frame, and abnormal actions are identified using a deep learning model.

[1048] Step 4:

[1049] The server generates a work log based on the analysis results obtained from the generative model and records it in chronological order. The input is the error and anomaly detection results, and the output is the work log. Specifically, the analysis results are recorded in a log file in text format, and a timestamp is added to each log.

[1050] Step 5:

[1051] The server generates a non-fungible token (NFT) based on the generated work log and registers it on the blockchain. The input is the work log, and the output is the NFT registered on the blockchain. Specifically, the work log is converted to JSON format and registered on the blockchain via a smart contract.

[1052] Step 6:

[1053] The terminal receives real-time video data and error notifications sent from the server and displays them on the display. The input is video data and error notifications from the server, and the output is displayed on the display. Specifically, the video data is streamed, and if an error occurs, the worker is notified with sound and light.

[1054] Step 7:

[1055] When a user purchases a product, they receive the corresponding NFT and view it on a smartphone app or PC. The input is purchase information, and the output is the display of the NFT on the user's smartphone or PC. Specifically, the user enters their purchase information and transfers NFT ownership on the blockchain.

[1056] Step 8:

[1057] The server uses an emotion analysis engine to analyze the user's emotional data when viewing NFTs and evaluate their satisfaction. The input is the user's facial expression data, and the output is the emotion analysis results and satisfaction rating. Specifically, it uses Affectiva and the Microsoft Emotion API to analyze emotions from the user's facial expressions and provides feedback to the user along with the results.

[1058] Example prompt sentence:

[1059] / / Customer sentiment analysis prompt

[1060] Input data: Video data captured by a camera

[1061] Goal: Identify customer emotions (happiness, anger, confusion, excitement, etc.) from video data and display the results in real time.

[1062] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1063] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1064] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1065] [Fourth embodiment]

[1066] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1067] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1068] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1069] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1070] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1071] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1072] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1073] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1074] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1075] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1076] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1077] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1078] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1079] This invention is a system that utilizes video data acquired from a monitoring device to visualize food manufacturing processes and improve reliability. Below, we will explain the processing of a program that is centered around a server, terminals, and users.

[1080] Server Processing

[1081] The server acquires video data in real time from monitoring devices installed in the factory. The video data sent from each camera is temporarily stored in the server's storage. This video data is pre-processed and subjected to necessary processing such as noise removal before being input into the generative model.

[1082] The generative model analyzes video data using deep learning and other artificial intelligence techniques. Specifically, it monitors worker movements and product status in real time during the manufacturing process to detect errors and anomalies. Based on the analysis results, the server generates a work log, recording the details of the work process in chronological order.

[1083] The generated work logs are then compiled and generated as a non-fungible token (NFT), which includes information on the product's origin, manufacturing process, quality inspection results, etc. Finally, the server registers the NFT on the blockchain, and ownership is transferred to the consumer when the product is shipped.

[1084] Terminal handling

[1085] Terminals in the factory display the video data sent from the server in real time. Workers and managers can use the terminals to check whether their work is being carried out correctly. Notifications of errors and abnormalities sent from the server are also displayed in real time, providing immediate feedback to workers and managers.

[1086] For example, if a worker accidentally performs an inappropriate operation on a product, the generative model will detect the anomaly and display an error notification on the terminal, allowing the manager to quickly resolve the problem and maintain quality.

[1087] User Action

[1088] When users purchase a product, they receive a corresponding NFT, which records the product's detailed manufacturing process, origin information, quality inspection results, etc., allowing users to verify the product's authenticity.

[1089] For example, if a user purchases a high-quality melon, they can view the corresponding NFT through a smartphone app. By viewing the NFT, the user can obtain a series of information about the farm where the melon was grown, harvested, and shipped. This ensures the authenticity of the product, allowing users to make a purchase with confidence.

[1090] This system will enable food manufacturers to provide consumers with reliable information, and its error detection function in the manufacturing process is expected to improve product quality.

[1091] The processing flow will be explained below.

[1092] Server Processing

[1093] Step 1:

[1094] The server acquires video data in real time from the monitoring devices installed in the factory. Each monitoring camera transmits the video data to the server via the network. The received video data is first temporarily stored in the server's storage.

[1095] Step 2:

[1096] The server pre-processes the stored video data, performing noise removal and frame division to optimize the data for analysis. This pre-processed data is then input into the generative model.

[1097] Step 3:

[1098] The server analyzes the pre-processed video data using a generative model (a model that uses artificial intelligence techniques such as deep learning). The generative model monitors the movements of workers and the status of products in real time to detect errors and anomalies.

[1099] Step 4:

[1100] The server generates a work log based on the results of the analysis by the generative model. The work log includes information such as what operations were performed in which process, and details of any detected errors or anomalies. This work log is recorded in chronological order.

[1101] Step 5:

[1102] The server generates a non-fungible token (NFT) based on the generated work log. The NFT is configured to include information on the product's origin, manufacturing process, quality inspection results, etc.

[1103] Step 6:

[1104] The server registers the generated NFT on the blockchain. The registered NFT is irreversibly stored on the blockchain, and the risk of tampering is low.

[1105] Step 7:

[1106] The server transfers ownership of the corresponding NFT to the consumer when the product is shipped, and this ownership transfer information is also recorded on the blockchain.

[1107] Terminal handling

[1108] Step 1:

[1109] The terminal receives real-time video data sent from the server, and the received video data is displayed on a screen, allowing workers or managers to monitor the status of the manufacturing site in real time.

[1110] Step 2:

[1111] The terminal receives notifications of errors and abnormality detection sent from the server. If an error is detected, the details are displayed on the terminal display in real time.

[1112] Step 3:

[1113] The terminal notifies the operator and manager of the error using sound and light, allowing them to quickly identify and address the problem.

[1114] User Action

[1115] Step 1:

[1116] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[1117] Step 2:

[1118] Users can access the blockchain using a smartphone app or PC, view the NFTs recorded on the blockchain, and check the details of the NFTs they own.

[1119] Step 3:

[1120] Users can view product information written on the NFT, including detailed information such as the product's place of origin, manufacturing process, and quality inspection results. This allows users to confirm the authenticity of the product and consume it with confidence.

[1121] Example 1

[1122] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1123] In conventional food manufacturing processes, there is a lack of visualization of the product manufacturing process and an insufficient improvement in reliability. This makes it difficult to guarantee product quality and gain consumer trust. In addition, detecting errors and abnormalities in the manufacturing process is often done manually, requiring quick and accurate response.

[1124] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1125] In this invention, the server includes means for preprocessing video data acquired from a monitoring device, deep learning model means for analyzing the video data, means for generating a work log based on the analysis results, means for generating the work log as a non-fungible token, and means for providing the non-fungible token to a user. This enables visualization and improvement of reliability of the product manufacturing process, as well as rapid and accurate detection of errors and abnormalities in the manufacturing process.

[1126] "Monitoring equipment" refers to equipment used to monitor and record manufacturing processes in a factory in real time.

[1127] "Video data" refers to visual information acquired by a monitoring device expressed in digital form.

[1128] "Preprocessing" refers to processing of acquired video data, such as noise removal and brightness adjustment.

[1129] A "deep learning model" is an algorithm that uses artificial intelligence technology to analyze large amounts of data and perform pattern recognition and prediction.

[1130] A "work log" is data that records details such as work content and abnormality detection results in the manufacturing process.

[1131] A "non-fungible token" is a digital certificate that uniquely records detailed information about a product or digital asset based on blockchain technology.

[1132] "User" means a person who uses the information and services provided through this system.

[1133] "Error or Anomaly" refers to any condition or behavior in the manufacturing process that deviates from standard procedures or quality standards.

[1134] This invention is a system that utilizes video data acquired from a monitoring device to visualize food manufacturing processes and improve reliability. Below, we will explain in detail the processing of a program that is centered around a server, terminals, and users.

[1135] Server Processing

[1136] The server acquires video data in real time from monitoring devices installed in the factory. The video data sent from each camera is temporarily stored in the server's storage. This video data is pre-processed. The pre-processing process includes noise removal and brightness adjustment for each frame, and image processing filters are applied using the OpenCV library.

[1137] After preprocessing, the video data is input into a generative model using deep learning libraries such as TensorFlow and PyTorch. This generative model analyzes the video data frame by frame to monitor the movements of workers and the status of products in the manufacturing process. The generative model detects product defects and inconsistencies in worker behavior in real time.

[1138] Based on the model's analysis results, the server generates an operation log, which records details such as the time the anomaly was detected, the nature of the anomaly, and countermeasures. The operation log is created in JSON format and stored in a database. The server then compiles the operation logs and generates a single non-fungible token (NFT). The NFT contains information about the product's origin, manufacturing process, and quality inspection results. Specifically, a smart contract is created using a blockchain platform such as Ethereum, and the NFT is registered.

[1139] Terminal handling

[1140] Terminals in the factory display the video data sent from the server in real time. The software used is VLC Media Player or other streaming playback software. Workers and managers can use the terminals to check whether their work is being carried out correctly. Notifications of errors and abnormalities sent from the server are also displayed in real time, providing immediate feedback to workers and managers.

[1141] For example, if a worker accidentally performs an inappropriate operation on a product, the generative model will detect the anomaly and display an error notification on the terminal, allowing the manager to quickly resolve the problem and maintain product quality.

[1142] User Action

[1143] After purchasing a product, users will receive a corresponding NFT, which records the product's detailed manufacturing process, country of origin, and quality inspection results. Users can check the NFT's detailed information by scanning the QR code with a smartphone app.

[1144] For example, if a user purchases a high-quality melon, they can view the corresponding NFT in a smartphone app. By viewing the NFT, the user can obtain a series of information about the farm where the melon was grown, how it passed quality inspections, and how it was shipped.

[1145] Examples and prompts

[1146] A specific example is a process that collects video data from surveillance cameras, uses AI to detect anomalies, records the results in a JSON-formatted log, and generates it as an NFT.

[1147] When processing the device, the video is displayed in real time using VLC media player, and if an abnormality is detected, a pop-up alert is displayed.

[1148] The user process involves scanning the NFT of the purchased product with the app to check the manufacturing process and quality inspection results.

[1149] An example prompt might look like this:

[1150] "I want to check the manufacturing process of the melon I purchased."

[1151] "Show me the quality inspection results recorded on the NFT."

[1152] "Display surveillance camera video data in real time."

[1153] This system will enable food manufacturers to provide consumers with reliable information, and its error detection function in the manufacturing process is expected to improve product quality.

[1154] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1155] Server Processing

[1156] Step 1:

[1157] The server acquires video data in real time from surveillance cameras installed in the factory. Specifically, it receives streaming data generated by IP cameras and high-resolution cameras and temporarily stores it in the server's storage. Input: Video streaming data from surveillance cameras. Output: Raw video data stored in the server's storage.

[1158] Step 2:

[1159] The server preprocesses the acquired video data. During this process, noise is removed and brightness is adjusted for each frame. Specifically, image processing filters are applied using the OpenCV library. Input: Raw video data stored in the server's storage. Output: Preprocessed, clean video data.

[1160] Step 3:

[1161] After preprocessing, the video data is input into a generative model. The generative model uses deep learning libraries such as TensorFlow and PyTorch. The model analyzes the video data frame by frame and monitors the movements of workers and the state of products in real time. Input: Preprocessed, clean video data. Output: Analysis results from the generative model.

[1162] Step 4:

[1163] The generative model analyzes the data and detects errors and anomalies. Specifically, it detects product defects and inconsistencies in worker behavior as anomalies. In this process, pre-set thresholds and pattern recognition algorithms are used. Input: Analysis results by the generative model. Output: Detected data for errors and anomalies.

[1164] Step 5:

[1165] The server generates a work log based on the error or anomaly detection data. The work log records details such as the time the anomaly was detected, the nature of the anomaly, and countermeasures. Specifically, it creates log data in JSON format and saves it in a database. Input: Error or anomaly detection data. Output: JSON format work log.

[1166] Step 6:

[1167] The server compiles the work logs and generates an NFT. The NFT contains information about the product's origin, manufacturing process, and quality inspection results. Specifically, a smart contract is created using a blockchain platform such as Ethereum to register the NFT. Input: Work logs in JSON format. Output: NFT recorded on the blockchain.

[1168] Terminal handling

[1169] Step 1:

[1170] The terminals in the factory display the video data sent from the server in real time. Specifically, VLC Media Player or other streaming playback software is used. Input: Real-time video data sent from the server. Output: Video displayed on the terminal.

[1171] Step 2:

[1172] The terminal displays error and abnormality notifications sent from the server in real time. Specifically, when an abnormality is detected, a pop-up window is displayed and an alert is sent to the worker or administrator. Input: Error notification sent from the server. Output: Error pop-up displayed on the terminal.

[1173] Step 3:

[1174] Workers and managers check the notifications on their devices and take the necessary action quickly. For example, if an abnormality is displayed, the work is stopped immediately and the problem is corrected. Input: Error pop-up displayed on the device. Output: Workers and managers quickly correct the problem.

[1175] User Action

[1176] Step 1:

[1177] After purchasing a product, users receive the corresponding NFT. Specifically, they can check detailed information about the NFT by scanning the QR code with a smartphone app. Input: QR code provided with the product. Output: Detailed information about the NFT displayed on the smartphone app.

[1178] Step 2:

[1179] Through the NFT, users can check the product's detailed manufacturing process, country of origin, and quality inspection results. For example, if they purchase a high-quality melon, they can find out which farm it was grown on and what quality inspections it passed. Input: NFT information read by a smartphone app. Output: Displayed product manufacturing process and quality inspection results.

[1180] Step 3:

[1181] Users will recognize the reliability of the product through NFT and use it with peace of mind. Specifically, they will have access to the product's manufacturing process and inspection results, and will feel secure by obtaining reliable information. Input: Displayed product manufacturing process and quality inspection results. Output: Recognition of product reliability and a sense of security.

[1182] (Application example 1)

[1183] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1184] In modern factories and production lines, improving the efficiency of product quality control and security management is a major challenge. However, existing systems make it difficult to detect anomalies in real time and take immediate action. Furthermore, to increase product reliability, detailed records of the manufacturing process and the provision of that information to consumers are necessary, but systems for doing so are often not in place. As such, integrating security and quality control within factories remains a major challenge.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1186] In this invention, the server includes generative model means for analyzing video data acquired from a monitoring device, means for generating a work log based on the results of analysis by the generative model means, means for generating the work log as a non-fungible token, means for providing the non-fungible token to a user, notification means for acquiring video data from the monitoring device in real time and detecting errors or abnormalities, and means for displaying a notification of abnormality detection by the notification means in real time. This not only makes it possible to detect abnormalities in real time and take immediate action, but also makes it possible to provide consumers with detailed information to increase product reliability.

[1187] A "monitoring device" is a device that is installed in a factory and acquires video data in real time.

[1188] A "generative model" is a system that uses deep learning and artificial intelligence techniques to analyze video data and detect errors and anomalies.

[1189] A "work log" is data that records the details of the manufacturing process in chronological order based on the results of analysis by the generative model.

[1190] A "non-fungible token" is digital data that uses blockchain technology and cannot be altered or copied once issued.

[1191] "Users" refer to consumers who purchase the factory's products and receive non-fungible tokens, as well as factory managers.

[1192] "Notification means" is a mechanism that presents information in real time when an abnormality or error is detected.

[1193] The "real-time display means" is a device that instantly displays information from the notification means on a terminal and quickly provides information to the relevant parties.

[1194] This invention is a system that utilizes video data acquired from monitoring devices in factories to improve the efficiency of factory security and quality control, and to increase the reliability of products for consumers. The following describes the processing of a program that is centered around a server, terminals, and users.

[1195] Server Action:

[1196] The server first acquires video data in real time from monitoring devices installed in the factory. The acquired video data is temporarily stored in the server's storage. After undergoing preprocessing such as noise removal, the video data is input into a generative model such as a deep learning model. The generative model analyzes the video data using deep learning and other artificial intelligence technologies, monitoring the movements of workers and the status of products in real time during the manufacturing process and detecting abnormalities and errors. Based on the results of this analysis, the server generates a work log, recording details of the work process in chronological order. The generated work log is generated as an NFT (non-fungible token), which includes information such as the product's place of origin, manufacturing process, and quality inspection results. Finally, the server registers this NFT on the blockchain, and ownership is transferred to the consumer when the product is shipped.

[1197] Terminal handling:

[1198] Video data sent from the server can be displayed in real time on terminals within the factory. Workers and managers can use the terminals to check whether their work is being carried out correctly, and notifications of errors and abnormalities detected from the server are also displayed in real time. For example, if a worker mistakenly performs an inappropriate operation on a product, the generative model will detect the abnormality and an error notification will immediately appear on the terminal. As a result, managers can quickly resolve the problem and maintain product quality.

[1199] User Action:

[1200] When users purchase a product, they receive a corresponding NFT. The NFT contains detailed information about the product's manufacturing process, country of origin, and quality inspection results, allowing users to verify the product's authenticity. For example, when a user purchases a luxury food ingredient, they can view the corresponding NFT through a smartphone app. By viewing the NFT, users can obtain a series of information about how the product was manufactured and how quality inspections were carried out. This ensures the authenticity of the product, allowing users to purchase with confidence.

[1201] Prompt Sentence Examples

[1202] Examples of prompts to input to a generative AI model include:

[1203] "We set up IP addresses for cameras in factories and automatically preprocess the video data captured by the cameras. We then input that data into a TensorFlow model to detect anomalies in real time. If an anomaly is detected, we send a push notification to the administrator and register the details of the anomaly as an NFT on the blockchain."

[1204] This system not only strengthens factory security and enables immediate detection and response to abnormalities and errors, but also provides consumers with detailed product information and increases their reliability.

[1205] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1206] Step 1:

[1207] The server acquires video data in real time from monitoring devices installed in the factory. The video data sent from the camera is first sent to the server and temporarily saved in storage. This saved data undergoes pre-processing such as noise removal and image sharpening.

[1208] Input: Real-time video data from the camera

[1209] Output: Pre-processed video data

[1210] Specific operations: noise removal, grayscale conversion, edge enhancement

[1211] Step 2:

[1212] The preprocessed video data is input into a generative AI model. The server uses this model to analyze the video data and monitor the movements of workers and the status of products during the manufacturing process. If an error or abnormality is detected, the server proceeds to the next step based on the analysis results.

[1213] Input: Preprocessed video data

[1214] Output: Analysis results (presence or absence of errors or abnormalities)

[1215] Specific operation: Detecting anomalies using deep learning models and generating analysis results

[1216] Step 3:

[1217] The server generates a work log based on the analysis results. The work log records the timestamp of the anomaly or error, the product information in question, and other details. This work log is then generated as a non-fungible token (NFT) that includes further related information.

[1218] Input: Analysis results

[1219] Output: Work log (detailed in chronological order) and NFT

[1220] Specific operations: generating work logs, creating NFTs

[1221] Step 4:

[1222] The server registers the generated NFT on the blockchain, which makes it impossible to tamper with work logs and product information, ensuring authenticity. When the product is shipped, ownership of the corresponding NFT is transferred to the consumer.

[1223] Input: NFT

[1224] Output: NFT registered on the blockchain

[1225] Specific operations: NFT blockchain registration, ownership management

[1226] Step 5:

[1227] Terminals in the factory display the video data and analysis results sent from the server in real time. When an abnormality is detected, the terminal displays a notification on the screen and sends feedback to workers and managers, allowing workers to quickly respond to the problem.

[1228] Input: Video data and analysis results from the server

[1229] Output: Real-time video display, abnormality notification display

[1230] Specific operations: Display of video data, real-time display of abnormality notifications

[1231] Step 6:

[1232] When users purchase a product, they receive a corresponding NFT, which records the product's detailed manufacturing process, country of origin, and quality inspection results, and users can access this information on their smartphones.

[1233] Input: NFT

[1234] Output: User-visible product information

[1235] Specific operation: Displaying NFT and checking product information through a smartphone app

[1236] These steps will ensure efficient security and quality control at the factory and enable reliable information to be provided to consumers.

[1237] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1238] This invention combines a system that utilizes video data acquired from a monitoring device to visualize and improve the reliability of food manufacturing processes with an emotion engine that recognizes user emotions. Below, we will explain the processing of the program, which is centered on the server, terminal, and user.

[1239] Server Processing

[1240] The server acquires video data in real time from the monitoring devices installed in the factory. Each monitoring camera transmits the video data to the server via the network. The received video data is first temporarily stored in the server's storage.

[1241] The server performs preprocessing on the stored video data. After preprocessing, such as noise removal and frame division, the data is input into a generative model. The generative model analyzes the video data using artificial intelligence techniques such as deep learning, monitors the movements of workers and the status of products in real time, and detects errors and abnormalities.

[1242] Based on the results of the analysis by the generative model, the server generates a work log, which includes details of which operations were performed in which process, as well as any detected errors or anomalies. This work log is recorded in chronological order.

[1243] The server then generates a non-fungible token (NFT) based on the generated work log. The NFT contains information about the product's origin, manufacturing process, and quality inspection results. The server registers the generated NFT on the blockchain and transfers ownership of the corresponding NFT to the consumer when the product is shipped.

[1244] Furthermore, the system incorporates an emotion engine that recognizes users' emotions. The server uses the emotion engine to analyze the emotional data of users who receive NFTs and evaluate their satisfaction.

[1245] Terminal handling

[1246] Terminals in the factory receive real-time video data sent from the server, which is then displayed on a screen, allowing workers and managers to monitor the status of the manufacturing site in real time.

[1247] The terminal receives notifications of errors and abnormalities sent from the server and displays them in real time. The details of the error are notified to the worker or manager using sound or light, allowing the problem to be identified and dealt with quickly.

[1248] User Action

[1249] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[1250] Users can access the blockchain using a smartphone app or PC and view the recorded NFT. The NFT contains detailed information about the product's manufacturing process, country of origin, quality inspection results, etc., allowing users to verify the authenticity of the product.

[1251] For example, if a user purchases a high-quality melon, they receive an NFT. The emotion engine analyzes the emotional data when the user views the NFT through a smartphone app. If the user's satisfaction is rated high, additional product information or special offers can be provided based on that information. Conversely, if the user's satisfaction is low, feedback is collected and areas for improvement are identified.

[1252] This system will enable food manufacturers to provide consumers with reliable information, and its error detection function in the manufacturing process is expected to improve product quality. Furthermore, by incorporating an emotion engine, it will be possible to grasp user satisfaction in real time, leading to improved services.

[1253] The processing flow will be explained below.

[1254] Server Processing

[1255] Step 1:

[1256] The server acquires video data in real time from monitoring devices installed in the factory. Each camera transmits the video data to the server via the network, and the server temporarily stores it in storage.

[1257] Step 2:

[1258] The server performs preprocessing on the stored video data, such as noise removal, frame division, and resolution adjustment, to prepare the data in a format suitable for the generative model.

[1259] Step 3:

[1260] The server then inputs the pre-processed video data into a generative model, which then uses deep learning and other artificial intelligence techniques to analyze the data and monitor the manufacturing process in real time.

[1261] Step 4:

[1262] Based on the results of the analysis by the generative model, the server generates a work log that records the details of each work step, including operation details, time information, errors and abnormalities, etc.

[1263] Step 5:

[1264] The server generates a non-fungible token (NFT) based on the generated work log, which includes information such as the product's origin and quality inspection results, in addition to the work log.

[1265] Step 6:

[1266] The server registers the generated NFT on the blockchain, where it is recorded and stored in an immutable form.

[1267] Step 7:

[1268] The server transfers ownership of the corresponding NFT to the consumer when the product is shipped, and this ownership transfer information is also recorded on the blockchain.

[1269] Step 8:

[1270] The server launches an emotion engine that analyzes the emotional data of users who receive NFTs, receives emotional data from the user's smartphone app, and evaluates their level of satisfaction.

[1271] Terminal handling

[1272] Step 1:

[1273] The terminal receives real-time video data sent from the server and displays it on a screen, allowing workers and managers to monitor the status of the manufacturing site in real time.

[1274] Step 2:

[1275] The terminal receives notifications of errors and abnormalities sent from the server, and the notification contents are displayed on the screen in real time.

[1276] Step 3:

[1277] The terminal notifies the operator and manager of any errors with sound and light, allowing them to quickly identify and address the problem.

[1278] User Action

[1279] Step 1:

[1280] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[1281] Step 2:

[1282] Users can access the blockchain using a smartphone app or PC and view the recorded NFT, which contains detailed information about the product's manufacturing process, country of origin, and quality inspection results, allowing users to verify the product's authenticity.

[1283] Step 3:

[1284] When a user browses an NFT, the emotion engine collects emotional data from the user, for example, by analyzing facial expressions and voice using a smartphone's camera and sensors.

[1285] Step 4:

[1286] The emotion engine analyzes the collected emotion data and evaluates the user's satisfaction. The evaluation results are sent to the server and used to improve user satisfaction.

[1287] For example, when a user purchases a high-quality melon and views the corresponding NFT, the emotion engine analyzes the emotional data. If the user's satisfaction is rated high, additional product information and special offers will be provided. If the user's satisfaction is low, feedback will be collected and used to improve the service next time.

[1288] Example 2

[1289] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1290] While conventional food manufacturing process monitoring systems have the ability to visualize the manufacturing process and detect errors, they lack the means to provide information to increase product reliability or to grasp consumer satisfaction in real time. Furthermore, consumers today are increasingly demanding transparency and reliability in products, and want detailed information about the product's manufacturing process and quality. To solve these issues, a new system is needed that combines real-time monitoring data analysis with detailed information provided by non-fungible tokens (NFTs), and also incorporates an emotion engine to evaluate consumer satisfaction.

[1291] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1292] In this invention, the server includes generative model means for analyzing video data acquired from a monitoring device, means for generating a work log based on the results of analysis by the generative model means, means for generating the work log as a non-fungible token, means for providing the non-fungible token to a user, and emotion engine means for analyzing user emotion data. This enables visualization of the manufacturing process and real-time detection of errors, and allows detailed information on product reliability to be provided to consumers, as well as real-time evaluation of consumer satisfaction.

[1293] A "monitoring device" is a device that captures images of the manufacturing process in a factory in real time and acquires video data.

[1294] The "generative model means" is a means for analyzing video data acquired from a surveillance device using deep learning and artificial intelligence techniques.

[1295] The "work log" is a log that records details of operations and abnormality detection in the manufacturing process, and is generated based on the results of analysis by the generative model means.

[1296] A non-fungible token (NFT) is a token that records detailed product information and has uniqueness as a digital asset.

[1297] The "emotion engine means" is a means for analyzing the user's emotion data and evaluating the degree of satisfaction.

[1298] The "function to detect errors and abnormalities" is a function for detecting defects and abnormal operations in the manufacturing process in real time.

[1299] "Product origin information" is detailed information about the place of origin of a product.

[1300] "Manufacturing process information" is detailed information about each step in the manufacturing process of a product.

[1301] "User" refers to a consumer who purchases a product and receives non-fungible tokens.

[1302] This invention combines a system that utilizes video data acquired from a monitoring device to visualize food manufacturing processes and improve reliability, with an emotion engine that recognizes user emotions. Specific embodiments of the system, which is centered around a server, terminals, and users, are described below.

[1303] Server Processing

[1304] The server acquires video data in real time from monitoring devices installed in the factory. Specifically, the monitoring cameras use RTSP (Real-Time Streaming Protocol) to stream video data to the server. The received video data is first temporarily stored in cloud storage such as Amazon S3.

[1305] The server preprocesses the stored video data using Python's OpenCV library. This preprocessing includes noise removal and frame segmentation, which improves the accuracy of the analysis. The preprocessed video data is then input into a generative model. The generative model is built using frameworks such as TensorFlow and PyTorch and uses deep learning to analyze the video data. The analysis results include worker movements and product status, and also detect errors and abnormalities in the manufacturing process.

[1306] Based on the results of the analysis by the generative model, the server generates and records a work log in a NoSQL database such as MongoDB. The work log includes details of operations performed at each stage and any abnormalities.

[1307] The server then generates a non-fungible token (NFT) based on the transaction log and registers it on the Ethereum blockchain using Python's Web3 library. The NFT contains information about the product's origin, manufacturing process, and quality inspection results, which can be verified by consumers.

[1308] The emotion engine analyzes the emotional data when a user receives an NFT. It uses an NLP library for emotion analysis and evaluates the user's satisfaction. This allows us to understand how users feel about the product information in real time.

[1309] Terminal handling

[1310] Terminals in the factory receive real-time video data sent from the server. The video data is received via WebSocket or similar and displayed on the terminal's display. Workers and managers can monitor the video using software such as VLC media player.

[1311] The terminal receives notifications of errors and abnormalities sent from the server and notifies workers and managers in real time with sound and light, enabling them to quickly identify and address problems.

[1312] User Action

[1313] When users purchase a product, they receive the corresponding NFT. The purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user.

[1314] Users can access a blockchain viewer such as EtherScan to view the recorded NFT, which contains detailed information about the product's manufacturing process, origin, and quality inspection results, allowing users to verify the product's authenticity.

[1315] For example, if a user purchases a luxury melon, they can view the NFT and its detailed information on their smartphone app. The emotion engine then analyzes the user's emotional data and evaluates their level of satisfaction. If satisfaction is high, additional product information and special offers are provided. Conversely, if satisfaction is low, feedback is collected and areas for improvement are identified.

[1316] Examples of prompt statements

[1317] Here are some example prompts to input to a generative AI model:

[1318] "Design a program to analyze video data acquired from factory monitoring equipment, visualize the manufacturing process, and detect errors."

[1319] This system will enable food manufacturers to provide consumers with reliable information, and the error detection function in the manufacturing process is expected to improve product quality. In addition, by incorporating an emotion engine, it will be possible to grasp user satisfaction in real time, leading to improved services.

[1320] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1321] Step 1: Acquire video data

[1322] The server acquires video data in real time from the monitoring devices installed in the factory. Each monitoring camera transmits the video data to the server via streaming using RTSP (Real-Time Streaming Protocol).

[1323] Input: Real-time video data sent from a surveillance device

[1324] Output: Raw data stored on the server

[1325] Specific operation: Save video data to cloud storage such as Amazon S3.

[1326] Step 2: Preprocessing the video data

[1327] The server performs preprocessing on the stored video data, using the Python OpenCV library to remove noise and split the video data into frames.

[1328] Input: Raw data stored on the server

[1329] Output: Pre-processed video data

[1330] Specific operation: Apply an average filter to reduce noise and extract a specific frame.

[1331] Step 3: Analysis with a generative model

[1332] The pre-processed video data is then fed into a generative model, which is built using the TensorFlow and PyTorch frameworks to analyze the data and detect errors and anomalies in the manufacturing process.

[1333] Input: Preprocessed video data

[1334] Output: Analysis results for each process and abnormality detection information

[1335] Specific operation: Data is input into the generative AI model, which analyzes the worker's actions and the product's condition. If an abnormality is detected, an alert is generated.

[1336] Step 4: Generate a work log

[1337] Based on the analysis results, the server generates a work log, using a NoSQL database such as MongoDB to record information about each process and any anomalies in chronological order.

[1338] Input: Analysis results from the generative model

[1339] Output: Work log

[1340] Specific operation: A text log is generated based on the analysis results and saved in a database.

[1341] Step 5: Generate and register your NFT

[1342] The server generates a non-fungible token (NFT) based on the generated work log and registers the NFT on the Ethereum blockchain using Python's Web3 library.

[1343] Input: Work Log

[1344] Output: NFT registered on the blockchain

[1345] Specific operation: Generates an NFT containing information on the product's origin, manufacturing process, and quality inspection results, and manages it using a smart contract.

[1346] Step 6: Receive real-time video and error reports

[1347] The terminal receives real-time video data and error notifications sent from the server, allowing workers and managers to monitor the status of the manufacturing site in real time.

[1348] Input: Video data and error notifications from the server

[1349] Output: Real-time video and notifications displayed on the display

[1350] Specific operation: Receives video data via WebSocket and displays it in VLC media player. If an abnormality is detected, it notifies you with sound and light.

[1351] Step 7: Receive and view your NFT

[1352] Users receive the corresponding NFT when they purchase a product, transfer ownership of the NFT through a smartphone app or online platform, and then access a blockchain viewer such as EtherScan to view the recorded NFT.

[1353] Input: NFT data from the server

[1354] Output: NFT information viewed on the user's smartphone or PC

[1355] How it works: Use an app like MetaMask to receive your NFT and then use EtherScan to view detailed manufacturing and origin information.

[1356] Step 8: Analyze the sentiment data

[1357] The server analyzes the emotional data when users receive NFTs, and uses an NLP library to analyze user feedback and comments to evaluate user satisfaction.

[1358] Input: User feedback and comments

[1359] Output: User satisfaction evaluation results

[1360] How it works: Feedback data is fed into an NLP library, and satisfaction is scored based on sentiment analysis. Additional product information and special offers are provided based on the evaluation results.

[1361] (Application example 2)

[1362] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1363] In conventional manufacturing processes, there are systems for product quality control and improving the reliability of data related to the manufacturing process, but there are limited means of providing reliable information directly to consumers. Furthermore, it is difficult to grasp consumer satisfaction in real time, making it difficult to quickly implement specific countermeasures. This has prevented companies from properly understanding consumer needs and emotions and improving the quality of their services.

[1364] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1365] In this invention, the server includes generative model means for analyzing video data acquired from a monitoring device, means for generating a work log based on the results of analysis by the generative model means, means for generating the work log as a non-fungible token, means for providing the non-fungible token to a user, emotion analysis means for recognizing user emotions, and means for evaluating user satisfaction based on emotion data analyzed by the emotion analysis means. This makes it possible to provide consumers with reliable product information, grasp consumer purchasing behavior and satisfaction in real time, and improve service quality.

[1366] A "monitoring device" is a device used to monitor a specific area, such as a factory or a brick-and-mortar store, in real time and acquire video data.

[1367] "Video data" refers to real-time visual information acquired from a monitoring device, and is the data to be analyzed.

[1368] A "generative model means" is a model incorporating deep learning and artificial intelligence techniques that is used to analyze captured video data and extract specific information.

[1369] The "work log" is data that records which work was performed in which process based on the results of analysis by the generative model means.

[1370] A "non-fungible token" is a unique digital asset created based on blockchain technology that contains information about the product's origin and manufacturing process.

[1371] "User" refers to a consumer or customer who uses the system and receives information about products and services.

[1372] "Emotion analysis means" is a technology for analyzing a user's facial expressions and behavior to determine their emotional state at that time.

[1373] "Satisfaction" refers to the degree of satisfaction a user feels with a product or service, and is evaluated by sentiment analysis means.

[1374] The present invention provides a system that combines a monitoring device, a generative model, work log generation, non-fungible token generation and provision, emotion analysis means, and satisfaction assessment means.

[1375] Server Processing

[1376] The server acquires video data in real time from monitoring devices installed in factories and brick-and-mortar stores. Each monitoring camera transmits video data to the server via the network. The received video data is first temporarily stored in the server's storage.

[1377] The server performs preprocessing on the stored video data, including noise removal and frame segmentation. The preprocessed data is then input into a generative model (e.g., TensorFlow, Keras). This generative model uses deep learning technology to analyze the video data, monitor the actions of workers and the status of products in real time, and detect errors and anomalies.

[1378] Based on the analysis results, the server generates a work log, which includes details of what operations were performed at which stage and any errors or anomalies detected. This work log is recorded in chronological order. A non-fungible token (NFT) is then generated based on the generated work log. The NFT includes information about the product's origin, manufacturing process, quality inspection results, etc. The server registers the generated NFT on a blockchain (e.g., Ethereum, Hyperledger Fabric), and transfers ownership of the corresponding NFT to the consumer when the product is shipped.

[1379] Furthermore, this system incorporates an emotion analysis mechanism to recognize users' emotions. The server uses an emotion analysis engine (e.g., Affectiva, Microsoft Emotion API) to analyze the emotional data of users who receive NFTs and evaluate their satisfaction.

[1380] Terminal handling

[1381] Terminals in factories or physical stores receive real-time video data sent from the server. The received video data is displayed on a screen, allowing workers and managers to monitor the status of the manufacturing site or physical store in real time. The terminals receive notifications of errors and abnormality detection sent from the server and display them in real time. The details of the error are notified to workers and managers with sound or light, allowing them to quickly identify and address the problem.

[1382] User Action

[1383] When a user purchases a product, they receive a corresponding NFT. Purchase information is sent to a server via a smartphone app or online platform, and ownership of the NFT is transferred to the user. Users can access the blockchain using a smartphone app or PC to view the recorded NFT. The NFT contains detailed information about the product's manufacturing process, country of origin, and quality inspection results, allowing users to verify the product's authenticity.

[1384] For example, if a user purchases a smartphone, they can receive a corresponding NFT. The sentiment analysis engine analyzes emotional data when the user browses the NFT through a smartphone app. For example, if the user expresses excitement or satisfaction while browsing, that information can be used to provide additional product information or special offers. Conversely, if the user expresses dissatisfaction, feedback can be collected and areas for improvement identified.

[1385] Example prompt for a generative AI model:

[1386] / / Customer sentiment analysis prompt

[1387] Input data: Video data captured by a camera

[1388] Goal: Identify customer emotions (happiness, anger, confusion, excitement, etc.) from video data and display the results in real time.

[1389] This system will enable manufacturers and brick-and-mortar store operators to provide consumers with reliable information, and is expected to improve product quality through error detection functions in the manufacturing process and sales environment.Furthermore, by using sentiment analysis tools, it will be possible to grasp user satisfaction in real time, leading to improved services.

[1390] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1391] Step 1:

[1392] The server acquires video data in real time from monitoring devices installed in factories and brick-and-mortar stores. Each monitoring camera sends video data to the server via the network, and this video data is temporarily stored in the server's storage. The input is video data from the monitoring device, and the output is the temporarily stored video data. Specifically, the server checks the connection of the monitoring camera and transfers the video data to the server in a stream format.

[1393] Step 2:

[1394] The server performs preprocessing on the stored video data, such as noise removal and frame division. This preprocessing improves data quality and increases analysis accuracy. The input is temporarily stored video data, and the output is preprocessed video data. Specifically, OpenCV is used to perform noise filtering and divide continuous video into individual frames.

[1395] Step 3:

[1396] The server inputs the preprocessed video data into a generative model to analyze the worker's actions and the product's status. Generative AI models (e.g., TensorFlow, Keras) are used to detect errors and anomalies in real time. The input is the preprocessed video data, and the output is the error and anomaly detection results. Specifically, features are extracted for each frame, and abnormal actions are identified using a deep learning model.

[1397] Step 4:

[1398] The server generates a work log based on the analysis results obtained from the generative model and records it in chronological order. The input is the error and anomaly detection results, and the output is the work log. Specifically, the analysis results are recorded in a log file in text format, and a timestamp is added to each log.

[1399] Step 5:

[1400] The server generates a non-fungible token (NFT) based on the generated work log and registers it on the blockchain. The input is the work log, and the output is the NFT registered on the blockchain. Specifically, the work log is converted to JSON format and registered on the blockchain via a smart contract.

[1401] Step 6:

[1402] The terminal receives real-time video data and error notifications sent from the server and displays them on the display. The input is video data and error notifications from the server, and the output is displayed on the display. Specifically, the video data is streamed, and if an error occurs, the worker is notified with sound and light.

[1403] Step 7:

[1404] When a user purchases a product, they receive the corresponding NFT and view it on a smartphone app or PC. The input is purchase information, and the output is the display of the NFT on the user's smartphone or PC. Specifically, the user enters their purchase information and transfers NFT ownership on the blockchain.

[1405] Step 8:

[1406] The server uses an emotion analysis engine to analyze the user's emotional data when viewing NFTs and evaluate their satisfaction. The input is the user's facial expression data, and the output is the emotion analysis results and satisfaction rating. Specifically, it uses Affectiva and the Microsoft Emotion API to analyze emotions from the user's facial expressions and provides feedback to the user along with the results.

[1407] Example prompt sentence:

[1408] / / Customer sentiment analysis prompt

[1409] Input data: Video data captured by a camera

[1410] Goal: Identify customer emotions (happiness, anger, confusion, excitement, etc.) from video data and display the results in real time.

[1411] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1412] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1413] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1414] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1415] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1416] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1417] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1418] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1419] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1420] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1421] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1422] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1423] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1424] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1425] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1426] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1427] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1428] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1429] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1430] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1431] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1432] The following is further disclosed regarding the above embodiment.

[1433] (Claim 1)

[1434] a generative model means for analyzing video data acquired from a monitoring device;

[1435] a means for generating a work log based on the results of analysis by the generative model means;

[1436] means for generating the work log as a non-fungible token;

[1437] A means for providing the non-fungible tokens to users;

[1438] A system including:

[1439] (Claim 2)

[1440] 2. The system according to claim 1, wherein the generative model means has a function of detecting errors and abnormalities in the manufacturing process.

[1441] (Claim 3)

[1442] 2. The system of claim 1, wherein the non-fungible token includes product origin information and manufacturing process information.

[1443] "Example 1"

[1444] (Claim 1)

[1445] means for pre-processing video data acquired from a monitoring device;

[1446] deep learning model means for analyzing the video data;

[1447] means for generating a work log based on the analysis results;

[1448] means for generating the work log as a non-fungible token;

[1449] A means for providing the non-fungible tokens to users;

[1450] A system including:

[1451] (Claim 2)

[1452] The system of claim 1, wherein the deep learning model means has a function of detecting errors and abnormalities in the manufacturing process.

[1453] (Claim 3)

[1454] 2. The system of claim 1, wherein the non-fungible token includes product origin information and manufacturing process information.

[1455] "Application Example 1"

[1456] (Claim 1)

[1457] a generative model means for analyzing video data acquired from a monitoring device;

[1458] a means for generating a work log based on the results of analysis by the generative model means;

[1459] means for generating the work log as a non-fungible token;

[1460] A means for providing the non-fungible tokens to users;

[1461] a notification means for acquiring video data from the monitoring device in real time and detecting errors or abnormalities;

[1462] a means for displaying a notification of abnormality detection by the notification means in real time;

[1463] A system including:

[1464] (Claim 2)

[1465] 2. The system according to claim 1, wherein the generative model means has a function of detecting errors and abnormalities in the manufacturing process.

[1466] (Claim 3)

[1467] 2. The system of claim 1, wherein the non-fungible token includes product origin information and manufacturing process information.

[1468] "Example 2: Combining Emotion Engines"

[1469] (Claim 1)

[1470] a generative model means for analyzing video data acquired from a monitoring device;

[1471] a means for generating a work log based on the results of analysis by the generative model means;

[1472] means for generating the work log as a non-fungible token;

[1473] A means for providing the non-fungible tokens to users;

[1474] emotion engine means for analyzing emotion data of a user;

[1475] A system including:

[1476] (Claim 2)

[1477] 2. The system according to claim 1, wherein the generative model means has a function of detecting errors and abnormalities in the manufacturing process.

[1478] (Claim 3)

[1479] 2. The system of claim 1, wherein the non-fungible token includes product origin information and manufacturing process information.

[1480] "Application example 2 when combining emotion engines"

[1481] (Claim 1)

[1482] a generative model means for analyzing video data acquired from a monitoring device;

[1483] a means for generating a work log based on the results of analysis by the generative model means;

[1484] means for generating the work log as a non-fungible token;

[1485] A means for providing the non-fungible tokens to users;

[1486] emotion analysis means for recognizing the emotion of a user;

[1487] means for evaluating user satisfaction based on the emotion data analyzed by the emotion analysis means;

[1488] A system including:

[1489] (Claim 2)

[1490] 2. The system according to claim 1, wherein the generative model means has a function of detecting errors and abnormalities in the manufacturing process.

[1491] (Claim 3)

[1492] 2. The system of claim 1, wherein the non-fungible token includes product origin information and manufacturing process information.

[1493] (Claim 4)

[1494] 2. The system according to claim 1, wherein the emotion analysis means has a function of analyzing facial expression data of the customer and determining the emotional state of the customer, such as interest, satisfaction, or dissatisfaction. [Explanation of symbols]

[1495] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a generative model means for analyzing video data acquired from a monitoring device; a means for generating a work log based on the results of analysis by the generative model means; means for generating the work log as a non-fungible token; A means for providing the non-fungible tokens to users; A system including:

2. 2. The system according to claim 1, wherein the generative model means has a function of detecting errors and anomalies in the manufacturing process.

3. 10. The system of claim 1, wherein the non-fungible token includes product origin and manufacturing process information.

Images