system

Abstract

Provide a system. [Solution] A means to input or upload past inspection data; A means of analyzing stored past inspection data and learning data characteristics and patterns; A means to enter or upload new claim data; means for analyzing new billing data and generating acceptance information based on patterns learned from past data; A means for storing the generated acceptance information and providing it to a user; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The conventional method of creating inspection information was done manually, which required a lot of time and effort. Furthermore, when it is difficult to change the inspection method, efficient data generation is required. As a result, there was a need to improve the accuracy and efficiency of inspection work. [Means for solving the problem]

[0005] The present invention proposes a system that includes a means for inputting or uploading past inspection data and a means for analyzing the saved past inspection data to learn data characteristics and patterns. It also includes a means for inputting or uploading new billing data and a means for analyzing the new billing data and generating inspection information based on patterns learned from the past data. It also includes a means for saving the generated inspection information and providing it to users. This system efficiently generates inspection information, reduces the burden of manual work, and provides flexibility that makes it applicable even when it is difficult to change the inspection method.

[0006] "Inspection data" is a collection of information that describes the details of the request and the results of the request, and serves as the basis for verifying the legitimacy and completeness of the transaction.

[0007] An "input or uploading means" is any device or software that includes an interface or functionality for providing data to the system.

[0008] "Storing" refers to the process of temporarily or permanently storing received data in a storage device.

[0009] "Means of analysis" are algorithms or programs that read data, interpret its contents, and extract patterns and features.

[0010] The "learning tool" is a machine learning algorithm that uses past data to identify specific rules and patterns and use them to inform future decisions.

[0011] "New Claim Data" means information about a new claim that has recently been entered or uploaded.

[0012] "Inspection information" is information that includes the result indicating whether or not inspection is possible for the request data.

[0013] "Means of generation" are algorithms and programs that use analysis results and learning models to generate new data and information.

[0014] The "means for providing" refers to an interface or output device for displaying the created inspection results to the user.

[0015] A "system" is a set of devices and software that includes the above means, and is constructed to achieve a specific purpose. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention relates to a system that improves the efficiency of inspection work and solves the problem of conventional manual generation of inspection information. This system operates in conjunction with the server, terminals, and users to automate the process from inputting inspection data to analyzing it, generating inspection information, and outputting the results.

[0038] System configuration

[0039] First, the terminal provides an interface for users to input or upload past data. Users can manually input past inspection data or upload it in a CSV file format. This data includes the invoice date, invoice item, amount, and whether or not the data was accepted.

[0040] The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area, after which it stores it in a database.

[0041] Analysis and learning from past data

[0042] The server analyzes the stored past inspection data and uses a machine learning algorithm to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an item can be inspected. For example, if the amount is small or if certain items are included, it will find patterns that indicate "acceptable inspection."

[0043] Receiving and analyzing new claims data

[0044] Users enter or upload new billing data through a similar interface. The new data contains the same information as the previous data (billing date, billing item, amount, etc.). The server receives this new data, checks its format, and stores it.

[0045] Generating acceptance information

[0046] The server automatically generates inspection information based on newly received billing data using a model learned from past data. For example, based on past learning results, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[0047] Saving and providing inspection results

[0048] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[0049] Specific examples

[0050] For example, consider the case where a user uploads billing data and its inspection results from the past year to the system. This past data includes the billing date, billing item, amount, and whether or not inspection was possible. Data such as "January 1, 2022, Equipment Purchase, ¥10,000, Inspection Possible" and "February 1, 2022, Conference Expenses, ¥20,000, Inspection Not Possible" are entered.

[0051] The server analyzes this data and later learns patterns such as "the smaller the amount, the higher the probability of acceptance" and "if the conference fee is high, it will not be accepted."

[0052] When the user enters new billing data "January 1, 2023, equipment purchase, ¥15,000" and "February 1, 2023, conference fee, ¥30,000," the server analyzes this and generates new acceptance information "January 1, 2023, equipment purchase, ¥15,000, accepted" and "February 1, 2023, conference fee, ¥30,000, not accepted."

[0053] This system allows users to efficiently carry out inspection work and significantly reduces the burden of manual work. In addition, even when it is difficult to change the inspection method, the system can be used to flexibly respond.

[0054] The processing flow will be explained below.

[0055] Step 1:

[0056] User enters or uploads historical data

[0057] Users can manually enter past inspection data through the system interface or upload it as a CSV file.

[0058] Specifically, the user clicks the "Select File" button to select the CSV file.

[0059] Step 2:

[0060] The server receives and stores the data

[0061] The server receives the past data sent by the user and checks the format and content of the data.

[0062] If the format is correct, the data is stored in a temporary storage area.

[0063] Step 3:

[0064] The server stores the data in a database

[0065] The server stores the data in the database after the format check is complete.

[0066] At this time, checks are also carried out to prevent errors such as data duplication or loss.

[0067] Step 4:

[0068] The server analyzes past data

[0069] The server analyzes past inspection data stored in a database and extracts various categories of data.

[0070] For example, classify data with tags such as "acceptable" and "unacceptable."

[0071] Step 5:

[0072] The server uses a machine learning model to learn the characteristics of the data.

[0073] The server uses machine learning algorithms to learn patterns and characteristics of inspections.

[0074] For example, patterns that determine whether or not a product can be accepted are searched for and saved as a model.

[0075] Step 6:

[0076] A user enters or uploads new claim data

[0077] Users enter new claim data through the system interface or upload it as a CSV file.

[0078] Step 7:

[0079] The server receives the new data and checks the format.

[0080] The server receives the newly submitted billing data and verifies that the data format is correct.

[0081] Step 8:

[0082] The server saves the new data

[0083] The server stores the data in a database after format checking is complete.

[0084] Step 9:

[0085] The server parses the new billing data

[0086] The server retrieves the new billing data and parses the information.

[0087] For example, extract billing items and amounts.

[0088] Step 10:

[0089] The server generates inspection information using the learning model.

[0090] The server uses the trained model to automatically generate an acceptance or rejection decision for new billing data.

[0091] For example, if the purchase of equipment on January 1, 2023 costs 15,000 yen, it is determined that the equipment can be inspected.

[0092] Step 11:

[0093] The server stores the generated inspection results and provides them to the user.

[0094] The server stores the generated inspection information in a database.

[0095] At the same time, the inspection results are displayed to the user on the system interface.

[0096] Step 12:

[0097] The user checks, corrects, and approves the inspection results

[0098] The user checks the inspection results through the system interface, makes corrections as necessary, and then approves them.

[0099] Example 1

[0100] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0101] Conventional inspection work is performed manually, which makes it inefficient and prone to human error. It is also difficult to analyze past inspection data and use it to make future inspection decisions. This makes the criteria for inspection decisions unclear, and increases the likelihood of inconsistent results. To solve this problem, a system is needed that streamlines inspection work and automatically generates inspection information using past data.

[0102] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0103] In this invention, the server includes means for receiving past inspection data, performing a format check, and saving it in a temporary storage area, means for analyzing the saved past inspection data and storing it in a database, means for analyzing the saved past inspection data and learning data characteristics and patterns using a machine learning algorithm, means for receiving new billing data, performing a format check, and saving it in a database, means for analyzing the new billing data and generating inspection information based on patterns learned from the past data, and means for saving the generated inspection information in a database and providing it to a user so that it can be checked via a terminal. This makes it possible to improve the efficiency and accuracy of inspection work.

[0104] "Inspection data" refers to data that includes information related to the inspection work, such as the invoice date, invoice item, amount, and whether or not the inspection is possible.

[0105] A "server" is a computer system used for the purposes of processing, storing, analyzing received data, and generating necessary information.

[0106] A "terminal" is a device that allows a user to input data or view system results. This includes computers, smartphones, tablets, etc.

[0107] "User" means a person or organization that uses the system to enter or upload acceptance data and verify the generated acceptance information.

[0108] "Format check" is the process of verifying that the received data conforms to the expected format and structure.

[0109] The "temporary storage area" is a storage space for temporarily storing data before it is finally stored in the database.

[0110] A "database" is a system that stores data in an organized, searchable, updateable, and manageable manner. This includes relational databases and NoSQL databases.

[0111] A "machine learning algorithm" is an algorithm that automatically learns patterns and features from data and makes predictions and classifications for new data. Examples include decision trees and neural networks.

[0112] "Analysis" is the process of examining data in detail to find useful information and patterns within it.

[0113] "Inspection information" is information that includes the results of a judgment, such as whether or not a certain request can be inspected, generated based on past data and machine learning algorithms.

[0114] "Automatic generation" is a function in which the system automatically generates data without manual human intervention.

[0115] "Learning" is the process by which a machine learning model uses past data to understand patterns and features and apply them to future data.

[0116] A "prompt sentence" is a guide message that is displayed to prompt the user to take a specific action.

[0117] This invention relates to a system for streamlining inspection work and solving the problem of conventional manual generation of inspection information. This system involves the collaboration of a server, terminals, and users to automate the entire process from inputting inspection data to analyzing it, generating inspection information, and outputting the results.

[0118] System configuration

[0119] First, the user is provided with an interface on the terminal to input or upload past inspection data. The user can manually input the past inspection data or upload it in a format such as a CSV file. This data includes the invoice date, invoice item, amount, whether or not the inspection was successful, etc.

[0120] For example, a user enters "January 1, 2022, Equipment Purchase, 10,000 yen, Accepted for Inspection" into an input form in a web application, or selects and uploads a CSV file.

[0121] Receiving and storing data

[0122] The server receives the data sent from the terminal. After receiving the data, it performs a format check and eliminates any inappropriate data. Next, it saves the data in a temporary storage area and then stores it in a database. For format checks and database operations, the Python pandas library and MySQL (registered trademark) are used, for example.

[0123] Analysis and learning from past data

[0124] The server analyzes the stored past inspection data. For the analysis, it uses machine learning libraries such as Python's scikit-learn and TENSORFLOW (registered trademark) to learn the characteristics and patterns of the data. This allows it to build criteria for determining whether or not an inspection is acceptable. Specifically, it trains a model using scikit-learn's DecisionTreeClassifier.

[0125] For example, it learns patterns such as "the smaller the amount, the higher the probability that inspection is possible" and "if the conference fee is high, inspection is impossible."

[0126] Entering New Claim Data

[0127] Users enter or upload new billing data using the web application interface, just like with past data. New data includes billing date, billing item, and amount.

[0128] As an example, let's assume a process in which a user enters "January 1, 2023, Equipment Purchase, ¥15,000" in the data entry field and uploads it.

[0129] Analyzing New Data

[0130] The server receives the newly received billing data, checks the format, then stores the data and analyzes it using a machine learning model learned from past data.

[0131] Generating acceptance information

[0132] The server automatically generates acceptance information based on the analysis results. Specifically, it uses a trained machine learning model (for example, using the predict method in scikit-learn) to generate prediction results. The prediction results are provided in the form of "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be accepted for inspection."

[0133] Saving and providing inspection results

[0134] The generated inspection information is stored in a database on the server, and users can check these inspection results through the terminal interface. If necessary, users can correct or approve the results.

[0135] Prompt statement

[0136] As an example of a prompt sentence, the message to prompt the user to enter past billing data is shown below.

[0137] Please upload your past billing data. The data must include the billing date, billing item, amount, and whether or not it can be inspected. For example, please enter it in the following format: "January 1, 2022, Equipment Purchase, 10,000 yen, Acceptable."

[0138] This system makes it possible to improve the efficiency and accuracy of inspection work. Users can perform inspection work efficiently, significantly reducing the burden of manual work. It also allows flexible inspection decisions to be made based on past data.

[0139] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0140] Step 1: Data entry

[0141] Users input or upload past inspection data. Using an interface on their device (e.g., a web application), they select and upload a CSV file, etc. The input data includes the billing date, billing item, amount, and whether or not the data was accepted.

[0142] Input: Past inspection data (e.g., "January 1, 2022, Equipment Purchase, ¥10,000, Accepted")

[0143] Output: Uploaded CSV file or input form data

[0144] Specific operation: The user selects a CSV file and clicks the upload button, or manually enters past billing data into the data entry field.

[0145] Step 2: Receiving and storing data

[0146] The server receives the data sent from the terminal. It checks the format of the received data and eliminates any inappropriate data. It then saves the data in a temporary storage area and stores the correct data in a database. Specifically, it uses the Python pandas library to read the CSV file and MySQL as the database.

[0147] Input: Past inspection data uploaded by the user

[0148] Output: Correctly formatted acceptance data stored in the database

[0149] Specific operation: Read a CSV file using pandas, validate the data format, and insert the validated data into a MySQL database.

[0150] Step 3: Analyze and learn from past data

[0151] The server analyzes the stored past inspection data. It uses machine learning libraries such as Python's scikit-learn and TensorFlow to learn the features and patterns of the data. This allows it to build criteria for determining whether or not an inspection is acceptable. Specifically, it trains a model using scikit-learn's DecisionTreeClassifier.

[0152] Input: Past inspection data stored in the database

[0153] Output: A trained machine learning model

[0154] Specific operation: A decision tree model is trained using scikit-learn to learn the criteria for determining whether or not a product can be accepted.

[0155] Step 4: Enter new claim data

[0156] Users enter or upload new billing data using the web application interface, just like with past data. New data includes billing date, billing item, and amount.

[0157] Input: New billing data (e.g., "January 1, 2023, Equipment Purchase, ¥15,000")

[0158] Output: New billing data uploaded

[0159] Specific operation: The user enters "January 1, 2023, Equipment Purchase, ¥15,000" in the data input field and clicks the upload button.

[0160] Step 5: Analyze the new claims data

[0161] The server receives newly received billing data, performs format checks, stores valid data in a database, and analyzes new data using machine learning models learned from past data.

[0162] Input: New claim data

[0163] Output: New, correctly formatted billing data, analysis results

[0164] Specific operations: Read new billing data using pandas, check the format, and save it to the database. Analyze the new data using scikit-learn's predict method.

[0165] Step 6: Generate acceptance information

[0166] The server automatically generates inspection information based on the analysis results. It generates predictions using a machine learning model (for example, using the predict method in scikit-learn) that has learned from past data.

[0167] Input: Parsed new claims data

[0168] Output: Generated acceptance information (e.g., "January 1, 2023, Equipment Purchase, ¥15,000, Accepted")

[0169] Specific operation: Automatically generate inspection information using a trained machine learning model.

[0170] Step 7: Save and provide inspection results

[0171] The server stores the generated inspection information in a database. Users can check these inspection results through the terminal interface and make corrections or approvals as necessary.

[0172] Input: Generated acceptance information

[0173] Output: Inspection information stored in the database, inspection results provided to the user

[0174] Specific operation: The inspection results are inserted into the database and displayed in the web interface.

[0175] (Application example 1)

[0176] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0177] Inspection work at logistics centers involves a lot of manual work, which takes time and effort. In addition, inspection standards vary from person to person, which can lead to a lack of consistency in inspection results. Furthermore, in many cases, past inspection data cannot be fully utilized, which makes it difficult to make efficient inspection decisions.

[0178] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0179] In this invention, the server includes means for inputting or uploading past inspection data, means for analyzing the saved past inspection data and learning data characteristics and patterns, means for inputting or uploading new billing data, means for analyzing the new billing data and generating inspection information based on patterns learned from the past data, means for saving the generated inspection information and providing it to users, and means for using a robot in a logistics facility to scan specified products and generate inspection information. This makes inspection work more efficient and enables inspections to be performed according to consistent standards.

[0180] "Past inspection data" is information related to inspection work that has been carried out in the past, and includes detailed data such as the date of invoice, the item of invoice, the amount, and whether or not the item was inspected.

[0181] "Means for input or upload" refers to the functionality of the interface for a user to manually enter data or provide an existing data file to the system.

[0182] "Means of analyzing stored past inspection data and learning data features and patterns" refers to the functionality of machine learning algorithms that analyze past inspection data, learn specific patterns and features, and build criteria for judging future data.

[0183] "New Claims Data" means data containing information about new claims that are currently or will be made in the future.

[0184] "Means for entering or uploading new claim data" refers to the functionality of the interface that allows a user to provide new claim data to the system.

[0185] "Means for analyzing new billing data and generating inspection information based on patterns learned from past data" refers to the function of analyzing new billing data, determining whether or not to inspect it based on patterns learned in the past, and generating that information.

[0186] "Means for saving the generated inspection information and providing it to the user" refers to the interface function for saving the inspection results generated by the system and providing the information so that the user can check it.

[0187] A "logistics facility" is a facility used for logistics operations such as receiving, storing, inspecting, and shipping goods.

[0188] "Robots that scan designated products and generate inspection information" refer to autonomous robots that scan barcodes, QR codes (registered trademark), etc. of designated products within a logistics facility and automatically generate inspection information based on that information.

[0189] This invention provides a system for improving the efficiency of inspection work at logistics facilities. The specific configuration and operation of this system will be described below.

[0190] First, the user uses an interface on the terminal to input or upload past inspection data. The user can input the past inspection data manually or upload it as a CSV file. This past inspection data includes information such as the invoice date, invoice item, amount, and whether or not the inspection was successful.

[0191] The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area.Then, it stores this data in a database.

[0192] The server then analyzes the stored past inspection data and uses machine learning algorithms to learn the characteristics and patterns of the data. This analysis and learning is performed using machine learning frameworks such as TensorFlow. Through this learning, criteria for determining whether or not an item can be inspected are built within the server. For example, patterns that indicate "acceptance" are found when "the amount is small" or "specific items are included."

[0193] The user then enters or uploads new billing data through a similar interface. This new data contains the same information as the previous data (billing date, billing item, amount, etc.). The server receives this new data, checks its format, and stores it.

[0194] The server automatically generates inspection information based on newly received billing data using a model learned from past data. For example, based on past learning results, the server can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[0195] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[0196] Furthermore, this system includes a robot that scans designated products at logistics facilities and generates inspection information. The robot scans the products within the logistics facility and sends the data to a server. The server analyzes the data and uses a model trained on past data to determine whether or not the product can be inspected. The generated inspection information is displayed on the robot's display, allowing users to easily check it.

[0197] As a concrete example, consider the case where a user uploads billing data from the past year to the system. This data includes information such as "January 1, 2022, Equipment Purchase, ¥10,000, Acceptable" and "February 1, 2022, Meeting Expenses, ¥20,000, Unacceptable." The server analyzes this data and learns patterns. Subsequently, when new billing data "January 1, 2023, Equipment Purchase, ¥15,000" and "February 1, 2023, Meeting Expenses, ¥30,000" are received, the server generates acceptance information for "January 1, 2023, Equipment Purchase, ¥15,000, Acceptable" and "February 1, 2023, Meeting Expenses, ¥30,000, Unacceptable."

[0198] An example of a prompt is, "Based on the inspection data from the past year, determine whether or not new products can be inspected and record this automatically." In this way, it is possible to improve the efficiency of inspection work at logistics facilities and provide consistent inspection standards.

[0199] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0200] Step 1:

[0201] The user inputs or uploads past inspection data.

[0202] Specifically, users can manually enter past billing data (billing date, billing item, amount, acceptance / acceptance status, etc.) through the terminal interface, or upload a CSV file. The entered data is checked for formatting on the terminal and then sent to the server.

[0203] Input: Past inspection data

[0204] Output: Format-checked past inspection data is sent to the server

[0205] Step 2:

[0206] The server receives and stores past inspection data.

[0207] The server receives the data sent from the terminal, checks the format again, and saves the data in a temporary storage area.Then, it stores this data in a database.

[0208] Input: Format-checked past inspection data

[0209] Output: Past inspection data stored in a database

[0210] Step 3:

[0211] The server analyzes the stored past inspection data and learns the characteristics and patterns of the data.

[0212] The server analyzes past inspection data stored in a database using machine learning frameworks such as TensorFlow to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an inspection is acceptable.

[0213] Input: Past inspection data

[0214] Output: A trained machine learning model

[0215] Step 4:

[0216] A user enters or uploads new claim data.

[0217] Users can manually enter new billing data (billing date, billing item, amount, etc.) through their terminal or upload it as a CSV file, which is also format-checked on the terminal and then sent to the server.

[0218] Input: New claim data

[0219] Output: New format-checked claim data is sent to the server

[0220] Step 5:

[0221] The server receives the new billing data and generates acceptance information.

[0222] The server analyzes newly received billing data using a machine learning model and generates acceptance information based on patterns learned from past data. For example, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be accepted for inspection."

[0223] Input: New claims data, trained machine learning model

[0224] Output: Generated acceptance information

[0225] Step 6:

[0226] The server stores the generated acceptance information and provides it to the user.

[0227] The generated inspection information is stored in the server's database. An interface is then provided so that users can check the inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[0228] Input: Generated acceptance information

[0229] Output: Inspection information stored in the database, interface provided to the user

[0230] Step 7:

[0231] The robot scans the specified product and generates inspection information.

[0232] Robots used in logistics facilities scan the barcodes or QR codes of designated products and send the data to a server. The server analyzes the data and uses machine learning models to determine whether or not the product should be inspected. The inspection results are then displayed on the robot's display.

[0233] Input: Scanned product data

[0234] Output: Inspection information displayed on the robot's display

[0235] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0236] This invention is a system that streamlines inspection work and solves the problem of conventional manual generation of inspection information, and also combines an "emotion engine" that recognizes user emotions and reflects them in the operation of the system. This system operates in conjunction with the server, terminal, and user, automating the entire process from inputting inspection data to analyzing it, generating inspection information, outputting the results, and recognizing user emotions.

[0237] System configuration

[0238] First, the terminal provides an interface for users to input or upload past data. Users can manually input past inspection data or upload it in a CSV file format. This data includes the invoice date, invoice item, amount, and whether or not the data was accepted.

[0239] The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area, after which it stores it in a database.

[0240] Analysis and learning from past data

[0241] The server analyzes the stored past inspection data and uses a machine learning algorithm to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an item can be inspected. For example, if the amount is small or if certain items are included, it will find patterns that indicate "acceptable inspection."

[0242] Receiving and analyzing new claims data

[0243] Users enter or upload new billing data through a similar interface. The new data contains the same information as the previous data (billing date, billing item, amount, etc.). The server receives this new data, checks its format, and stores it.

[0244] Generating acceptance information

[0245] The server automatically generates inspection information based on newly received billing data using a model learned from past data. For example, based on past learning results, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[0246] Saving and providing inspection results

[0247] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[0248] Emotion engine integration

[0249] The present invention further integrates an emotion engine to recognize the user's emotions, which analyzes data such as the user's facial expressions and tone of voice to identify the user's emotional state.

[0250] The emotion engine allows the system to adjust the interface display when a user is stressed or confused, helping them to complete their tasks more smoothly. For example, if a user is in a negative emotional state, the system can display error messages in a gentle tone and provide additional help options. Additionally, if a user feels anxious, the system can send a notification prompting them to reconfirm the inspection information.

[0251] Specific examples

[0252] For example, consider the case where a user uploads billing data and its inspection results from the past year to the system. This past data includes the billing date, billing item, amount, and whether or not inspection was possible. Data such as "January 1, 2022, Equipment Purchase, ¥10,000, Inspection Possible" and "February 1, 2022, Conference Expenses, ¥20,000, Inspection Not Possible" are entered.

[0253] The server analyzes this data and later learns patterns such as "the smaller the amount, the higher the probability of acceptance" and "if the conference fee is high, it will not be accepted."

[0254] When the user enters new billing data "January 1, 2023, equipment purchase, ¥15,000" and "February 1, 2023, conference fee, ¥30,000," the server analyzes this and generates new acceptance information "January 1, 2023, equipment purchase, ¥15,000, accepted" and "February 1, 2023, conference fee, ¥30,000, not accepted."

[0255] Furthermore, if the emotion engine analyzes the user's facial expression data and determines that the user is feeling stressed, the system will display the inspection results in a more friendly and understandable format, providing additional options such as "Do you want to check again?" or "Do you need help?", reducing the burden on the user.

[0256] This system allows users to efficiently perform inspection work and significantly reduces the burden of manual work. In addition, the integration of an emotion engine allows for flexible responses that take into account the user's emotional state, providing excellent flexibility that can be applied even in cases where it is difficult to change the inspection method.

[0257] The processing flow will be explained below.

[0258] Step 1:

[0259] User enters or uploads historical data

[0260] Users can manually enter past inspection data through the system interface or upload it as a CSV file.

[0261] Specifically, the user clicks the "Select File" button to select the CSV file.

[0262] Step 2:

[0263] The server receives and stores the data

[0264] The server receives the past data sent by the user and checks the format and content of the data.

[0265] If the format is correct, the data is stored in a temporary storage area.

[0266] Step 3:

[0267] The server stores the data in a database

[0268] The server stores the data in the database after the format check is complete.

[0269] At this time, checks are also carried out to prevent errors such as data duplication or loss.

[0270] Step 4:

[0271] The server analyzes past data

[0272] The server analyzes past inspection data stored in a database and extracts various categories of data.

[0273] For example, classify data with tags such as "acceptable" and "unacceptable."

[0274] Step 5:

[0275] The server uses a machine learning model to learn the characteristics of the data.

[0276] The server uses machine learning algorithms to learn patterns and characteristics of inspections.

[0277] For example, patterns that determine whether or not a product can be accepted are searched for and saved as a model.

[0278] Step 6:

[0279] A user enters or uploads new claim data

[0280] Users enter new claim data through the system interface or upload it as a CSV file.

[0281] Step 7:

[0282] The server receives the new data and checks the format.

[0283] The server receives the newly submitted billing data and verifies that the data format is correct.

[0284] Step 8:

[0285] The server saves the new data

[0286] The server stores the data in a database after format checking is complete.

[0287] Step 9:

[0288] The server parses the new billing data

[0289] The server retrieves the new billing data and parses the information.

[0290] For example, extract billing items and amounts.

[0291] Step 10:

[0292] The server generates inspection information using the learning model.

[0293] The server uses the trained model to automatically generate an acceptance or rejection decision for new billing data.

[0294] For example, if the purchase of equipment on January 1, 2023 costs 15,000 yen, it is determined that the equipment can be inspected.

[0295] Step 11:

[0296] The server stores the generated inspection results and provides them to the user.

[0297] The server stores the generated inspection information in a database.

[0298] At the same time, the inspection results are displayed to the user on the system interface.

[0299] Step 12:

[0300] Emotion engine recognizes user emotions

[0301] The emotion engine analyzes facial expressions and tone of voice through the user's camera and microphone.

[0302] This identifies the user's emotional state, such as whether they are stressed or confused.

[0303] Step 13:

[0304] The server adjusts the interface based on the emotional state.

[0305] The server adjusts the way the system interface is displayed based on the user's emotional state obtained from the emotion engine.

[0306] For example, if a user is stressed, provide a more friendly and understandable display or additional help options.

[0307] Step 14:

[0308] The user checks, corrects, and approves the inspection results

[0309] The user checks the inspection results through the system interface, makes corrections as necessary, and then approves them.

[0310] The emotion engine will display a notification to encourage reconsideration if the user is feeling anxious.

[0311] Example 2

[0312] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0313] In conventional inspection work, inspection information is generated manually, which requires a lot of time and effort. In addition, because the user's emotional state is not taken into consideration, users often feel stressed when using the system. This reduces work efficiency and increases the risk of mistakes. To solve these problems, a system that automatically generates inspection information and takes the user's emotional state into consideration is needed.

[0314] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0315] In this invention, the server includes a means for inputting or uploading past inspection data, a means for analyzing the stored past inspection data and learning data characteristics and patterns, and a means for inputting or uploading new billing data, which enables automatic generation of inspection information and flexible system display that takes into account the user's emotional state.

[0316] "Past inspection data" is information relating to inspection work carried out in the past, including the date of invoice, invoice item, amount, whether or not inspection was possible, and the like.

[0317] "Input or upload means" means an interface through which a user can manually enter data or upload it to the system in the form of a file.

[0318] The "means for analyzing the stored past inspection data" refers to an algorithm or program for analyzing the past inspection data received by the server and extracting characteristics and patterns of the data.

[0319] "Methods of learning features and patterns in data" is the process of using machine learning algorithms to automatically learn useful information and trends from past data.

[0320] "New billing data" refers to information about a bill that is newly entered or uploaded into the system, including the billing date, billing items, amount, etc.

[0321] The "means for generating inspection information" refers to an algorithm or program that uses learned patterns or models to automatically generate information such as whether or not an item can be inspected from new invoice data.

[0322] The "means for storing the generated acceptance information" is a process in which the server stores the automatically generated acceptance information in a persistent storage device such as a database.

[0323] The "means for providing to the user" is an interface that allows the user to check, correct, and approve the stored acceptance information.

[0324] "Means for recognizing the user's emotions and adjusting the system's behavior" refers to algorithms or programs that analyze the user's facial expressions, tone of voice, etc., and change the system's display method and behavior based on the results.

[0325] This system improves the efficiency of inspection work and adjusts the operation of the system by recognizing the user's emotions. A specific embodiment of this system will be described below.

[0326] First, the user inputs or uploads past inspection data into the terminal. Specifically, past billing data (billing date, billing item, amount, whether inspection was possible, etc.) can be manually input or uploaded in CSV file format. This data is sent to the server through the terminal interface.

[0327] The server receives the data sent from the device and performs a format check. Format check is a process to ensure the data is correct, and invalid data is logged. The data is then saved in a temporary storage area and then permanently stored in a database (e.g., MySQL).

[0328] Next, the server analyzes the stored past inspection data and learns the data's features and patterns. This process uses machine learning algorithms (such as scikit-learn or TensorFlow). The data is loaded into a pandas DataFrame, features are extracted, and the data is split into training datasets. The data is trained using scikit-learn's RandomForestClassifier, and the generated model is saved in pickle format.

[0329] Next, the user enters or uploads new billing data, just like the previous data. The new data includes the billing date, billing item, amount, etc. This is also sent to the server via the terminal.

[0330] The server receives the new data, checks the format again, and saves it to the database. It then uses the saved learning model to generate acceptance information from the new data. Specifically, it inputs the new data into the trained model and uses the model's .predict method to predict whether the data will be accepted or not. The generated acceptance information is saved in the database.

[0331] The generated inspection information is provided to the user as the inspection result. The terminal provides an interface that allows the user to check the inspection result and make corrections or approvals. The user checks the results and makes corrections or approvals as necessary.

[0332] Furthermore, the system is equipped with an "emotion engine" to recognize the user's emotions. The device captures the user's facial expressions and tone of voice using a camera and microphone and sends this data to the server. The server uses the emotion engine to analyze the user's emotional state and adjusts the system's display method based on the results. For example, facial expression recognition is performed using OpenCV and voice analysis is performed using Google® Cloud Speech-to-Text. If the system determines that the user is feeling stressed, it displays additional options such as "Would you like to check again?" or "Do you need support?"

[0333] Specific examples

[0334] Consider the case where a user uploads billing data from the past year to the system. The past data includes information such as "January 1, 2022, equipment purchase, 10,000 yen, accepted" and "February 1, 2022, conference fee, 20,000 yen, not accepted." The server analyzes this data and learns patterns such as "the smaller the amount, the higher the probability of acceptance" and "if the conference fee is high, it is not accepted."

[0335] When new billing data "January 1, 2023, Equipment Purchase, ¥15,000" and "February 1, 2023, Conference Expenses, ¥30,000" are entered, the server analyzes this and generates new acceptance information "January 1, 2023, Equipment Purchase, ¥15,000, Accepted" and "February 1, 2023, Conference Expenses, ¥30,000, Not Accepted."

[0336] To achieve this, the following prompts are given to the generative AI model:

[0337] "Analyze the following historical data and generate a prompt to predict whether the new billing data will be accepted: 'January 1, 2022, Equipment Purchase, ¥10,000, Acceptable'; 'February 1, 2022, Conference Expenses, ¥20,000, Unacceptable'. The new billing data is 'January 1, 2023, Equipment Purchase, ¥15,000'."

[0338] This system allows users to efficiently perform inspection work and significantly reduces the manual workload. In addition, by utilizing an emotion engine, it is possible to respond flexibly according to the user's emotional state, providing a seamless user experience.

[0339] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0340] Step 1: User enters or uploads historical data

[0341] Users can input past inspection data into the terminal or upload it in CSV file format. Input information includes the billing date, billing item, amount, and whether or not inspection was possible. Specifically, users can either manually input data using the keyboard or select a CSV file from the file selection dialog and upload it.

[0342] Input: Invoice date, invoice item, amount, and inspection availability data

[0343] Output: Data transfer from device to server

[0344] Step 2: Server receives data and checks format

[0345] The server receives the data sent from the terminal. After receiving it, it performs a format check on the data. The format check is a process to check whether the data is written correctly in JSON format, and invalid data is recorded in a log.

[0346] Input: Data sent from the terminal

[0347] Output: Normalized data after format check, logging of invalid data

[0348] Step 3: Temporarily save the data and store it in the database

[0349] The server saves data that passes the format check in a temporary storage area. It then stores it permanently in a database (such as MySQL). Specific operations include registering the data in the MySQL database with an INSERT statement.

[0350] Input: Normalized data after format check

[0351] Output: Inspection data stored in the database

[0352] Step 4: Data analysis and model training by the server

[0353] The server analyzes the stored past inspection data and uses machine learning algorithms to learn the data's features and patterns. This process involves loading the data into a pandas DataFrame and extracting features. Next, it runs training using scikit-learn's RandomForestClassifier and saves the trained model in pickle format.

[0354] Input: Past inspection data stored in the database

[0355] Output: A trained machine learning model

[0356] Step 5: User enters or uploads new claim data

[0357] The user inputs or uploads new billing data to the terminal in the same way as past data. Input information includes billing date, billing item, amount, etc. Specific operations include manually entering data using the keyboard or uploading a CSV file by selecting it from the file selection dialog.

[0358] Input: New invoice date, invoice item, and amount data

[0359] Output: New data transfer from device to server

[0360] Step 6: Server receives new data and checks format

[0361] The server receives the new billing data and performs another format check to ensure that the data is correctly formatted in JSON, and any invalid data is logged.

[0362] Input: New billing data sent from the terminal

[0363] Output: Normalized data after format check, logging of invalid data

[0364] Step 7: Save the new data to the database

[0365] The server saves the new data that passes the format check to the database, which specifically involves the process of registering the data in the MySQL database with an INSERT statement.

[0366] Input: Normalized data after format check

[0367] Output: New billing data stored in the database

[0368] Step 8: Server analyzes new data and generates acceptance information

[0369] The server analyzes newly received data using the existing learning model and automatically generates inspection information. The new data is input into the trained model and the .predict method of the model is used to predict whether the data can be inspected or not.

[0370] Input: New claims data, trained machine learning model

[0371] Output: Automatically generated inspection information

[0372] Step 9: Save the generated acceptance information and provide it to the user

[0373] The server stores the generated inspection information in a database and provides an interface for users to view it. Specifically, it stores the inspection results in a MySQL database and displays the results on the dashboard of the web application.

[0374] Input: Automatically generated acceptance information

[0375] Output: Inspection information stored in the database, user interface for checking inspection results

[0376] Step 10: Analyze user emotions using the emotion engine and adjust the interface

[0377] The device captures the user's facial expressions and tone of voice using a camera and microphone and sends this data to a server. The server then uses an emotion engine to analyze the user's emotional state and adjusts the system's display accordingly. Specifically, it uses OpenCV for facial recognition, Google Cloud Speech-to-Text for speech analysis, and changes the display of UI components based on the results.

[0378] Input: User's facial expression data, voice data

[0379] Output: Sentiment analysis results, adjusted interface display

[0380] (Application example 2)

[0381] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0382] Conventional inspection work is often done manually, which is time-consuming and labor-intensive, resulting in inefficiency. Therefore, there is a demand for automation of the generation of inspection information. However, to further improve work efficiency and reduce worker stress and confusion, conventional systems lack the functionality to consider the user's emotional state. Therefore, the objective of this invention is to provide a flexible operation interface that takes into account the worker's emotional state while improving the efficiency of inspection work.

[0383] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting or uploading past inspection data, means for analyzing the saved past inspection data and learning data characteristics and patterns, means for inputting or uploading new billing data, means for analyzing the new billing data and generating inspection information based on patterns learned from the past data, means for saving the generated inspection information and providing it to the user, and means for recognizing the user's emotional state and adjusting the interface. This makes it possible to improve the efficiency of inspection work and reduce stress and confusion for the worker by using a flexible operation interface that takes the worker's emotional state into consideration.

[0384] "Past inspection data" refers to data related to inspections, such as previously received billing information and inspection results.

[0385] "Input or Upload Means" refers to an interface or mechanism through which a User manually enters data or uploads data in the form of a file.

[0386] "Means for analyzing stored past inspection data" refers to algorithms or systems that use accumulated past data to recognize and analyze its features and patterns.

[0387] "Means of learning data features and patterns" refers to the process of applying machine learning algorithms to stored data to find criteria and patterns for acceptance or rejection and build a model.

[0388] "Means for analyzing new claims data" refers to algorithms or systems that analyze the latest claims data provided by users and generate acceptance information using models learned from past data.

[0389] "Means for generating inspection information" refers to a system that uses past learning results based on new billing data to determine whether or not inspection is possible, and automatically creates inspection information based on that.

[0390] "Means for storing the generated inspection information and providing it to the user" refers to an interface or mechanism for storing the generated inspection information in a database, etc., and providing that information so that the user can check, modify, and approve it.

[0391] "Means for recognizing the user's emotional state and adjusting the interface" refers to a mechanism that analyzes data such as the user's facial expressions and voice to recognize their emotions, and changes the interface display method and operation guide according to the results.

[0392] "System" refers to a program or device with a set of functions designed to streamline inspection work, including the above means.

[0393] The present invention is a system that improves the efficiency of inspection work at logistics centers and provides a flexible interface that takes into account the emotional state of the user. This system operates in conjunction with the server, terminals, and users.

[0394] First, an interface is provided on the terminal side for users to input or upload past inspection data. Users can input past inspection data manually or upload it in a format such as a CSV file. This data includes the billing date, billing item, amount, and whether or not the data was inspected. The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area. It is then stored in the database.

[0395] Next, the server analyzes the stored past inspection data and uses a machine learning algorithm to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an item can be inspected. For example, it finds patterns that indicate "acceptable" when the amount is small or when certain items are included.

[0396] The user enters or uploads new billing data through a similar interface. This new data also contains the same information as the past data (billing date, billing item, amount, etc.). The server receives this new data, checks the format, and saves it. Next, the server automatically generates inspection information based on the newly received billing data using a model learned from past data. For example, based on past learning results, the server can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[0397] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[0398] Furthermore, the present invention integrates an emotion engine to recognize the user's emotions. The emotion engine analyzes data such as the user's facial expressions and tone of voice to identify the user's emotional state. If the user feels stressed or confused, the emotion engine can adjust the interface display to help the user perform their tasks more smoothly. For example, if the user is in a negative emotional state, the system can display an error message in a gentle tone and provide additional help options. Furthermore, if the user feels anxious, the system can send a notification prompting the user to reconfirm the inspection information.

[0399] As a concrete example, consider the case where a user uploads billing data and its inspection results from the past year to the system. This past data includes the billing date, billing item, amount, and whether or not inspection was possible. Data such as "January 1, 2022, equipment purchase, 10,000 yen, inspection possible" and "February 1, 2022, conference expenses, 20,000 yen, inspection impossible" are input. The server analyzes this data and later learns patterns such as "the smaller the amount, the higher the probability of inspection being possible" and "if conference expenses are expensive, inspection is impossible." When the user enters new billing data "January 1, 2023, equipment purchase, ¥15,000" and "February 1, 2023, conference fee, ¥30,000," the server analyzes this and generates new acceptance information "January 1, 2023, equipment purchase, ¥15,000, accepted" and "February 1, 2023, conference fee, ¥30,000, not accepted."

[0400] Furthermore, if the emotion engine analyzes the user's facial expression data and determines that the user is stressed, the system displays the inspection results in a more user-friendly and understandable format. For example, it provides additional options such as "Do you want to check again?" or "Do you need assistance?", reducing the burden on the user. This system allows users to perform inspection work efficiently and significantly reduces the burden of manual work. In addition, the integration of the emotion engine allows for flexible responses that take into account the user's emotional state, providing excellent flexibility that can be applied even in cases where it is difficult to change the inspection method.

[0401] An example of a prompt could be, "We are creating an application that recognizes user emotions. If a worker feels stressed or confused, how should the system change the interface? For example, by showing options such as 'Do you want to check again?' or 'Do you need help?'"

[0402] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0403] Step 1:

[0404] Users input or upload past inspection data. They can manually enter data using the terminal interface or upload data in a format such as a CSV file. This sends information such as the billing date, billing item, amount, and whether or not the inspection was successful to the server as input data.

[0405] Step 2:

[0406] The server receives past inspection data sent from the terminal and performs a format check. If the format is correct, it is saved in a temporary storage area and then stored in a database. In the format check process, the accuracy of the data format is confirmed and inaccurate data is excluded.

[0407] Step 3:

[0408] The server analyzes past inspection data stored in a database. It uses a machine learning algorithm to learn the characteristics and patterns of the data. For example, as criteria for determining whether an item can be inspected, it can find patterns such as "the smaller the amount, the higher the probability that it can be inspected" or "if certain items are included, it cannot be inspected." The input for this process is the stored data, and the output is a learning model.

[0409] Step 4:

[0410] The user also inputs or uploads new billing data through the terminal interface, which also includes the billing date, billing item, amount, etc. This becomes the input data and is sent to the server.

[0411] Step 5:

[0412] The server receives the new claim data, performs format checks and stores it in the database. Once the format check is complete, the new claim data is stored in the database. This data is used as input for subsequent analysis.

[0413] Step 6:

[0414] The server analyzes newly received billing data and automatically generates acceptance information using a machine learning model learned from past data. For example, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be accepted for inspection." The input data is the new billing data, and the output is the generated acceptance information.

[0415] Step 7:

[0416] The server stores the generated inspection information in a database and then provides the information via a terminal interface for the user to review. The user can review the inspection results and make corrections or approvals as necessary. The input is the generated inspection information, and the output is the information provided to the user.

[0417] Step 8:

[0418] The emotion engine recognizes the user's emotional state in real time by analyzing the user's facial expressions and tone of voice using the smartphone's camera and microphone. The input data is camera footage and audio data, and the output is the identified emotional state.

[0419] Step 9:

[0420] The server adjusts the interface based on the emotional state recognized by the emotion engine. For example, if the user is determined to be stressed, the server displays a more user-friendly error message and offers additional help options. Alternatively, if the user is anxious, the server sends a notification prompting the user to reconfirm the acceptance information. The input is the identified emotional state, and the output is the adjusted interface display.

[0421] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0422] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0423] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0424] [Second embodiment]

[0425] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0426] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0427] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0428] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0429] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0430] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0431] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0432] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0433] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0434] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0435] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0436] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0437] This invention relates to a system that improves the efficiency of inspection work and solves the problem of conventional manual generation of inspection information. This system operates in conjunction with the server, terminals, and users to automate the process from inputting inspection data to analyzing it, generating inspection information, and outputting the results.

[0438] System configuration

[0439] First, the terminal provides an interface for users to input or upload past data. Users can manually input past inspection data or upload it in a CSV file format. This data includes the invoice date, invoice item, amount, and whether or not the data was accepted.

[0440] The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area, after which it stores it in a database.

[0441] Analysis and learning from past data

[0442] The server analyzes the stored past inspection data and uses a machine learning algorithm to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an item can be inspected. For example, if the amount is small or if certain items are included, it will find patterns that indicate "acceptable inspection."

[0443] Receiving and analyzing new claims data

[0444] Users enter or upload new billing data through a similar interface. The new data contains the same information as the previous data (billing date, billing item, amount, etc.). The server receives this new data, checks its format, and stores it.

[0445] Generating acceptance information

[0446] The server automatically generates inspection information based on newly received billing data using a model learned from past data. For example, based on past learning results, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[0447] Saving and providing inspection results

[0448] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[0449] Specific examples

[0450] For example, consider the case where a user uploads billing data and its inspection results from the past year to the system. This past data includes the billing date, billing item, amount, and whether or not inspection was possible. Data such as "January 1, 2022, Equipment Purchase, ¥10,000, Inspection Possible" and "February 1, 2022, Conference Expenses, ¥20,000, Inspection Not Possible" are entered.

[0451] The server analyzes this data and later learns patterns such as "the smaller the amount, the higher the probability of acceptance" and "if the conference fee is high, it will not be accepted."

[0452] When the user enters new billing data "January 1, 2023, equipment purchase, ¥15,000" and "February 1, 2023, conference fee, ¥30,000," the server analyzes this and generates new acceptance information "January 1, 2023, equipment purchase, ¥15,000, accepted" and "February 1, 2023, conference fee, ¥30,000, not accepted."

[0453] This system allows users to efficiently carry out inspection work and significantly reduces the burden of manual work. In addition, even when it is difficult to change the inspection method, the system can be used to flexibly respond.

[0454] The processing flow will be explained below.

[0455] Step 1:

[0456] User enters or uploads historical data

[0457] Users can manually enter past inspection data through the system interface or upload it as a CSV file.

[0458] Specifically, the user clicks the "Select File" button to select the CSV file.

[0459] Step 2:

[0460] The server receives and stores the data

[0461] The server receives the past data sent by the user and checks the format and content of the data.

[0462] If the format is correct, the data is stored in a temporary storage area.

[0463] Step 3:

[0464] The server stores the data in a database

[0465] The server stores the data in the database after the format check is complete.

[0466] At this time, checks are also carried out to prevent errors such as data duplication or loss.

[0467] Step 4:

[0468] The server analyzes past data

[0469] The server analyzes past inspection data stored in a database and extracts various categories of data.

[0470] For example, classify data with tags such as "acceptable" and "unacceptable."

[0471] Step 5:

[0472] The server uses a machine learning model to learn the characteristics of the data.

[0473] The server uses machine learning algorithms to learn patterns and characteristics of inspections.

[0474] For example, patterns that determine whether or not a product can be accepted are searched for and saved as a model.

[0475] Step 6:

[0476] A user enters or uploads new claim data

[0477] Users enter new claim data through the system interface or upload it as a CSV file.

[0478] Step 7:

[0479] The server receives the new data and checks the format.

[0480] The server receives the newly submitted billing data and verifies that the data format is correct.

[0481] Step 8:

[0482] The server saves the new data

[0483] The server stores the data in a database after format checking is complete.

[0484] Step 9:

[0485] The server parses the new billing data

[0486] The server retrieves the new billing data and parses the information.

[0487] For example, extract billing items and amounts.

[0488] Step 10:

[0489] The server generates inspection information using the learning model.

[0490] The server uses the trained model to automatically generate an acceptance or rejection decision for new billing data.

[0491] For example, if the purchase of equipment on January 1, 2023 costs 15,000 yen, it is determined that the equipment can be inspected.

[0492] Step 11:

[0493] The server stores the generated inspection results and provides them to the user.

[0494] The server stores the generated inspection information in a database.

[0495] At the same time, the inspection results are displayed to the user on the system interface.

[0496] Step 12:

[0497] The user checks, corrects, and approves the inspection results

[0498] The user checks the inspection results through the system interface, makes corrections as necessary, and then approves them.

[0499] Example 1

[0500] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0501] Conventional inspection work is performed manually, which makes it inefficient and prone to human error. It is also difficult to analyze past inspection data and use it to make future inspection decisions. This makes the criteria for inspection decisions unclear, and increases the likelihood of inconsistent results. To solve this problem, a system is needed that streamlines inspection work and automatically generates inspection information using past data.

[0502] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0503] In this invention, the server includes means for receiving past inspection data, performing a format check, and saving it in a temporary storage area, means for analyzing the saved past inspection data and storing it in a database, means for analyzing the saved past inspection data and learning data characteristics and patterns using a machine learning algorithm, means for receiving new billing data, performing a format check, and saving it in a database, means for analyzing the new billing data and generating inspection information based on patterns learned from the past data, and means for saving the generated inspection information in a database and providing it to a user so that it can be checked via a terminal. This makes it possible to improve the efficiency and accuracy of inspection work.

[0504] "Inspection data" refers to data that includes information related to the inspection work, such as the invoice date, invoice item, amount, and whether or not the inspection is possible.

[0505] A "server" is a computer system used for the purposes of processing, storing, analyzing received data, and generating necessary information.

[0506] A "terminal" is a device that allows a user to input data or view system results. This includes computers, smartphones, tablets, etc.

[0507] "User" means a person or organization that uses the system to enter or upload acceptance data and verify the generated acceptance information.

[0508] "Format check" is the process of verifying that the received data conforms to the expected format and structure.

[0509] The "temporary storage area" is a storage space for temporarily storing data before it is finally stored in the database.

[0510] A "database" is a system that stores data in an organized, searchable, updateable, and manageable manner. This includes relational databases and NoSQL databases.

[0511] A "machine learning algorithm" is an algorithm that automatically learns patterns and features from data and makes predictions and classifications for new data. Examples include decision trees and neural networks.

[0512] "Analysis" is the process of examining data in detail to find useful information and patterns within it.

[0513] "Inspection information" is information that includes the results of a judgment, such as whether or not a certain request can be inspected, generated based on past data and machine learning algorithms.

[0514] "Automatic generation" is a function in which the system automatically generates data without manual human intervention.

[0515] "Learning" is the process by which a machine learning model uses past data to understand patterns and features and apply them to future data.

[0516] A "prompt sentence" is a guide message that is displayed to prompt the user to take a specific action.

[0517] This invention relates to a system for streamlining inspection work and solving the problem of conventional manual generation of inspection information. This system involves the collaboration of a server, terminals, and users to automate the entire process from inputting inspection data to analyzing it, generating inspection information, and outputting the results.

[0518] System configuration

[0519] First, the user is provided with an interface on the terminal to input or upload past inspection data. The user can manually input the past inspection data or upload it in a format such as a CSV file. This data includes the invoice date, invoice item, amount, whether or not the inspection was successful, etc.

[0520] For example, a user enters "January 1, 2022, Equipment Purchase, 10,000 yen, Accepted for Inspection" into an input form in a web application, or selects and uploads a CSV file.

[0521] Receiving and storing data

[0522] The server receives the data sent from the terminal. After receiving it, it performs a format check and eliminates any inappropriate data. Next, it saves the data in a temporary storage area and then stores it in a database. For format checks and database operations, it uses, for example, the Python pandas library or MySQL.

[0523] Analysis and learning from past data

[0524] The server analyzes the stored past inspection data. For the analysis, it uses machine learning libraries such as Python's scikit-learn and TensorFlow to learn the characteristics and patterns of the data. This allows it to build criteria for determining whether or not an inspection is acceptable. Specifically, it trains a model using scikit-learn's DecisionTreeClassifier.

[0525] For example, it learns patterns such as "the smaller the amount, the higher the probability that inspection is possible" and "if the conference fee is high, inspection is impossible."

[0526] Entering New Claim Data

[0527] Users enter or upload new billing data using the web application interface, just like with past data. New data includes billing date, billing item, and amount.

[0528] As an example, let's assume a process in which a user enters "January 1, 2023, Equipment Purchase, ¥15,000" in the data entry field and uploads it.

[0529] Analyzing New Data

[0530] The server receives the newly received billing data, checks the format, then stores the data and analyzes it using a machine learning model learned from past data.

[0531] Generating acceptance information

[0532] The server automatically generates acceptance information based on the analysis results. Specifically, it uses a trained machine learning model (for example, using the predict method in scikit-learn) to generate prediction results. The prediction results are provided in the form of "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be accepted for inspection."

[0533] Saving and providing inspection results

[0534] The generated inspection information is stored in a database on the server, and users can check these inspection results through the terminal interface. If necessary, users can correct or approve the results.

[0535] Prompt statement

[0536] As an example of a prompt sentence, the message to prompt the user to enter past billing data is shown below.

[0537] Please upload your past billing data. The data must include the billing date, billing item, amount, and whether or not it can be inspected. For example, please enter it in the following format: "January 1, 2022, Equipment Purchase, 10,000 yen, Acceptable."

[0538] This system makes it possible to improve the efficiency and accuracy of inspection work. Users can perform inspection work efficiently, significantly reducing the burden of manual work. It also allows flexible inspection decisions to be made based on past data.

[0539] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0540] Step 1: Data entry

[0541] Users input or upload past inspection data. Using an interface on their device (e.g., a web application), they select and upload a CSV file, etc. The input data includes the billing date, billing item, amount, and whether or not the data was accepted.

[0542] Input: Past inspection data (e.g., "January 1, 2022, Equipment Purchase, ¥10,000, Accepted")

[0543] Output: Uploaded CSV file or input form data

[0544] Specific operation: The user selects a CSV file and clicks the upload button, or manually enters past billing data into the data entry field.

[0545] Step 2: Receiving and storing data

[0546] The server receives the data sent from the terminal. It checks the format of the received data and eliminates any inappropriate data. It then saves the data in a temporary storage area and stores the correct data in a database. Specifically, it uses the Python pandas library to read the CSV file and MySQL as the database.

[0547] Input: Past inspection data uploaded by the user

[0548] Output: Correctly formatted acceptance data stored in the database

[0549] Specific operation: Read a CSV file using pandas, validate the data format, and insert the validated data into a MySQL database.

[0550] Step 3: Analyze and learn from past data

[0551] The server analyzes the stored past inspection data. It uses machine learning libraries such as Python's scikit-learn and TensorFlow to learn the features and patterns of the data. This allows it to build criteria for determining whether or not an inspection is acceptable. Specifically, it trains a model using scikit-learn's DecisionTreeClassifier.

[0552] Input: Past inspection data stored in the database

[0553] Output: A trained machine learning model

[0554] Specific operation: A decision tree model is trained using scikit-learn to learn the criteria for determining whether or not a product can be accepted.

[0555] Step 4: Enter new claim data

[0556] Users enter or upload new billing data using the web application interface, just like with past data. New data includes billing date, billing item, and amount.

[0557] Input: New billing data (e.g., "January 1, 2023, Equipment Purchase, ¥15,000")

[0558] Output: New billing data uploaded

[0559] Specific operation: The user enters "January 1, 2023, Equipment Purchase, ¥15,000" in the data input field and clicks the upload button.

[0560] Step 5: Analyze the new claims data

[0561] The server receives newly received billing data, performs format checks, stores valid data in a database, and analyzes new data using machine learning models learned from past data.

[0562] Input: New claim data

[0563] Output: New, correctly formatted billing data, analysis results

[0564] Specific operations: Read new billing data using pandas, check the format, and save it to the database. Analyze the new data using scikit-learn's predict method.

[0565] Step 6: Generate acceptance information

[0566] The server automatically generates inspection information based on the analysis results. It generates predictions using a machine learning model (for example, using the predict method in scikit-learn) that has learned from past data.

[0567] Input: Parsed new claims data

[0568] Output: Generated acceptance information (e.g., "January 1, 2023, Equipment Purchase, ¥15,000, Accepted")

[0569] Specific operation: Automatically generate inspection information using a trained machine learning model.

[0570] Step 7: Save and provide inspection results

[0571] The server stores the generated inspection information in a database. Users can check these inspection results through the terminal interface and make corrections or approvals as necessary.

[0572] Input: Generated acceptance information

[0573] Output: Inspection information stored in the database, inspection results provided to the user

[0574] Specific operation: The inspection results are inserted into the database and displayed in the web interface.

[0575] (Application example 1)

[0576] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0577] Inspection work at logistics centers involves a lot of manual work, which takes time and effort. In addition, inspection standards vary from person to person, which can lead to a lack of consistency in inspection results. Furthermore, in many cases, past inspection data cannot be fully utilized, which makes it difficult to make efficient inspection decisions.

[0578] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0579] In this invention, the server includes means for inputting or uploading past inspection data, means for analyzing the saved past inspection data and learning data characteristics and patterns, means for inputting or uploading new billing data, means for analyzing the new billing data and generating inspection information based on patterns learned from the past data, means for saving the generated inspection information and providing it to users, and means for using a robot in a logistics facility to scan specified products and generate inspection information. This makes inspection work more efficient and enables inspections to be performed according to consistent standards.

[0580] "Past inspection data" is information related to inspection work that has been carried out in the past, and includes detailed data such as the date of invoice, the item of invoice, the amount, and whether or not the item was inspected.

[0581] "Means for input or upload" refers to the functionality of the interface for a user to manually enter data or provide an existing data file to the system.

[0582] "Means of analyzing stored past inspection data and learning data features and patterns" refers to the functionality of machine learning algorithms that analyze past inspection data, learn specific patterns and features, and build criteria for judging future data.

[0583] "New Claims Data" means data containing information about new claims that are currently or will be made in the future.

[0584] "Means for entering or uploading new claim data" refers to the functionality of the interface that allows a user to provide new claim data to the system.

[0585] "Means for analyzing new billing data and generating inspection information based on patterns learned from past data" refers to the function of analyzing new billing data, determining whether or not to inspect it based on patterns learned in the past, and generating that information.

[0586] "Means for saving the generated inspection information and providing it to the user" refers to the interface function for saving the inspection results generated by the system and providing the information so that the user can check it.

[0587] A "logistics facility" is a facility used for logistics operations such as receiving, storing, inspecting, and shipping goods.

[0588] "Robots that scan specified products and generate inspection information" refer to autonomous robots that scan barcodes, QR codes, etc. of specified products within a logistics facility and automatically generate inspection information based on that information.

[0589] This invention provides a system for improving the efficiency of inspection work at logistics facilities. The specific configuration and operation of this system will be described below.

[0590] First, the user uses an interface on the terminal to input or upload past inspection data. The user can input the past inspection data manually or upload it as a CSV file. This past inspection data includes information such as the invoice date, invoice item, amount, and whether or not the inspection was successful.

[0591] The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area.Then, it stores this data in a database.

[0592] The server then analyzes the stored past inspection data and uses machine learning algorithms to learn the characteristics and patterns of the data. This analysis and learning is performed using machine learning frameworks such as TensorFlow. Through this learning, criteria for determining whether or not an item can be inspected are built within the server. For example, patterns that indicate "acceptance" are found when "the amount is small" or "specific items are included."

[0593] The user then enters or uploads new billing data through a similar interface. This new data contains the same information as the previous data (billing date, billing item, amount, etc.). The server receives this new data, checks its format, and stores it.

[0594] The server automatically generates inspection information based on newly received billing data using a model learned from past data. For example, based on past learning results, the server can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[0595] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[0596] Furthermore, this system includes a robot that scans designated products at logistics facilities and generates inspection information. The robot scans the products within the logistics facility and sends the data to a server. The server analyzes the data and uses a model trained on past data to determine whether or not the product can be inspected. The generated inspection information is displayed on the robot's display, allowing users to easily check it.

[0597] As a concrete example, consider the case where a user uploads billing data from the past year to the system. This data includes information such as "January 1, 2022, Equipment Purchase, ¥10,000, Acceptable" and "February 1, 2022, Meeting Expenses, ¥20,000, Unacceptable." The server analyzes this data and learns patterns. Subsequently, when new billing data "January 1, 2023, Equipment Purchase, ¥15,000" and "February 1, 2023, Meeting Expenses, ¥30,000" are received, the server generates acceptance information for "January 1, 2023, Equipment Purchase, ¥15,000, Acceptable" and "February 1, 2023, Meeting Expenses, ¥30,000, Unacceptable."

[0598] An example of a prompt is, "Based on the inspection data from the past year, determine whether or not new products can be inspected and record this automatically." In this way, it is possible to improve the efficiency of inspection work at logistics facilities and provide consistent inspection standards.

[0599] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0600] Step 1:

[0601] The user inputs or uploads past inspection data.

[0602] Specifically, users can manually enter past billing data (billing date, billing item, amount, acceptance / acceptance status, etc.) through the terminal interface, or upload a CSV file. The entered data is checked for formatting on the terminal and then sent to the server.

[0603] Input: Past inspection data

[0604] Output: Format-checked past inspection data is sent to the server

[0605] Step 2:

[0606] The server receives and stores past inspection data.

[0607] The server receives the data sent from the terminal, checks the format again, and saves the data in a temporary storage area.Then, it stores this data in a database.

[0608] Input: Format-checked past inspection data

[0609] Output: Past inspection data stored in a database

[0610] Step 3:

[0611] The server analyzes the stored past inspection data and learns the characteristics and patterns of the data.

[0612] The server analyzes past inspection data stored in a database using machine learning frameworks such as TensorFlow to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an inspection is acceptable.

[0613] Input: Past inspection data

[0614] Output: A trained machine learning model

[0615] Step 4:

[0616] A user enters or uploads new claim data.

[0617] Users can manually enter new billing data (billing date, billing item, amount, etc.) through their terminal or upload it as a CSV file, which is also format-checked on the terminal and then sent to the server.

[0618] Input: New claim data

[0619] Output: New format-checked claim data is sent to the server

[0620] Step 5:

[0621] The server receives the new billing data and generates acceptance information.

[0622] The server analyzes newly received billing data using a machine learning model and generates acceptance information based on patterns learned from past data. For example, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be accepted for inspection."

[0623] Input: New claims data, trained machine learning model

[0624] Output: Generated acceptance information

[0625] Step 6:

[0626] The server stores the generated acceptance information and provides it to the user.

[0627] The generated inspection information is stored in the server's database. An interface is then provided so that users can check the inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[0628] Input: Generated acceptance information

[0629] Output: Inspection information stored in the database, interface provided to the user

[0630] Step 7:

[0631] The robot scans the specified product and generates inspection information.

[0632] Robots used in logistics facilities scan the barcodes or QR codes of designated products and send the data to a server. The server analyzes the data and uses machine learning models to determine whether or not the product should be inspected. The inspection results are then displayed on the robot's display.

[0633] Input: Scanned product data

[0634] Output: Inspection information displayed on the robot's display

[0635] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0636] This invention is a system that streamlines inspection work and solves the problem of conventional manual generation of inspection information, and also combines an "emotion engine" that recognizes user emotions and reflects them in the operation of the system. This system operates in conjunction with the server, terminal, and user, automating the entire process from inputting inspection data to analyzing it, generating inspection information, outputting the results, and recognizing user emotions.

[0637] System configuration

[0638] First, the terminal provides an interface for users to input or upload past data. Users can manually input past inspection data or upload it in a CSV file format. This data includes the invoice date, invoice item, amount, and whether or not the data was accepted.

[0639] The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area, after which it stores it in a database.

[0640] Analysis and learning from past data

[0641] The server analyzes the stored past inspection data and uses a machine learning algorithm to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an item can be inspected. For example, if the amount is small or if certain items are included, it will find patterns that indicate "acceptable inspection."

[0642] Receiving and analyzing new claims data

[0643] Users enter or upload new billing data through a similar interface. The new data contains the same information as the previous data (billing date, billing item, amount, etc.). The server receives this new data, checks its format, and stores it.

[0644] Generating acceptance information

[0645] The server automatically generates inspection information based on newly received billing data using a model learned from past data. For example, based on past learning results, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[0646] Saving and providing inspection results

[0647] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[0648] Emotion engine integration

[0649] The present invention further integrates an emotion engine to recognize the user's emotions, which analyzes data such as the user's facial expressions and tone of voice to identify the user's emotional state.

[0650] The emotion engine allows the system to adjust the interface display when a user is stressed or confused, helping them to complete their tasks more smoothly. For example, if a user is in a negative emotional state, the system can display error messages in a gentle tone and provide additional help options. Additionally, if a user feels anxious, the system can send a notification prompting them to reconfirm the inspection information.

[0651] Specific examples

[0652] For example, consider the case where a user uploads billing data and its inspection results from the past year to the system. This past data includes the billing date, billing item, amount, and whether or not inspection was possible. Data such as "January 1, 2022, Equipment Purchase, ¥10,000, Inspection Possible" and "February 1, 2022, Conference Expenses, ¥20,000, Inspection Not Possible" are entered.

[0653] The server analyzes this data and later learns patterns such as "the smaller the amount, the higher the probability of acceptance" and "if the conference fee is high, it will not be accepted."

[0654] When the user enters new billing data "January 1, 2023, equipment purchase, ¥15,000" and "February 1, 2023, conference fee, ¥30,000," the server analyzes this and generates new acceptance information "January 1, 2023, equipment purchase, ¥15,000, accepted" and "February 1, 2023, conference fee, ¥30,000, not accepted."

[0655] Furthermore, if the emotion engine analyzes the user's facial expression data and determines that the user is feeling stressed, the system will display the inspection results in a more friendly and understandable format, providing additional options such as "Do you want to check again?" or "Do you need help?", reducing the burden on the user.

[0656] This system allows users to efficiently perform inspection work and significantly reduces the burden of manual work. In addition, the integration of an emotion engine allows for flexible responses that take into account the user's emotional state, providing excellent flexibility that can be applied even in cases where it is difficult to change the inspection method.

[0657] The processing flow will be explained below.

[0658] Step 1:

[0659] User enters or uploads historical data

[0660] Users can manually enter past inspection data through the system interface or upload it as a CSV file.

[0661] Specifically, the user clicks the "Select File" button to select the CSV file.

[0662] Step 2:

[0663] The server receives and stores the data

[0664] The server receives the past data sent by the user and checks the format and content of the data.

[0665] If the format is correct, the data is stored in a temporary storage area.

[0666] Step 3:

[0667] The server stores the data in a database

[0668] The server stores the data in the database after the format check is complete.

[0669] At this time, checks are also carried out to prevent errors such as data duplication or loss.

[0670] Step 4:

[0671] The server analyzes past data

[0672] The server analyzes past inspection data stored in a database and extracts various categories of data.

[0673] For example, classify data with tags such as "acceptable" and "unacceptable."

[0674] Step 5:

[0675] The server uses a machine learning model to learn the characteristics of the data.

[0676] The server uses machine learning algorithms to learn patterns and characteristics of inspections.

[0677] For example, patterns that determine whether or not a product can be accepted are searched for and saved as a model.

[0678] Step 6:

[0679] A user enters or uploads new claim data

[0680] Users enter new claim data through the system interface or upload it as a CSV file.

[0681] Step 7:

[0682] The server receives the new data and checks the format.

[0683] The server receives the newly submitted billing data and verifies that the data format is correct.

[0684] Step 8:

[0685] The server saves the new data

[0686] The server stores the data in a database after format checking is complete.

[0687] Step 9:

[0688] The server parses the new billing data

[0689] The server retrieves the new billing data and parses the information.

[0690] For example, extract billing items and amounts.

[0691] Step 10:

[0692] The server generates inspection information using the learning model.

[0693] The server uses the trained model to automatically generate an acceptance or rejection decision for new billing data.

[0694] For example, if the purchase of equipment on January 1, 2023 costs 15,000 yen, it is determined that the equipment can be inspected.

[0695] Step 11:

[0696] The server stores the generated inspection results and provides them to the user.

[0697] The server stores the generated inspection information in a database.

[0698] At the same time, the inspection results are displayed to the user on the system interface.

[0699] Step 12:

[0700] Emotion engine recognizes user emotions

[0701] The emotion engine analyzes facial expressions and tone of voice through the user's camera and microphone.

[0702] This identifies the user's emotional state, such as whether they are stressed or confused.

[0703] Step 13:

[0704] The server adjusts the interface based on the emotional state.

[0705] The server adjusts the way the system interface is displayed based on the user's emotional state obtained from the emotion engine.

[0706] For example, if a user is stressed, provide a more friendly and understandable display or additional help options.

[0707] Step 14:

[0708] The user checks, corrects, and approves the inspection results

[0709] The user checks the inspection results through the system interface, makes corrections as necessary, and then approves them.

[0710] The emotion engine will display a notification to encourage reconsideration if the user is feeling anxious.

[0711] Example 2

[0712] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0713] In conventional inspection work, inspection information is generated manually, which requires a lot of time and effort. In addition, because the user's emotional state is not taken into consideration, users often feel stressed when using the system. This reduces work efficiency and increases the risk of mistakes. To solve these problems, a system that automatically generates inspection information and takes the user's emotional state into consideration is needed.

[0714] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0715] In this invention, the server includes a means for inputting or uploading past inspection data, a means for analyzing the stored past inspection data and learning data characteristics and patterns, and a means for inputting or uploading new billing data, which enables automatic generation of inspection information and flexible system display that takes into account the user's emotional state.

[0716] "Past inspection data" is information relating to inspection work carried out in the past, including the date of invoice, invoice item, amount, whether or not inspection was possible, and the like.

[0717] "Input or upload means" means an interface through which a user can manually enter data or upload it to the system in the form of a file.

[0718] The "means for analyzing the stored past inspection data" refers to an algorithm or program for analyzing the past inspection data received by the server and extracting characteristics and patterns of the data.

[0719] "Methods of learning features and patterns in data" is the process of using machine learning algorithms to automatically learn useful information and trends from past data.

[0720] "New billing data" refers to information about a bill that is newly entered or uploaded into the system, including the billing date, billing items, amount, etc.

[0721] The "means for generating inspection information" refers to an algorithm or program that uses learned patterns or models to automatically generate information such as whether or not an item can be inspected from new invoice data.

[0722] The "means for storing the generated acceptance information" is a process in which the server stores the automatically generated acceptance information in a persistent storage device such as a database.

[0723] The "means for providing to the user" is an interface that allows the user to check, correct, and approve the stored acceptance information.

[0724] "Means for recognizing the user's emotions and adjusting the system's behavior" refers to algorithms or programs that analyze the user's facial expressions, tone of voice, etc., and change the system's display method and behavior based on the results.

[0725] This system improves the efficiency of inspection work and adjusts the operation of the system by recognizing the user's emotions. A specific embodiment of this system will be described below.

[0726] First, the user inputs or uploads past inspection data into the terminal. Specifically, past billing data (billing date, billing item, amount, whether inspection was possible, etc.) can be manually input or uploaded in CSV file format. This data is sent to the server through the terminal interface.

[0727] The server receives the data sent from the device and performs a format check. Format check is a process to ensure the data is correct, and invalid data is logged. The data is then saved in a temporary storage area and then permanently stored in a database (e.g., MySQL).

[0728] Next, the server analyzes the stored past inspection data and learns the data's features and patterns. This process uses machine learning algorithms (such as scikit-learn or TensorFlow). The data is loaded into a pandas DataFrame, features are extracted, and the data is split into training datasets. The data is trained using scikit-learn's RandomForestClassifier, and the generated model is saved in pickle format.

[0729] Next, the user enters or uploads new billing data, just like the previous data. The new data includes the billing date, billing item, amount, etc. This is also sent to the server via the terminal.

[0730] The server receives the new data, checks the format again, and saves it to the database. It then uses the saved learning model to generate acceptance information from the new data. Specifically, it inputs the new data into the trained model and uses the model's .predict method to predict whether the data will be accepted or not. The generated acceptance information is saved in the database.

[0731] The generated inspection information is provided to the user as the inspection result. The terminal provides an interface that allows the user to check the inspection result and make corrections or approvals. The user checks the results and makes corrections or approvals as necessary.

[0732] The system also incorporates an "emotion engine" to recognize the user's emotions. The device captures the user's facial expressions and tone of voice using a camera and microphone and sends this data to a server. The server then uses the emotion engine to analyze the user's emotional state and adjusts the system's display accordingly. For example, facial expression recognition is performed using OpenCV and voice analysis is performed using Google Cloud Speech-to-Text. If the system determines that the user is stressed, it displays additional options such as "Would you like to check again?" or "Do you need support?"

[0733] Specific examples

[0734] Consider the case where a user uploads billing data from the past year to the system. The past data includes information such as "January 1, 2022, equipment purchase, 10,000 yen, accepted" and "February 1, 2022, conference fee, 20,000 yen, not accepted." The server analyzes this data and learns patterns such as "the smaller the amount, the higher the probability of acceptance" and "if the conference fee is high, it is not accepted."

[0735] When new billing data "January 1, 2023, Equipment Purchase, ¥15,000" and "February 1, 2023, Conference Expenses, ¥30,000" are entered, the server analyzes this and generates new acceptance information "January 1, 2023, Equipment Purchase, ¥15,000, Accepted" and "February 1, 2023, Conference Expenses, ¥30,000, Not Accepted."

[0736] To achieve this, the following prompts are given to the generative AI model:

[0737] "Analyze the following historical data and generate a prompt to predict whether the new billing data will be accepted: 'January 1, 2022, Equipment Purchase, ¥10,000, Acceptable'; 'February 1, 2022, Conference Expenses, ¥20,000, Unacceptable'. The new billing data is 'January 1, 2023, Equipment Purchase, ¥15,000'."

[0738] This system allows users to efficiently perform inspection work and significantly reduces the manual workload. In addition, by utilizing an emotion engine, it is possible to respond flexibly according to the user's emotional state, providing a seamless user experience.

[0739] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0740] Step 1: User enters or uploads historical data

[0741] Users can input past inspection data into the terminal or upload it in CSV file format. Input information includes the billing date, billing item, amount, and whether or not inspection was possible. Specifically, users can either manually input data using the keyboard or select a CSV file from the file selection dialog and upload it.

[0742] Input: Invoice date, invoice item, amount, and inspection availability data

[0743] Output: Data transfer from device to server

[0744] Step 2: Server receives data and checks format

[0745] The server receives the data sent from the terminal. After receiving it, it performs a format check on the data. The format check is a process to check whether the data is written correctly in JSON format, and invalid data is recorded in a log.

[0746] Input: Data sent from the terminal

[0747] Output: Normalized data after format check, logging of invalid data

[0748] Step 3: Temporarily save the data and store it in the database

[0749] The server saves data that passes the format check in a temporary storage area. It then stores it permanently in a database (such as MySQL). Specific operations include registering the data in the MySQL database with an INSERT statement.

[0750] Input: Normalized data after format check

[0751] Output: Inspection data stored in the database

[0752] Step 4: Data analysis and model training by the server

[0753] The server analyzes the stored past inspection data and uses machine learning algorithms to learn the data's features and patterns. This process involves loading the data into a pandas DataFrame and extracting features. Next, it runs training using scikit-learn's RandomForestClassifier and saves the trained model in pickle format.

[0754] Input: Past inspection data stored in the database

[0755] Output: A trained machine learning model

[0756] Step 5: User enters or uploads new claim data

[0757] The user inputs or uploads new billing data to the terminal in the same way as past data. Input information includes billing date, billing item, amount, etc. Specific operations include manually entering data using the keyboard or uploading a CSV file by selecting it from the file selection dialog.

[0758] Input: New invoice date, invoice item, and amount data

[0759] Output: New data transfer from device to server

[0760] Step 6: Server receives new data and checks format

[0761] The server receives the new billing data and performs another format check to ensure that the data is correctly formatted in JSON, and any invalid data is logged.

[0762] Input: New billing data sent from the terminal

[0763] Output: Normalized data after format check, logging of invalid data

[0764] Step 7: Save the new data to the database

[0765] The server saves the new data that passes the format check to the database, which specifically involves the process of registering the data in the MySQL database with an INSERT statement.

[0766] Input: Normalized data after format check

[0767] Output: New billing data stored in the database

[0768] Step 8: Server analyzes new data and generates acceptance information

[0769] The server analyzes newly received data using the existing learning model and automatically generates inspection information. The new data is input into the trained model and the .predict method of the model is used to predict whether the data can be inspected or not.

[0770] Input: New claims data, trained machine learning model

[0771] Output: Automatically generated inspection information

[0772] Step 9: Save the generated acceptance information and provide it to the user

[0773] The server stores the generated inspection information in a database and provides an interface for users to view it. Specifically, it stores the inspection results in a MySQL database and displays the results on the dashboard of the web application.

[0774] Input: Automatically generated acceptance information

[0775] Output: Inspection information stored in the database, user interface for checking inspection results

[0776] Step 10: Analyze user emotions using the emotion engine and adjust the interface

[0777] The device captures the user's facial expressions and tone of voice using a camera and microphone and sends this data to a server. The server then uses an emotion engine to analyze the user's emotional state and adjusts the system's display accordingly. Specifically, it uses OpenCV for facial recognition, Google Cloud Speech-to-Text for speech analysis, and changes the display of UI components based on the results.

[0778] Input: User's facial expression data, voice data

[0779] Output: Sentiment analysis results, adjusted interface display

[0780] (Application example 2)

[0781] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0782] Conventional inspection work is often done manually, which is time-consuming and labor-intensive, resulting in inefficiency. Therefore, there is a demand for automation of the generation of inspection information. However, to further improve work efficiency and reduce worker stress and confusion, conventional systems lack the functionality to consider the user's emotional state. Therefore, the objective of this invention is to provide a flexible operation interface that takes into account the worker's emotional state while improving the efficiency of inspection work.

[0783] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting or uploading past inspection data, means for analyzing the saved past inspection data and learning data characteristics and patterns, means for inputting or uploading new billing data, means for analyzing the new billing data and generating inspection information based on patterns learned from the past data, means for saving the generated inspection information and providing it to the user, and means for recognizing the user's emotional state and adjusting the interface. This makes it possible to improve the efficiency of inspection work and reduce stress and confusion for the worker by using a flexible operation interface that takes the worker's emotional state into consideration.

[0784] "Past inspection data" refers to data related to inspections, such as previously received billing information and inspection results.

[0785] "Input or Upload Means" refers to an interface or mechanism through which a User manually enters data or uploads data in the form of a file.

[0786] "Means for analyzing stored past inspection data" refers to algorithms or systems that use accumulated past data to recognize and analyze its features and patterns.

[0787] "Means of learning data features and patterns" refers to the process of applying machine learning algorithms to stored data to find criteria and patterns for acceptance or rejection and build a model.

[0788] "Means for analyzing new claims data" refers to algorithms or systems that analyze the latest claims data provided by users and generate acceptance information using models learned from past data.

[0789] "Means for generating inspection information" refers to a system that uses past learning results based on new billing data to determine whether or not inspection is possible, and automatically creates inspection information based on that.

[0790] "Means for storing the generated inspection information and providing it to the user" refers to an interface or mechanism for storing the generated inspection information in a database, etc., and providing that information so that the user can check, modify, and approve it.

[0791] "Means for recognizing the user's emotional state and adjusting the interface" refers to a mechanism that analyzes data such as the user's facial expressions and voice to recognize their emotions, and changes the interface display method and operation guide according to the results.

[0792] "System" refers to a program or device with a set of functions designed to streamline inspection work, including the above means.

[0793] The present invention is a system that improves the efficiency of inspection work at logistics centers and provides a flexible interface that takes into account the emotional state of the user. This system operates in conjunction with the server, terminals, and users.

[0794] First, an interface is provided on the terminal side for users to input or upload past inspection data. Users can input past inspection data manually or upload it in a format such as a CSV file. This data includes the billing date, billing item, amount, and whether or not the data was inspected. The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area. It is then stored in the database.

[0795] Next, the server analyzes the stored past inspection data and uses a machine learning algorithm to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an item can be inspected. For example, it finds patterns that indicate "acceptable" when the amount is small or when certain items are included.

[0796] The user enters or uploads new billing data through a similar interface. This new data also contains the same information as the past data (billing date, billing item, amount, etc.). The server receives this new data, checks the format, and saves it. Next, the server automatically generates inspection information based on the newly received billing data using a model learned from past data. For example, based on past learning results, the server can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[0797] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[0798] Furthermore, the present invention integrates an emotion engine to recognize the user's emotions. The emotion engine analyzes data such as the user's facial expressions and tone of voice to identify the user's emotional state. If the user feels stressed or confused, the emotion engine can adjust the interface display to help the user perform their tasks more smoothly. For example, if the user is in a negative emotional state, the system can display an error message in a gentle tone and provide additional help options. Furthermore, if the user feels anxious, the system can send a notification prompting the user to reconfirm the inspection information.

[0799] As a concrete example, consider the case where a user uploads billing data and its inspection results from the past year to the system. This past data includes the billing date, billing item, amount, and whether or not inspection was possible. Data such as "January 1, 2022, equipment purchase, 10,000 yen, inspection possible" and "February 1, 2022, conference expenses, 20,000 yen, inspection impossible" are input. The server analyzes this data and later learns patterns such as "the smaller the amount, the higher the probability of inspection being possible" and "if conference expenses are expensive, inspection is impossible." When the user enters new billing data "January 1, 2023, equipment purchase, ¥15,000" and "February 1, 2023, conference fee, ¥30,000," the server analyzes this and generates new acceptance information "January 1, 2023, equipment purchase, ¥15,000, accepted" and "February 1, 2023, conference fee, ¥30,000, not accepted."

[0800] Furthermore, if the emotion engine analyzes the user's facial expression data and determines that the user is stressed, the system displays the inspection results in a more user-friendly and understandable format. For example, it provides additional options such as "Do you want to check again?" or "Do you need assistance?", reducing the burden on the user. This system allows users to perform inspection work efficiently and significantly reduces the burden of manual work. In addition, the integration of the emotion engine allows for flexible responses that take into account the user's emotional state, providing excellent flexibility that can be applied even in cases where it is difficult to change the inspection method.

[0801] An example of a prompt could be, "We are creating an application that recognizes user emotions. If a worker feels stressed or confused, how should the system change the interface? For example, by showing options such as 'Do you want to check again?' or 'Do you need help?'"

[0802] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0803] Step 1:

[0804] Users input or upload past inspection data. They can manually enter data using the terminal interface or upload data in a format such as a CSV file. This sends information such as the billing date, billing item, amount, and whether or not the inspection was successful to the server as input data.

[0805] Step 2:

[0806] The server receives past inspection data sent from the terminal and performs a format check. If the format is correct, it is saved in a temporary storage area and then stored in a database. In the format check process, the accuracy of the data format is confirmed and inaccurate data is excluded.

[0807] Step 3:

[0808] The server analyzes past inspection data stored in a database. It uses a machine learning algorithm to learn the characteristics and patterns of the data. For example, as criteria for determining whether an item can be inspected, it can find patterns such as "the smaller the amount, the higher the probability that it can be inspected" or "if certain items are included, it cannot be inspected." The input for this process is the stored data, and the output is a learning model.

[0809] Step 4:

[0810] The user also inputs or uploads new billing data through the terminal interface, which also includes the billing date, billing item, amount, etc. This becomes the input data and is sent to the server.

[0811] Step 5:

[0812] The server receives the new claim data, performs format checks and stores it in the database. Once the format check is complete, the new claim data is stored in the database. This data is used as input for subsequent analysis.

[0813] Step 6:

[0814] The server analyzes newly received billing data and automatically generates acceptance information using a machine learning model learned from past data. For example, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be accepted for inspection." The input data is the new billing data, and the output is the generated acceptance information.

[0815] Step 7:

[0816] The server stores the generated inspection information in a database and then provides the information via a terminal interface for the user to review. The user can review the inspection results and make corrections or approvals as necessary. The input is the generated inspection information, and the output is the information provided to the user.

[0817] Step 8:

[0818] The emotion engine recognizes the user's emotional state in real time by analyzing the user's facial expressions and tone of voice using the smartphone's camera and microphone. The input data is camera footage and audio data, and the output is the identified emotional state.

[0819] Step 9:

[0820] The server adjusts the interface based on the emotional state recognized by the emotion engine. For example, if the user is determined to be stressed, the server displays a more user-friendly error message and offers additional help options. Alternatively, if the user is anxious, the server sends a notification prompting the user to reconfirm the acceptance information. The input is the identified emotional state, and the output is the adjusted interface display.

[0821] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0822] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0823] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0824] [Third embodiment]

[0825] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0826] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0827] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0828] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0829] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0830] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0831] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0832] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0833] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0834] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0835] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0836] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0837] This invention relates to a system that improves the efficiency of inspection work and solves the problem of conventional manual generation of inspection information. This system operates in conjunction with the server, terminals, and users to automate the process from inputting inspection data to analyzing it, generating inspection information, and outputting the results.

[0838] System configuration

[0839] First, the terminal provides an interface for users to input or upload past data. Users can manually input past inspection data or upload it in a CSV file format. This data includes the invoice date, invoice item, amount, and whether or not the data was accepted.

[0840] The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area, after which it stores it in a database.

[0841] Analysis and learning from past data

[0842] The server analyzes the stored past inspection data and uses a machine learning algorithm to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an item can be inspected. For example, if the amount is small or if certain items are included, it will find patterns that indicate "acceptable inspection."

[0843] Receiving and analyzing new claims data

[0844] Users enter or upload new billing data through a similar interface. The new data contains the same information as the previous data (billing date, billing item, amount, etc.). The server receives this new data, checks its format, and stores it.

[0845] Generating acceptance information

[0846] The server automatically generates inspection information based on newly received billing data using a model learned from past data. For example, based on past learning results, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[0847] Saving and providing inspection results

[0848] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[0849] Specific examples

[0850] For example, consider the case where a user uploads billing data and its inspection results from the past year to the system. This past data includes the billing date, billing item, amount, and whether or not inspection was possible. Data such as "January 1, 2022, Equipment Purchase, ¥10,000, Inspection Possible" and "February 1, 2022, Conference Expenses, ¥20,000, Inspection Not Possible" are entered.

[0851] The server analyzes this data and later learns patterns such as "the smaller the amount, the higher the probability of acceptance" and "if the conference fee is high, it will not be accepted."

[0852] When the user enters new billing data "January 1, 2023, equipment purchase, ¥15,000" and "February 1, 2023, conference fee, ¥30,000," the server analyzes this and generates new acceptance information "January 1, 2023, equipment purchase, ¥15,000, accepted" and "February 1, 2023, conference fee, ¥30,000, not accepted."

[0853] This system allows users to efficiently carry out inspection work and significantly reduces the burden of manual work. In addition, even when it is difficult to change the inspection method, the system can be used to flexibly respond.

[0854] The processing flow will be explained below.

[0855] Step 1:

[0856] User enters or uploads historical data

[0857] Users can manually enter past inspection data through the system interface or upload it as a CSV file.

[0858] Specifically, the user clicks the "Select File" button to select the CSV file.

[0859] Step 2:

[0860] The server receives and stores the data

[0861] The server receives the past data sent by the user and checks the format and content of the data.

[0862] If the format is correct, the data is stored in a temporary storage area.

[0863] Step 3:

[0864] The server stores the data in a database

[0865] The server stores the data in the database after the format check is complete.

[0866] At this time, checks are also carried out to prevent errors such as data duplication or loss.

[0867] Step 4:

[0868] The server analyzes past data

[0869] The server analyzes past inspection data stored in a database and extracts various categories of data.

[0870] For example, classify data with tags such as "acceptable" and "unacceptable."

[0871] Step 5:

[0872] The server uses a machine learning model to learn the characteristics of the data.

[0873] The server uses machine learning algorithms to learn patterns and characteristics of inspections.

[0874] For example, patterns that determine whether or not a product can be accepted are searched for and saved as a model.

[0875] Step 6:

[0876] A user enters or uploads new claim data

[0877] Users enter new claim data through the system interface or upload it as a CSV file.

[0878] Step 7:

[0879] The server receives the new data and checks the format.

[0880] The server receives the newly submitted billing data and verifies that the data format is correct.

[0881] Step 8:

[0882] The server saves the new data

[0883] The server stores the data in a database after format checking is complete.

[0884] Step 9:

[0885] The server parses the new billing data

[0886] The server retrieves the new billing data and parses the information.

[0887] For example, extract billing items and amounts.

[0888] Step 10:

[0889] The server generates inspection information using the learning model.

[0890] The server uses the trained model to automatically generate an acceptance or rejection decision for new billing data.

[0891] For example, if the purchase of equipment on January 1, 2023 costs 15,000 yen, it is determined that the equipment can be inspected.

[0892] Step 11:

[0893] The server stores the generated inspection results and provides them to the user.

[0894] The server stores the generated inspection information in a database.

[0895] At the same time, the inspection results are displayed to the user on the system interface.

[0896] Step 12:

[0897] The user checks, corrects, and approves the inspection results

[0898] The user checks the inspection results through the system interface, makes corrections as necessary, and then approves them.

[0899] Example 1

[0900] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0901] Conventional inspection work is performed manually, which makes it inefficient and prone to human error. It is also difficult to analyze past inspection data and use it to make future inspection decisions. This makes the criteria for inspection decisions unclear, and increases the likelihood of inconsistent results. To solve this problem, a system is needed that streamlines inspection work and automatically generates inspection information using past data.

[0902] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0903] In this invention, the server includes means for receiving past inspection data, performing a format check, and saving it in a temporary storage area, means for analyzing the saved past inspection data and storing it in a database, means for analyzing the saved past inspection data and learning data characteristics and patterns using a machine learning algorithm, means for receiving new billing data, performing a format check, and saving it in a database, means for analyzing the new billing data and generating inspection information based on patterns learned from the past data, and means for saving the generated inspection information in a database and providing it to a user so that it can be checked via a terminal. This makes it possible to improve the efficiency and accuracy of inspection work.

[0904] "Inspection data" refers to data that includes information related to the inspection work, such as the invoice date, invoice item, amount, and whether or not the inspection is possible.

[0905] A "server" is a computer system used for the purposes of processing, storing, analyzing received data, and generating necessary information.

[0906] A "terminal" is a device that allows a user to input data or view system results. This includes computers, smartphones, tablets, etc.

[0907] "User" means a person or organization that uses the system to enter or upload acceptance data and verify the generated acceptance information.

[0908] "Format check" is the process of verifying that the received data conforms to the expected format and structure.

[0909] The "temporary storage area" is a storage space for temporarily storing data before it is finally stored in the database.

[0910] A "database" is a system that stores data in an organized, searchable, updateable, and manageable manner. This includes relational databases and NoSQL databases.

[0911] A "machine learning algorithm" is an algorithm that automatically learns patterns and features from data and makes predictions and classifications for new data. Examples include decision trees and neural networks.

[0912] "Analysis" is the process of examining data in detail to find useful information and patterns within it.

[0913] "Inspection information" is information that includes the results of a judgment, such as whether or not a certain request can be inspected, generated based on past data and machine learning algorithms.

[0914] "Automatic generation" is a function in which the system automatically generates data without manual human intervention.

[0915] "Learning" is the process by which a machine learning model uses past data to understand patterns and features and apply them to future data.

[0916] A "prompt sentence" is a guide message that is displayed to prompt the user to take a specific action.

[0917] This invention relates to a system for streamlining inspection work and solving the problem of conventional manual generation of inspection information. This system involves the collaboration of a server, terminals, and users to automate the entire process from inputting inspection data to analyzing it, generating inspection information, and outputting the results.

[0918] System configuration

[0919] First, the user is provided with an interface on the terminal to input or upload past inspection data. The user can manually input the past inspection data or upload it in a format such as a CSV file. This data includes the invoice date, invoice item, amount, whether or not the inspection was successful, etc.

[0920] For example, a user enters "January 1, 2022, Equipment Purchase, 10,000 yen, Accepted for Inspection" into an input form in a web application, or selects and uploads a CSV file.

[0921] Receiving and storing data

[0922] The server receives the data sent from the terminal. After receiving it, it performs a format check and eliminates any inappropriate data. Next, it saves the data in a temporary storage area and then stores it in a database. For format checks and database operations, it uses, for example, the Python pandas library or MySQL.

[0923] Analysis and learning from past data

[0924] The server analyzes the stored past inspection data. For the analysis, it uses machine learning libraries such as Python's scikit-learn and TensorFlow to learn the characteristics and patterns of the data. This allows it to build criteria for determining whether or not an inspection is acceptable. Specifically, it trains a model using scikit-learn's DecisionTreeClassifier.

[0925] For example, it learns patterns such as "the smaller the amount, the higher the probability that inspection is possible" and "if the conference fee is high, inspection is impossible."

[0926] Entering New Claim Data

[0927] Users enter or upload new billing data using the web application interface, just like with past data. New data includes billing date, billing item, and amount.

[0928] As an example, let's assume a process in which a user enters "January 1, 2023, Equipment Purchase, ¥15,000" in the data entry field and uploads it.

[0929] Analyzing New Data

[0930] The server receives the newly received billing data, checks the format, then stores the data and analyzes it using a machine learning model learned from past data.

[0931] Generating acceptance information

[0932] The server automatically generates acceptance information based on the analysis results. Specifically, it uses a trained machine learning model (for example, using the predict method in scikit-learn) to generate prediction results. The prediction results are provided in the form of "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be accepted for inspection."

[0933] Saving and providing inspection results

[0934] The generated inspection information is stored in a database on the server, and users can check these inspection results through the terminal interface. If necessary, users can correct or approve the results.

[0935] Prompt statement

[0936] As an example of a prompt sentence, the message to prompt the user to enter past billing data is shown below.

[0937] Please upload your past billing data. The data must include the billing date, billing item, amount, and whether or not it can be inspected. For example, please enter it in the following format: "January 1, 2022, Equipment Purchase, 10,000 yen, Acceptable."

[0938] This system makes it possible to improve the efficiency and accuracy of inspection work. Users can perform inspection work efficiently, significantly reducing the burden of manual work. It also allows flexible inspection decisions to be made based on past data.

[0939] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0940] Step 1: Data entry

[0941] Users input or upload past inspection data. Using an interface on their device (e.g., a web application), they select and upload a CSV file, etc. The input data includes the billing date, billing item, amount, and whether or not the data was accepted.

[0942] Input: Past inspection data (e.g., "January 1, 2022, Equipment Purchase, ¥10,000, Accepted")

[0943] Output: Uploaded CSV file or input form data

[0944] Specific operation: The user selects a CSV file and clicks the upload button, or manually enters past billing data into the data entry field.

[0945] Step 2: Receiving and storing data

[0946] The server receives the data sent from the terminal. It checks the format of the received data and eliminates any inappropriate data. It then saves the data in a temporary storage area and stores the correct data in a database. Specifically, it uses the Python pandas library to read the CSV file and MySQL as the database.

[0947] Input: Past inspection data uploaded by the user

[0948] Output: Correctly formatted acceptance data stored in the database

[0949] Specific operation: Read a CSV file using pandas, validate the data format, and insert the validated data into a MySQL database.

[0950] Step 3: Analyze and learn from past data

[0951] The server analyzes the stored past inspection data. It uses machine learning libraries such as Python's scikit-learn and TensorFlow to learn the features and patterns of the data. This allows it to build criteria for determining whether or not an inspection is acceptable. Specifically, it trains a model using scikit-learn's DecisionTreeClassifier.

[0952] Input: Past inspection data stored in the database

[0953] Output: A trained machine learning model

[0954] Specific operation: A decision tree model is trained using scikit-learn to learn the criteria for determining whether or not a product can be accepted.

[0955] Step 4: Enter new claim data

[0956] Users enter or upload new billing data using the web application interface, just like with past data. New data includes billing date, billing item, and amount.

[0957] Input: New billing data (e.g., "January 1, 2023, Equipment Purchase, ¥15,000")

[0958] Output: New billing data uploaded

[0959] Specific operation: The user enters "January 1, 2023, Equipment Purchase, ¥15,000" in the data input field and clicks the upload button.

[0960] Step 5: Analyze the new claims data

[0961] The server receives newly received billing data, performs format checks, stores valid data in a database, and analyzes new data using machine learning models learned from past data.

[0962] Input: New claim data

[0963] Output: New, correctly formatted billing data, analysis results

[0964] Specific operations: Read new billing data using pandas, check the format, and save it to the database. Analyze the new data using scikit-learn's predict method.

[0965] Step 6: Generate acceptance information

[0966] The server automatically generates inspection information based on the analysis results. It generates predictions using a machine learning model (for example, using the predict method in scikit-learn) that has learned from past data.

[0967] Input: Parsed new claims data

[0968] Output: Generated acceptance information (e.g., "January 1, 2023, Equipment Purchase, ¥15,000, Accepted")

[0969] Specific operation: Automatically generate inspection information using a trained machine learning model.

[0970] Step 7: Save and provide inspection results

[0971] The server stores the generated inspection information in a database. Users can check these inspection results through the terminal interface and make corrections or approvals as necessary.

[0972] Input: Generated acceptance information

[0973] Output: Inspection information stored in the database, inspection results provided to the user

[0974] Specific operation: The inspection results are inserted into the database and displayed in the web interface.

[0975] (Application example 1)

[0976] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0977] Inspection work at logistics centers involves a lot of manual work, which takes time and effort. In addition, inspection standards vary from person to person, which can lead to a lack of consistency in inspection results. Furthermore, in many cases, past inspection data cannot be fully utilized, which makes it difficult to make efficient inspection decisions.

[0978] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0979] In this invention, the server includes means for inputting or uploading past inspection data, means for analyzing the saved past inspection data and learning data characteristics and patterns, means for inputting or uploading new billing data, means for analyzing the new billing data and generating inspection information based on patterns learned from the past data, means for saving the generated inspection information and providing it to users, and means for using a robot in a logistics facility to scan specified products and generate inspection information. This makes inspection work more efficient and enables inspections to be performed according to consistent standards.

[0980] "Past inspection data" is information related to inspection work that has been carried out in the past, and includes detailed data such as the date of invoice, the item of invoice, the amount, and whether or not the item was inspected.

[0981] "Means for input or upload" refers to the functionality of the interface for a user to manually enter data or provide an existing data file to the system.

[0982] "Means of analyzing stored past inspection data and learning data features and patterns" refers to the functionality of machine learning algorithms that analyze past inspection data, learn specific patterns and features, and build criteria for judging future data.

[0983] "New Claims Data" means data containing information about new claims that are currently or will be made in the future.

[0984] "Means for entering or uploading new claim data" refers to the functionality of the interface that allows a user to provide new claim data to the system.

[0985] "Means for analyzing new billing data and generating inspection information based on patterns learned from past data" refers to the function of analyzing new billing data, determining whether or not to inspect it based on patterns learned in the past, and generating that information.

[0986] "Means for saving the generated inspection information and providing it to the user" refers to the interface function for saving the inspection results generated by the system and providing the information so that the user can check it.

[0987] A "logistics facility" is a facility used for logistics operations such as receiving, storing, inspecting, and shipping goods.

[0988] "Robots that scan specified products and generate inspection information" refer to autonomous robots that scan barcodes, QR codes, etc. of specified products within a logistics facility and automatically generate inspection information based on that information.

[0989] This invention provides a system for improving the efficiency of inspection work at logistics facilities. The specific configuration and operation of this system will be described below.

[0990] First, the user uses an interface on the terminal to input or upload past inspection data. The user can input the past inspection data manually or upload it as a CSV file. This past inspection data includes information such as the invoice date, invoice item, amount, and whether or not the inspection was successful.

[0991] The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area.Then, it stores this data in a database.

[0992] The server then analyzes the stored past inspection data and uses machine learning algorithms to learn the characteristics and patterns of the data. This analysis and learning is performed using machine learning frameworks such as TensorFlow. Through this learning, criteria for determining whether or not an item can be inspected are built within the server. For example, patterns that indicate "acceptance" are found when "the amount is small" or "specific items are included."

[0993] The user then enters or uploads new billing data through a similar interface. This new data contains the same information as the previous data (billing date, billing item, amount, etc.). The server receives this new data, checks its format, and stores it.

[0994] The server automatically generates inspection information based on newly received billing data using a model learned from past data. For example, based on past learning results, the server can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[0995] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[0996] Furthermore, this system includes a robot that scans designated products at logistics facilities and generates inspection information. The robot scans the products within the logistics facility and sends the data to a server. The server analyzes the data and uses a model trained on past data to determine whether or not the product can be inspected. The generated inspection information is displayed on the robot's display, allowing users to easily check it.

[0997] As a concrete example, consider the case where a user uploads billing data from the past year to the system. This data includes information such as "January 1, 2022, Equipment Purchase, ¥10,000, Acceptable" and "February 1, 2022, Meeting Expenses, ¥20,000, Unacceptable." The server analyzes this data and learns patterns. Subsequently, when new billing data "January 1, 2023, Equipment Purchase, ¥15,000" and "February 1, 2023, Meeting Expenses, ¥30,000" are received, the server generates acceptance information for "January 1, 2023, Equipment Purchase, ¥15,000, Acceptable" and "February 1, 2023, Meeting Expenses, ¥30,000, Unacceptable."

[0998] An example of a prompt is, "Based on the inspection data from the past year, determine whether or not new products can be inspected and record this automatically." In this way, it is possible to improve the efficiency of inspection work at logistics facilities and provide consistent inspection standards.

[0999] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1000] Step 1:

[1001] The user inputs or uploads past inspection data.

[1002] Specifically, users can manually enter past billing data (billing date, billing item, amount, acceptance / acceptance status, etc.) through the terminal interface, or upload a CSV file. The entered data is checked for formatting on the terminal and then sent to the server.

[1003] Input: Past inspection data

[1004] Output: Format-checked past inspection data is sent to the server

[1005] Step 2:

[1006] The server receives and stores past inspection data.

[1007] The server receives the data sent from the terminal, checks the format again, and saves the data in a temporary storage area.Then, it stores this data in a database.

[1008] Input: Format-checked past inspection data

[1009] Output: Past inspection data stored in a database

[1010] Step 3:

[1011] The server analyzes the stored past inspection data and learns the characteristics and patterns of the data.

[1012] The server analyzes past inspection data stored in a database using machine learning frameworks such as TensorFlow to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an inspection is acceptable.

[1013] Input: Past inspection data

[1014] Output: A trained machine learning model

[1015] Step 4:

[1016] A user enters or uploads new claim data.

[1017] Users can manually enter new billing data (billing date, billing item, amount, etc.) through their terminal or upload it as a CSV file, which is also format-checked on the terminal and then sent to the server.

[1018] Input: New claim data

[1019] Output: New format-checked claim data is sent to the server

[1020] Step 5:

[1021] The server receives the new billing data and generates acceptance information.

[1022] The server analyzes newly received billing data using a machine learning model and generates acceptance information based on patterns learned from past data. For example, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be accepted for inspection."

[1023] Input: New claims data, trained machine learning model

[1024] Output: Generated acceptance information

[1025] Step 6:

[1026] The server stores the generated acceptance information and provides it to the user.

[1027] The generated inspection information is stored in the server's database. An interface is then provided so that users can check the inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[1028] Input: Generated acceptance information

[1029] Output: Inspection information stored in the database, interface provided to the user

[1030] Step 7:

[1031] The robot scans the specified product and generates inspection information.

[1032] Robots used in logistics facilities scan the barcodes or QR codes of designated products and send the data to a server. The server analyzes the data and uses machine learning models to determine whether or not the product should be inspected. The inspection results are then displayed on the robot's display.

[1033] Input: Scanned product data

[1034] Output: Inspection information displayed on the robot's display

[1035] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1036] This invention is a system that streamlines inspection work and solves the problem of conventional manual generation of inspection information, and also combines an "emotion engine" that recognizes user emotions and reflects them in the operation of the system. This system operates in conjunction with the server, terminal, and user, automating the entire process from inputting inspection data to analyzing it, generating inspection information, outputting the results, and recognizing user emotions.

[1037] System configuration

[1038] First, the terminal provides an interface for users to input or upload past data. Users can manually input past inspection data or upload it in a CSV file format. This data includes the invoice date, invoice item, amount, and whether or not the data was accepted.

[1039] The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area, after which it stores it in a database.

[1040] Analysis and learning from past data

[1041] The server analyzes the stored past inspection data and uses a machine learning algorithm to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an item can be inspected. For example, if the amount is small or if certain items are included, it will find patterns that indicate "acceptable inspection."

[1042] Receiving and analyzing new claims data

[1043] Users enter or upload new billing data through a similar interface. The new data contains the same information as the previous data (billing date, billing item, amount, etc.). The server receives this new data, checks its format, and stores it.

[1044] Generating acceptance information

[1045] The server automatically generates inspection information based on newly received billing data using a model learned from past data. For example, based on past learning results, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[1046] Saving and providing inspection results

[1047] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[1048] Emotion engine integration

[1049] The present invention further integrates an emotion engine to recognize the user's emotions, which analyzes data such as the user's facial expressions and tone of voice to identify the user's emotional state.

[1050] The emotion engine allows the system to adjust the interface display when a user is stressed or confused, helping them to complete their tasks more smoothly. For example, if a user is in a negative emotional state, the system can display error messages in a gentle tone and provide additional help options. Additionally, if a user feels anxious, the system can send a notification prompting them to reconfirm the inspection information.

[1051] Specific examples

[1052] For example, consider the case where a user uploads billing data and its inspection results from the past year to the system. This past data includes the billing date, billing item, amount, and whether or not inspection was possible. Data such as "January 1, 2022, Equipment Purchase, ¥10,000, Inspection Possible" and "February 1, 2022, Conference Expenses, ¥20,000, Inspection Not Possible" are entered.

[1053] The server analyzes this data and later learns patterns such as "the smaller the amount, the higher the probability of acceptance" and "if the conference fee is high, it will not be accepted."

[1054] When the user enters new billing data "January 1, 2023, equipment purchase, ¥15,000" and "February 1, 2023, conference fee, ¥30,000," the server analyzes this and generates new acceptance information "January 1, 2023, equipment purchase, ¥15,000, accepted" and "February 1, 2023, conference fee, ¥30,000, not accepted."

[1055] Furthermore, if the emotion engine analyzes the user's facial expression data and determines that the user is feeling stressed, the system will display the inspection results in a more friendly and understandable format, providing additional options such as "Do you want to check again?" or "Do you need help?", reducing the burden on the user.

[1056] This system allows users to efficiently perform inspection work and significantly reduces the burden of manual work. In addition, the integration of an emotion engine allows for flexible responses that take into account the user's emotional state, providing excellent flexibility that can be applied even in cases where it is difficult to change the inspection method.

[1057] The processing flow will be explained below.

[1058] Step 1:

[1059] User enters or uploads historical data

[1060] Users can manually enter past inspection data through the system interface or upload it as a CSV file.

[1061] Specifically, the user clicks the "Select File" button to select the CSV file.

[1062] Step 2:

[1063] The server receives and stores the data

[1064] The server receives the past data sent by the user and checks the format and content of the data.

[1065] If the format is correct, the data is stored in a temporary storage area.

[1066] Step 3:

[1067] The server stores the data in a database

[1068] The server stores the data in the database after the format check is complete.

[1069] At this time, checks are also carried out to prevent errors such as data duplication or loss.

[1070] Step 4:

[1071] The server analyzes past data

[1072] The server analyzes past inspection data stored in a database and extracts various categories of data.

[1073] For example, classify data with tags such as "acceptable" and "unacceptable."

[1074] Step 5:

[1075] The server uses a machine learning model to learn the characteristics of the data.

[1076] The server uses machine learning algorithms to learn patterns and characteristics of inspections.

[1077] For example, patterns that determine whether or not a product can be accepted are searched for and saved as a model.

[1078] Step 6:

[1079] A user enters or uploads new claim data

[1080] Users enter new claim data through the system interface or upload it as a CSV file.

[1081] Step 7:

[1082] The server receives the new data and checks the format.

[1083] The server receives the newly submitted billing data and verifies that the data format is correct.

[1084] Step 8:

[1085] The server saves the new data

[1086] The server stores the data in a database after format checking is complete.

[1087] Step 9:

[1088] The server parses the new billing data

[1089] The server retrieves the new billing data and parses the information.

[1090] For example, extract billing items and amounts.

[1091] Step 10:

[1092] The server generates inspection information using the learning model.

[1093] The server uses the trained model to automatically generate an acceptance or rejection decision for new billing data.

[1094] For example, if the purchase of equipment on January 1, 2023 costs 15,000 yen, it is determined that the equipment can be inspected.

[1095] Step 11:

[1096] The server stores the generated inspection results and provides them to the user.

[1097] The server stores the generated inspection information in a database.

[1098] At the same time, the inspection results are displayed to the user on the system interface.

[1099] Step 12:

[1100] Emotion engine recognizes user emotions

[1101] The emotion engine analyzes facial expressions and tone of voice through the user's camera and microphone.

[1102] This identifies the user's emotional state, such as whether they are stressed or confused.

[1103] Step 13:

[1104] The server adjusts the interface based on the emotional state.

[1105] The server adjusts the way the system interface is displayed based on the user's emotional state obtained from the emotion engine.

[1106] For example, if a user is stressed, provide a more friendly and understandable display or additional help options.

[1107] Step 14:

[1108] The user checks, corrects, and approves the inspection results

[1109] The user checks the inspection results through the system interface, makes corrections as necessary, and then approves them.

[1110] The emotion engine will display a notification to encourage reconsideration if the user is feeling anxious.

[1111] Example 2

[1112] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1113] In conventional inspection work, inspection information is generated manually, which requires a lot of time and effort. In addition, because the user's emotional state is not taken into consideration, users often feel stressed when using the system. This reduces work efficiency and increases the risk of mistakes. To solve these problems, a system that automatically generates inspection information and takes the user's emotional state into consideration is needed.

[1114] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1115] In this invention, the server includes a means for inputting or uploading past inspection data, a means for analyzing the stored past inspection data and learning data characteristics and patterns, and a means for inputting or uploading new billing data, which enables automatic generation of inspection information and flexible system display that takes into account the user's emotional state.

[1116] "Past inspection data" is information relating to inspection work carried out in the past, including the date of invoice, invoice item, amount, whether or not inspection was possible, and the like.

[1117] "Input or upload means" means an interface through which a user can manually enter data or upload it to the system in the form of a file.

[1118] The "means for analyzing the stored past inspection data" refers to an algorithm or program for analyzing the past inspection data received by the server and extracting characteristics and patterns of the data.

[1119] "Methods of learning features and patterns in data" is the process of using machine learning algorithms to automatically learn useful information and trends from past data.

[1120] "New billing data" refers to information about a bill that is newly entered or uploaded into the system, including the billing date, billing items, amount, etc.

[1121] The "means for generating inspection information" refers to an algorithm or program that uses learned patterns or models to automatically generate information such as whether or not an item can be inspected from new invoice data.

[1122] The "means for storing the generated acceptance information" is a process in which the server stores the automatically generated acceptance information in a persistent storage device such as a database.

[1123] The "means for providing to the user" is an interface that allows the user to check, correct, and approve the stored acceptance information.

[1124] "Means for recognizing the user's emotions and adjusting the system's behavior" refers to algorithms or programs that analyze the user's facial expressions, tone of voice, etc., and change the system's display method and behavior based on the results.

[1125] This system improves the efficiency of inspection work and adjusts the operation of the system by recognizing the user's emotions. A specific embodiment of this system will be described below.

[1126] First, the user inputs or uploads past inspection data into the terminal. Specifically, past billing data (billing date, billing item, amount, whether inspection was possible, etc.) can be manually input or uploaded in CSV file format. This data is sent to the server through the terminal interface.

[1127] The server receives the data sent from the device and performs a format check. Format check is a process to ensure the data is correct, and invalid data is logged. The data is then saved in a temporary storage area and then permanently stored in a database (e.g., MySQL).

[1128] Next, the server analyzes the stored past inspection data and learns the data's features and patterns. This process uses machine learning algorithms (such as scikit-learn or TensorFlow). The data is loaded into a pandas DataFrame, features are extracted, and the data is split into training datasets. The data is trained using scikit-learn's RandomForestClassifier, and the generated model is saved in pickle format.

[1129] Next, the user enters or uploads new billing data, just like the previous data. The new data includes the billing date, billing item, amount, etc. This is also sent to the server via the terminal.

[1130] The server receives the new data, checks the format again, and saves it to the database. It then uses the saved learning model to generate acceptance information from the new data. Specifically, it inputs the new data into the trained model and uses the model's .predict method to predict whether the data will be accepted or not. The generated acceptance information is saved in the database.

[1131] The generated inspection information is provided to the user as the inspection result. The terminal provides an interface that allows the user to check the inspection result and make corrections or approvals. The user checks the results and makes corrections or approvals as necessary.

[1132] The system also incorporates an "emotion engine" to recognize the user's emotions. The device captures the user's facial expressions and tone of voice using a camera and microphone and sends this data to a server. The server then uses the emotion engine to analyze the user's emotional state and adjusts the system's display accordingly. For example, facial expression recognition is performed using OpenCV and voice analysis is performed using Google Cloud Speech-to-Text. If the system determines that the user is stressed, it displays additional options such as "Would you like to check again?" or "Do you need support?"

[1133] Specific examples

[1134] Consider the case where a user uploads billing data from the past year to the system. The past data includes information such as "January 1, 2022, equipment purchase, 10,000 yen, accepted" and "February 1, 2022, conference fee, 20,000 yen, not accepted." The server analyzes this data and learns patterns such as "the smaller the amount, the higher the probability of acceptance" and "if the conference fee is high, it is not accepted."

[1135] When new billing data "January 1, 2023, Equipment Purchase, ¥15,000" and "February 1, 2023, Conference Expenses, ¥30,000" are entered, the server analyzes this and generates new acceptance information "January 1, 2023, Equipment Purchase, ¥15,000, Accepted" and "February 1, 2023, Conference Expenses, ¥30,000, Not Accepted."

[1136] To achieve this, the following prompts are given to the generative AI model:

[1137] "Analyze the following historical data and generate a prompt to predict whether the new billing data will be accepted: 'January 1, 2022, Equipment Purchase, ¥10,000, Acceptable'; 'February 1, 2022, Conference Expenses, ¥20,000, Unacceptable'. The new billing data is 'January 1, 2023, Equipment Purchase, ¥15,000'."

[1138] This system allows users to efficiently perform inspection work and significantly reduces the manual workload. In addition, by utilizing an emotion engine, it is possible to respond flexibly according to the user's emotional state, providing a seamless user experience.

[1139] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1140] Step 1: User enters or uploads historical data

[1141] Users can input past inspection data into the terminal or upload it in CSV file format. Input information includes the billing date, billing item, amount, and whether or not inspection was possible. Specifically, users can either manually input data using the keyboard or select a CSV file from the file selection dialog and upload it.

[1142] Input: Invoice date, invoice item, amount, and inspection availability data

[1143] Output: Data transfer from device to server

[1144] Step 2: Server receives data and checks format

[1145] The server receives the data sent from the terminal. After receiving it, it performs a format check on the data. The format check is a process to check whether the data is written correctly in JSON format, and invalid data is recorded in a log.

[1146] Input: Data sent from the terminal

[1147] Output: Normalized data after format check, logging of invalid data

[1148] Step 3: Temporarily save the data and store it in the database

[1149] The server saves data that passes the format check in a temporary storage area. It then stores it permanently in a database (such as MySQL). Specific operations include registering the data in the MySQL database with an INSERT statement.

[1150] Input: Normalized data after format check

[1151] Output: Inspection data stored in the database

[1152] Step 4: Data analysis and model training by the server

[1153] The server analyzes the stored past inspection data and uses machine learning algorithms to learn the data's features and patterns. This process involves loading the data into a pandas DataFrame and extracting features. Next, it runs training using scikit-learn's RandomForestClassifier and saves the trained model in pickle format.

[1154] Input: Past inspection data stored in the database

[1155] Output: A trained machine learning model

[1156] Step 5: User enters or uploads new claim data

[1157] The user inputs or uploads new billing data to the terminal in the same way as past data. Input information includes billing date, billing item, amount, etc. Specific operations include manually entering data using the keyboard or uploading a CSV file by selecting it from the file selection dialog.

[1158] Input: New invoice date, invoice item, and amount data

[1159] Output: New data transfer from device to server

[1160] Step 6: Server receives new data and checks format

[1161] The server receives the new billing data and performs another format check to ensure that the data is correctly formatted in JSON, and any invalid data is logged.

[1162] Input: New billing data sent from the terminal

[1163] Output: Normalized data after format check, logging of invalid data

[1164] Step 7: Save the new data to the database

[1165] The server saves the new data that passes the format check to the database, which specifically involves the process of registering the data in the MySQL database with an INSERT statement.

[1166] Input: Normalized data after format check

[1167] Output: New billing data stored in the database

[1168] Step 8: Server analyzes new data and generates acceptance information

[1169] The server analyzes newly received data using the existing learning model and automatically generates inspection information. The new data is input into the trained model and the .predict method of the model is used to predict whether the data can be inspected or not.

[1170] Input: New claims data, trained machine learning model

[1171] Output: Automatically generated inspection information

[1172] Step 9: Save the generated acceptance information and provide it to the user

[1173] The server stores the generated inspection information in a database and provides an interface for users to view it. Specifically, it stores the inspection results in a MySQL database and displays the results on the dashboard of the web application.

[1174] Input: Automatically generated acceptance information

[1175] Output: Inspection information stored in the database, user interface for checking inspection results

[1176] Step 10: Analyze user emotions using the emotion engine and adjust the interface

[1177] The device captures the user's facial expressions and tone of voice using a camera and microphone and sends this data to a server. The server then uses an emotion engine to analyze the user's emotional state and adjusts the system's display accordingly. Specifically, it uses OpenCV for facial recognition, Google Cloud Speech-to-Text for speech analysis, and changes the display of UI components based on the results.

[1178] Input: User's facial expression data, voice data

[1179] Output: Sentiment analysis results, adjusted interface display

[1180] (Application example 2)

[1181] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1182] Conventional inspection work is often done manually, which is time-consuming and labor-intensive, resulting in inefficiency. Therefore, there is a demand for automation of the generation of inspection information. However, to further improve work efficiency and reduce worker stress and confusion, conventional systems lack the functionality to consider the user's emotional state. Therefore, the objective of this invention is to provide a flexible operation interface that takes into account the worker's emotional state while improving the efficiency of inspection work.

[1183] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting or uploading past inspection data, means for analyzing the saved past inspection data and learning data characteristics and patterns, means for inputting or uploading new billing data, means for analyzing the new billing data and generating inspection information based on patterns learned from the past data, means for saving the generated inspection information and providing it to the user, and means for recognizing the user's emotional state and adjusting the interface. This makes it possible to improve the efficiency of inspection work and reduce stress and confusion for the worker by using a flexible operation interface that takes the worker's emotional state into consideration.

[1184] "Past inspection data" refers to data related to inspections, such as previously received billing information and inspection results.

[1185] "Input or Upload Means" refers to an interface or mechanism through which a User manually enters data or uploads data in the form of a file.

[1186] "Means for analyzing stored past inspection data" refers to algorithms or systems that use accumulated past data to recognize and analyze its features and patterns.

[1187] "Means of learning data features and patterns" refers to the process of applying machine learning algorithms to stored data to find criteria and patterns for acceptance or rejection and build a model.

[1188] "Means for analyzing new claims data" refers to algorithms or systems that analyze the latest claims data provided by users and generate acceptance information using models learned from past data.

[1189] "Means for generating inspection information" refers to a system that uses past learning results based on new billing data to determine whether or not inspection is possible, and automatically creates inspection information based on that.

[1190] "Means for storing the generated inspection information and providing it to the user" refers to an interface or mechanism for storing the generated inspection information in a database, etc., and providing that information so that the user can check, modify, and approve it.

[1191] "Means for recognizing the user's emotional state and adjusting the interface" refers to a mechanism that analyzes data such as the user's facial expressions and voice to recognize their emotions, and changes the interface display method and operation guide according to the results.

[1192] "System" refers to a program or device with a set of functions designed to streamline inspection work, including the above means.

[1193] The present invention is a system that improves the efficiency of inspection work at logistics centers and provides a flexible interface that takes into account the emotional state of the user. This system operates in conjunction with the server, terminals, and users.

[1194] First, an interface is provided on the terminal side for users to input or upload past inspection data. Users can input past inspection data manually or upload it in a format such as a CSV file. This data includes the billing date, billing item, amount, and whether or not the data was inspected. The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area. It is then stored in the database.

[1195] Next, the server analyzes the stored past inspection data and uses a machine learning algorithm to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an item can be inspected. For example, it finds patterns that indicate "acceptable" when the amount is small or when certain items are included.

[1196] The user enters or uploads new billing data through a similar interface. This new data also contains the same information as the past data (billing date, billing item, amount, etc.). The server receives this new data, checks the format, and saves it. Next, the server automatically generates inspection information based on the newly received billing data using a model learned from past data. For example, based on past learning results, the server can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[1197] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[1198] Furthermore, the present invention integrates an emotion engine to recognize the user's emotions. The emotion engine analyzes data such as the user's facial expressions and tone of voice to identify the user's emotional state. If the user feels stressed or confused, the emotion engine can adjust the interface display to help the user perform their tasks more smoothly. For example, if the user is in a negative emotional state, the system can display an error message in a gentle tone and provide additional help options. Furthermore, if the user feels anxious, the system can send a notification prompting the user to reconfirm the inspection information.

[1199] As a concrete example, consider the case where a user uploads billing data and its inspection results from the past year to the system. This past data includes the billing date, billing item, amount, and whether or not inspection was possible. Data such as "January 1, 2022, equipment purchase, 10,000 yen, inspection possible" and "February 1, 2022, conference expenses, 20,000 yen, inspection impossible" are input. The server analyzes this data and later learns patterns such as "the smaller the amount, the higher the probability of inspection being possible" and "if conference expenses are expensive, inspection is impossible." When the user enters new billing data "January 1, 2023, equipment purchase, ¥15,000" and "February 1, 2023, conference fee, ¥30,000," the server analyzes this and generates new acceptance information "January 1, 2023, equipment purchase, ¥15,000, accepted" and "February 1, 2023, conference fee, ¥30,000, not accepted."

[1200] Furthermore, if the emotion engine analyzes the user's facial expression data and determines that the user is stressed, the system displays the inspection results in a more user-friendly and understandable format. For example, it provides additional options such as "Do you want to check again?" or "Do you need assistance?", reducing the burden on the user. This system allows users to perform inspection work efficiently and significantly reduces the burden of manual work. In addition, the integration of the emotion engine allows for flexible responses that take into account the user's emotional state, providing excellent flexibility that can be applied even in cases where it is difficult to change the inspection method.

[1201] An example of a prompt could be, "We are creating an application that recognizes user emotions. If a worker feels stressed or confused, how should the system change the interface? For example, by showing options such as 'Do you want to check again?' or 'Do you need help?'"

[1202] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1203] Step 1:

[1204] Users input or upload past inspection data. They can manually enter data using the terminal interface or upload data in a format such as a CSV file. This sends information such as the billing date, billing item, amount, and whether or not the inspection was successful to the server as input data.

[1205] Step 2:

[1206] The server receives past inspection data sent from the terminal and performs a format check. If the format is correct, it is saved in a temporary storage area and then stored in a database. In the format check process, the accuracy of the data format is confirmed and inaccurate data is excluded.

[1207] Step 3:

[1208] The server analyzes past inspection data stored in a database. It uses a machine learning algorithm to learn the characteristics and patterns of the data. For example, as criteria for determining whether an item can be inspected, it can find patterns such as "the smaller the amount, the higher the probability that it can be inspected" or "if certain items are included, it cannot be inspected." The input for this process is the stored data, and the output is a learning model.

[1209] Step 4:

[1210] The user also inputs or uploads new billing data through the terminal interface, which also includes the billing date, billing item, amount, etc. This becomes the input data and is sent to the server.

[1211] Step 5:

[1212] The server receives the new claim data, performs format checks and stores it in the database. Once the format check is complete, the new claim data is stored in the database. This data is used as input for subsequent analysis.

[1213] Step 6:

[1214] The server analyzes newly received billing data and automatically generates acceptance information using a machine learning model learned from past data. For example, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be accepted for inspection." The input data is the new billing data, and the output is the generated acceptance information.

[1215] Step 7:

[1216] The server stores the generated inspection information in a database and then provides the information via a terminal interface for the user to review. The user can review the inspection results and make corrections or approvals as necessary. The input is the generated inspection information, and the output is the information provided to the user.

[1217] Step 8:

[1218] The emotion engine recognizes the user's emotional state in real time by analyzing the user's facial expressions and tone of voice using the smartphone's camera and microphone. The input data is camera footage and audio data, and the output is the identified emotional state.

[1219] Step 9:

[1220] The server adjusts the interface based on the emotional state recognized by the emotion engine. For example, if the user is determined to be stressed, the server displays a more user-friendly error message and offers additional help options. Alternatively, if the user is anxious, the server sends a notification prompting the user to reconfirm the acceptance information. The input is the identified emotional state, and the output is the adjusted interface display.

[1221] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1222] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1223] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1224] [Fourth embodiment]

[1225] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1226] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1227] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1228] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1229] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1230] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1231] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1232] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1233] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1234] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1235] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1236] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1237] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1238] This invention relates to a system that improves the efficiency of inspection work and solves the problem of conventional manual generation of inspection information. This system operates in conjunction with the server, terminals, and users to automate the process from inputting inspection data to analyzing it, generating inspection information, and outputting the results.

[1239] System configuration

[1240] First, the terminal provides an interface for users to input or upload past data. Users can manually input past inspection data or upload it in a CSV file format. This data includes the invoice date, invoice item, amount, and whether or not the data was accepted.

[1241] The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area, after which it stores it in a database.

[1242] Analysis and learning from past data

[1243] The server analyzes the stored past inspection data and uses a machine learning algorithm to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an item can be inspected. For example, if the amount is small or if certain items are included, it will find patterns that indicate "acceptable inspection."

[1244] Receiving and analyzing new claims data

[1245] Users enter or upload new billing data through a similar interface. The new data contains the same information as the previous data (billing date, billing item, amount, etc.). The server receives this new data, checks its format, and stores it.

[1246] Generating acceptance information

[1247] The server automatically generates inspection information based on newly received billing data using a model learned from past data. For example, based on past learning results, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[1248] Saving and providing inspection results

[1249] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[1250] Specific examples

[1251] For example, consider the case where a user uploads billing data and its inspection results from the past year to the system. This past data includes the billing date, billing item, amount, and whether or not inspection was possible. Data such as "January 1, 2022, Equipment Purchase, ¥10,000, Inspection Possible" and "February 1, 2022, Conference Expenses, ¥20,000, Inspection Not Possible" are entered.

[1252] The server analyzes this data and later learns patterns such as "the smaller the amount, the higher the probability of acceptance" and "if the conference fee is high, it will not be accepted."

[1253] When the user enters new billing data "January 1, 2023, equipment purchase, ¥15,000" and "February 1, 2023, conference fee, ¥30,000," the server analyzes this and generates new acceptance information "January 1, 2023, equipment purchase, ¥15,000, accepted" and "February 1, 2023, conference fee, ¥30,000, not accepted."

[1254] This system allows users to efficiently carry out inspection work and significantly reduces the burden of manual work. In addition, even when it is difficult to change the inspection method, the system can be used to flexibly respond.

[1255] The processing flow will be explained below.

[1256] Step 1:

[1257] User enters or uploads historical data

[1258] Users can manually enter past inspection data through the system interface or upload it as a CSV file.

[1259] Specifically, the user clicks the "Select File" button to select the CSV file.

[1260] Step 2:

[1261] The server receives and stores the data

[1262] The server receives the past data sent by the user and checks the format and content of the data.

[1263] If the format is correct, the data is stored in a temporary storage area.

[1264] Step 3:

[1265] The server stores the data in a database

[1266] The server stores the data in the database after the format check is complete.

[1267] At this time, checks are also carried out to prevent errors such as data duplication or loss.

[1268] Step 4:

[1269] The server analyzes past data

[1270] The server analyzes past inspection data stored in a database and extracts various categories of data.

[1271] For example, classify data with tags such as "acceptable" and "unacceptable."

[1272] Step 5:

[1273] The server uses a machine learning model to learn the characteristics of the data.

[1274] The server uses machine learning algorithms to learn patterns and characteristics of inspections.

[1275] For example, patterns that determine whether or not a product can be accepted are searched for and saved as a model.

[1276] Step 6:

[1277] A user enters or uploads new claim data

[1278] Users enter new claim data through the system interface or upload it as a CSV file.

[1279] Step 7:

[1280] The server receives the new data and checks the format.

[1281] The server receives the newly submitted billing data and verifies that the data format is correct.

[1282] Step 8:

[1283] The server saves the new data

[1284] The server stores the data in a database after format checking is complete.

[1285] Step 9:

[1286] The server parses the new billing data

[1287] The server retrieves the new billing data and parses the information.

[1288] For example, extract billing items and amounts.

[1289] Step 10:

[1290] The server generates inspection information using the learning model.

[1291] The server uses the trained model to automatically generate an acceptance or rejection decision for new billing data.

[1292] For example, if the purchase of equipment on January 1, 2023 costs 15,000 yen, it is determined that the equipment can be inspected.

[1293] Step 11:

[1294] The server stores the generated inspection results and provides them to the user.

[1295] The server stores the generated inspection information in a database.

[1296] At the same time, the inspection results are displayed to the user on the system interface.

[1297] Step 12:

[1298] The user checks, corrects, and approves the inspection results

[1299] The user checks the inspection results through the system interface, makes corrections as necessary, and then approves them.

[1300] Example 1

[1301] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1302] Conventional inspection work is performed manually, which makes it inefficient and prone to human error. It is also difficult to analyze past inspection data and use it to make future inspection decisions. This makes the criteria for inspection decisions unclear, and increases the likelihood of inconsistent results. To solve this problem, a system is needed that streamlines inspection work and automatically generates inspection information using past data.

[1303] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1304] In this invention, the server includes means for receiving past inspection data, performing a format check, and saving it in a temporary storage area, means for analyzing the saved past inspection data and storing it in a database, means for analyzing the saved past inspection data and learning data characteristics and patterns using a machine learning algorithm, means for receiving new billing data, performing a format check, and saving it in a database, means for analyzing the new billing data and generating inspection information based on patterns learned from the past data, and means for saving the generated inspection information in a database and providing it to a user so that it can be checked via a terminal. This makes it possible to improve the efficiency and accuracy of inspection work.

[1305] "Inspection data" refers to data that includes information related to the inspection work, such as the invoice date, invoice item, amount, and whether or not the inspection is possible.

[1306] A "server" is a computer system used for the purposes of processing, storing, analyzing received data, and generating necessary information.

[1307] A "terminal" is a device that allows a user to input data or view system results. This includes computers, smartphones, tablets, etc.

[1308] "User" means a person or organization that uses the system to enter or upload acceptance data and verify the generated acceptance information.

[1309] "Format check" is the process of verifying that the received data conforms to the expected format and structure.

[1310] The "temporary storage area" is a storage space for temporarily storing data before it is finally stored in the database.

[1311] A "database" is a system that stores data in an organized, searchable, updateable, and manageable manner. This includes relational databases and NoSQL databases.

[1312] A "machine learning algorithm" is an algorithm that automatically learns patterns and features from data and makes predictions and classifications for new data. Examples include decision trees and neural networks.

[1313] "Analysis" is the process of examining data in detail to find useful information and patterns within it.

[1314] "Inspection information" is information that includes the results of a judgment, such as whether or not a certain request can be inspected, generated based on past data and machine learning algorithms.

[1315] "Automatic generation" is a function in which the system automatically generates data without manual human intervention.

[1316] "Learning" is the process by which a machine learning model uses past data to understand patterns and features and apply them to future data.

[1317] A "prompt sentence" is a guide message that is displayed to prompt the user to take a specific action.

[1318] This invention relates to a system for streamlining inspection work and solving the problem of conventional manual generation of inspection information. This system involves the collaboration of a server, terminals, and users to automate the entire process from inputting inspection data to analyzing it, generating inspection information, and outputting the results.

[1319] System configuration

[1320] First, the user is provided with an interface on the terminal to input or upload past inspection data. The user can manually input the past inspection data or upload it in a format such as a CSV file. This data includes the invoice date, invoice item, amount, whether or not the inspection was successful, etc.

[1321] For example, a user enters "January 1, 2022, Equipment Purchase, 10,000 yen, Accepted for Inspection" into an input form in a web application, or selects and uploads a CSV file.

[1322] Receiving and storing data

[1323] The server receives the data sent from the terminal. After receiving it, it performs a format check and eliminates any inappropriate data. Next, it saves the data in a temporary storage area and then stores it in a database. For format checks and database operations, it uses, for example, the Python pandas library or MySQL.

[1324] Analysis and learning from past data

[1325] The server analyzes the stored past inspection data. For the analysis, it uses machine learning libraries such as Python's scikit-learn and TensorFlow to learn the characteristics and patterns of the data. This allows it to build criteria for determining whether or not an inspection is acceptable. Specifically, it trains a model using scikit-learn's DecisionTreeClassifier.

[1326] For example, it learns patterns such as "the smaller the amount, the higher the probability that inspection is possible" and "if the conference fee is high, inspection is impossible."

[1327] Entering New Claim Data

[1328] Users enter or upload new billing data using the web application interface, just like with past data. New data includes billing date, billing item, and amount.

[1329] As an example, let's assume a process in which a user enters "January 1, 2023, Equipment Purchase, ¥15,000" in the data entry field and uploads it.

[1330] Analyzing New Data

[1331] The server receives the newly received billing data, checks the format, then stores the data and analyzes it using a machine learning model learned from past data.

[1332] Generating acceptance information

[1333] The server automatically generates acceptance information based on the analysis results. Specifically, it uses a trained machine learning model (for example, using the predict method in scikit-learn) to generate prediction results. The prediction results are provided in the form of "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be accepted for inspection."

[1334] Saving and providing inspection results

[1335] The generated inspection information is stored in a database on the server, and users can check these inspection results through the terminal interface. If necessary, users can correct or approve the results.

[1336] Prompt statement

[1337] As an example of a prompt sentence, the message to prompt the user to enter past billing data is shown below.

[1338] Please upload your past billing data. The data must include the billing date, billing item, amount, and whether or not it can be inspected. For example, please enter it in the following format: "January 1, 2022, Equipment Purchase, 10,000 yen, Acceptable."

[1339] This system makes it possible to improve the efficiency and accuracy of inspection work. Users can perform inspection work efficiently, significantly reducing the burden of manual work. It also allows flexible inspection decisions to be made based on past data.

[1340] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1341] Step 1: Data entry

[1342] Users input or upload past inspection data. Using an interface on their device (e.g., a web application), they select and upload a CSV file, etc. The input data includes the billing date, billing item, amount, and whether or not the data was accepted.

[1343] Input: Past inspection data (e.g., "January 1, 2022, Equipment Purchase, ¥10,000, Accepted")

[1344] Output: Uploaded CSV file or input form data

[1345] Specific operation: The user selects a CSV file and clicks the upload button, or manually enters past billing data into the data entry field.

[1346] Step 2: Receiving and storing data

[1347] The server receives the data sent from the terminal. It checks the format of the received data and eliminates any inappropriate data. It then saves the data in a temporary storage area and stores the correct data in a database. Specifically, it uses the Python pandas library to read the CSV file and MySQL as the database.

[1348] Input: Past inspection data uploaded by the user

[1349] Output: Correctly formatted acceptance data stored in the database

[1350] Specific operation: Read a CSV file using pandas, validate the data format, and insert the validated data into a MySQL database.

[1351] Step 3: Analyze and learn from past data

[1352] The server analyzes the stored past inspection data. It uses machine learning libraries such as Python's scikit-learn and TensorFlow to learn the features and patterns of the data. This allows it to build criteria for determining whether or not an inspection is acceptable. Specifically, it trains a model using scikit-learn's DecisionTreeClassifier.

[1353] Input: Past inspection data stored in the database

[1354] Output: A trained machine learning model

[1355] Specific operation: A decision tree model is trained using scikit-learn to learn the criteria for determining whether or not a product can be accepted.

[1356] Step 4: Enter new claim data

[1357] Users enter or upload new billing data using the web application interface, just like with past data. New data includes billing date, billing item, and amount.

[1358] Input: New billing data (e.g., "January 1, 2023, Equipment Purchase, ¥15,000")

[1359] Output: New billing data uploaded

[1360] Specific operation: The user enters "January 1, 2023, Equipment Purchase, ¥15,000" in the data input field and clicks the upload button.

[1361] Step 5: Analyze the new claims data

[1362] The server receives newly received billing data, performs format checks, stores valid data in a database, and analyzes new data using machine learning models learned from past data.

[1363] Input: New claim data

[1364] Output: New, correctly formatted billing data, analysis results

[1365] Specific operations: Read new billing data using pandas, check the format, and save it to the database. Analyze the new data using scikit-learn's predict method.

[1366] Step 6: Generate acceptance information

[1367] The server automatically generates inspection information based on the analysis results. It generates predictions using a machine learning model (for example, using the predict method in scikit-learn) that has learned from past data.

[1368] Input: Parsed new claims data

[1369] Output: Generated acceptance information (e.g., "January 1, 2023, Equipment Purchase, ¥15,000, Accepted")

[1370] Specific operation: Automatically generate inspection information using a trained machine learning model.

[1371] Step 7: Save and provide inspection results

[1372] The server stores the generated inspection information in a database. Users can check these inspection results through the terminal interface and make corrections or approvals as necessary.

[1373] Input: Generated acceptance information

[1374] Output: Inspection information stored in the database, inspection results provided to the user

[1375] Specific operation: The inspection results are inserted into the database and displayed in the web interface.

[1376] (Application example 1)

[1377] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1378] Inspection work at logistics centers involves a lot of manual work, which takes time and effort. In addition, inspection standards vary from person to person, which can lead to a lack of consistency in inspection results. Furthermore, in many cases, past inspection data cannot be fully utilized, which makes it difficult to make efficient inspection decisions.

[1379] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1380] In this invention, the server includes means for inputting or uploading past inspection data, means for analyzing the saved past inspection data and learning data characteristics and patterns, means for inputting or uploading new billing data, means for analyzing the new billing data and generating inspection information based on patterns learned from the past data, means for saving the generated inspection information and providing it to users, and means for using a robot in a logistics facility to scan specified products and generate inspection information. This makes inspection work more efficient and enables inspections to be performed according to consistent standards.

[1381] "Past inspection data" is information related to inspection work that has been carried out in the past, and includes detailed data such as the date of invoice, the item of invoice, the amount, and whether or not the item was inspected.

[1382] "Means for input or upload" refers to the functionality of the interface for a user to manually enter data or provide an existing data file to the system.

[1383] "Means of analyzing stored past inspection data and learning data features and patterns" refers to the functionality of machine learning algorithms that analyze past inspection data, learn specific patterns and features, and build criteria for judging future data.

[1384] "New Claims Data" means data containing information about new claims that are currently or will be made in the future.

[1385] "Means for entering or uploading new claim data" refers to the functionality of the interface that allows a user to provide new claim data to the system.

[1386] "Means for analyzing new billing data and generating inspection information based on patterns learned from past data" refers to the function of analyzing new billing data, determining whether or not to inspect it based on patterns learned in the past, and generating that information.

[1387] "Means for saving the generated inspection information and providing it to the user" refers to the interface function for saving the inspection results generated by the system and providing the information so that the user can check it.

[1388] A "logistics facility" is a facility used for logistics operations such as receiving, storing, inspecting, and shipping goods.

[1389] "Robots that scan specified products and generate inspection information" refer to autonomous robots that scan barcodes, QR codes, etc. of specified products within a logistics facility and automatically generate inspection information based on that information.

[1390] This invention provides a system for improving the efficiency of inspection work at logistics facilities. The specific configuration and operation of this system will be described below.

[1391] First, the user uses an interface on the terminal to input or upload past inspection data. The user can input the past inspection data manually or upload it as a CSV file. This past inspection data includes information such as the invoice date, invoice item, amount, and whether or not the inspection was successful.

[1392] The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area.Then, it stores this data in a database.

[1393] The server then analyzes the stored past inspection data and uses machine learning algorithms to learn the characteristics and patterns of the data. This analysis and learning is performed using machine learning frameworks such as TensorFlow. Through this learning, criteria for determining whether or not an item can be inspected are built within the server. For example, patterns that indicate "acceptance" are found when "the amount is small" or "specific items are included."

[1394] The user then enters or uploads new billing data through a similar interface. This new data contains the same information as the previous data (billing date, billing item, amount, etc.). The server receives this new data, checks its format, and stores it.

[1395] The server automatically generates inspection information based on newly received billing data using a model learned from past data. For example, based on past learning results, the server can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[1396] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[1397] Furthermore, this system includes a robot that scans designated products at logistics facilities and generates inspection information. The robot scans the products within the logistics facility and sends the data to a server. The server analyzes the data and uses a model trained on past data to determine whether or not the product can be inspected. The generated inspection information is displayed on the robot's display, allowing users to easily check it.

[1398] As a concrete example, consider the case where a user uploads billing data from the past year to the system. This data includes information such as "January 1, 2022, Equipment Purchase, ¥10,000, Acceptable" and "February 1, 2022, Meeting Expenses, ¥20,000, Unacceptable." The server analyzes this data and learns patterns. Subsequently, when new billing data "January 1, 2023, Equipment Purchase, ¥15,000" and "February 1, 2023, Meeting Expenses, ¥30,000" are received, the server generates acceptance information for "January 1, 2023, Equipment Purchase, ¥15,000, Acceptable" and "February 1, 2023, Meeting Expenses, ¥30,000, Unacceptable."

[1399] An example of a prompt is, "Based on the inspection data from the past year, determine whether or not new products can be inspected and record this automatically." In this way, it is possible to improve the efficiency of inspection work at logistics facilities and provide consistent inspection standards.

[1400] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1401] Step 1:

[1402] The user inputs or uploads past inspection data.

[1403] Specifically, users can manually enter past billing data (billing date, billing item, amount, acceptance / acceptance status, etc.) through the terminal interface, or upload a CSV file. The entered data is checked for formatting on the terminal and then sent to the server.

[1404] Input: Past inspection data

[1405] Output: Format-checked past inspection data is sent to the server

[1406] Step 2:

[1407] The server receives and stores past inspection data.

[1408] The server receives the data sent from the terminal, checks the format again, and saves the data in a temporary storage area.Then, it stores this data in a database.

[1409] Input: Format-checked past inspection data

[1410] Output: Past inspection data stored in a database

[1411] Step 3:

[1412] The server analyzes the stored past inspection data and learns the characteristics and patterns of the data.

[1413] The server analyzes past inspection data stored in a database using machine learning frameworks such as TensorFlow to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an inspection is acceptable.

[1414] Input: Past inspection data

[1415] Output: A trained machine learning model

[1416] Step 4:

[1417] A user enters or uploads new claim data.

[1418] Users can manually enter new billing data (billing date, billing item, amount, etc.) through their terminal or upload it as a CSV file, which is also format-checked on the terminal and then sent to the server.

[1419] Input: New claim data

[1420] Output: New format-checked claim data is sent to the server

[1421] Step 5:

[1422] The server receives the new billing data and generates acceptance information.

[1423] The server analyzes newly received billing data using a machine learning model and generates acceptance information based on patterns learned from past data. For example, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be accepted for inspection."

[1424] Input: New claims data, trained machine learning model

[1425] Output: Generated acceptance information

[1426] Step 6:

[1427] The server stores the generated acceptance information and provides it to the user.

[1428] The generated inspection information is stored in the server's database. An interface is then provided so that users can check the inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[1429] Input: Generated acceptance information

[1430] Output: Inspection information stored in the database, interface provided to the user

[1431] Step 7:

[1432] The robot scans the specified product and generates inspection information.

[1433] Robots used in logistics facilities scan the barcodes or QR codes of designated products and send the data to a server. The server analyzes the data and uses machine learning models to determine whether or not the product should be inspected. The inspection results are then displayed on the robot's display.

[1434] Input: Scanned product data

[1435] Output: Inspection information displayed on the robot's display

[1436] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1437] This invention is a system that streamlines inspection work and solves the problem of conventional manual generation of inspection information, and also combines an "emotion engine" that recognizes user emotions and reflects them in the operation of the system. This system operates in conjunction with the server, terminal, and user, automating the entire process from inputting inspection data to analyzing it, generating inspection information, outputting the results, and recognizing user emotions.

[1438] System configuration

[1439] First, the terminal provides an interface for users to input or upload past data. Users can manually input past inspection data or upload it in a CSV file format. This data includes the invoice date, invoice item, amount, and whether or not the data was accepted.

[1440] The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area, after which it stores it in a database.

[1441] Analysis and learning from past data

[1442] The server analyzes the stored past inspection data and uses a machine learning algorithm to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an item can be inspected. For example, if the amount is small or if certain items are included, it will find patterns that indicate "acceptable inspection."

[1443] Receiving and analyzing new claims data

[1444] Users enter or upload new billing data through a similar interface. The new data contains the same information as the previous data (billing date, billing item, amount, etc.). The server receives this new data, checks its format, and stores it.

[1445] Generating acceptance information

[1446] The server automatically generates inspection information based on newly received billing data using a model learned from past data. For example, based on past learning results, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[1447] Saving and providing inspection results

[1448] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[1449] Emotion engine integration

[1450] The present invention further integrates an emotion engine to recognize the user's emotions, which analyzes data such as the user's facial expressions and tone of voice to identify the user's emotional state.

[1451] The emotion engine allows the system to adjust the interface display when a user is stressed or confused, helping them to complete their tasks more smoothly. For example, if a user is in a negative emotional state, the system can display error messages in a gentle tone and provide additional help options. Additionally, if a user feels anxious, the system can send a notification prompting them to reconfirm the inspection information.

[1452] Specific examples

[1453] For example, consider the case where a user uploads billing data and its inspection results from the past year to the system. This past data includes the billing date, billing item, amount, and whether or not inspection was possible. Data such as "January 1, 2022, Equipment Purchase, ¥10,000, Inspection Possible" and "February 1, 2022, Conference Expenses, ¥20,000, Inspection Not Possible" are entered.

[1454] The server analyzes this data and later learns patterns such as "the smaller the amount, the higher the probability of acceptance" and "if the conference fee is high, it will not be accepted."

[1455] When the user enters new billing data "January 1, 2023, equipment purchase, ¥15,000" and "February 1, 2023, conference fee, ¥30,000," the server analyzes this and generates new acceptance information "January 1, 2023, equipment purchase, ¥15,000, accepted" and "February 1, 2023, conference fee, ¥30,000, not accepted."

[1456] Furthermore, if the emotion engine analyzes the user's facial expression data and determines that the user is feeling stressed, the system will display the inspection results in a more friendly and understandable format, providing additional options such as "Do you want to check again?" or "Do you need help?", reducing the burden on the user.

[1457] This system allows users to efficiently perform inspection work and significantly reduces the burden of manual work. In addition, the integration of an emotion engine allows for flexible responses that take into account the user's emotional state, providing excellent flexibility that can be applied even in cases where it is difficult to change the inspection method.

[1458] The processing flow will be explained below.

[1459] Step 1:

[1460] User enters or uploads historical data

[1461] Users can manually enter past inspection data through the system interface or upload it as a CSV file.

[1462] Specifically, the user clicks the "Select File" button to select the CSV file.

[1463] Step 2:

[1464] The server receives and stores the data

[1465] The server receives the past data sent by the user and checks the format and content of the data.

[1466] If the format is correct, the data is stored in a temporary storage area.

[1467] Step 3:

[1468] The server stores the data in a database

[1469] The server stores the data in the database after the format check is complete.

[1470] At this time, checks are also carried out to prevent errors such as data duplication or loss.

[1471] Step 4:

[1472] The server analyzes past data

[1473] The server analyzes past inspection data stored in a database and extracts various categories of data.

[1474] For example, classify data with tags such as "acceptable" and "unacceptable."

[1475] Step 5:

[1476] The server uses a machine learning model to learn the characteristics of the data.

[1477] The server uses machine learning algorithms to learn patterns and characteristics of inspections.

[1478] For example, patterns that determine whether or not a product can be accepted are searched for and saved as a model.

[1479] Step 6:

[1480] A user enters or uploads new claim data

[1481] Users enter new claim data through the system interface or upload it as a CSV file.

[1482] Step 7:

[1483] The server receives the new data and checks the format.

[1484] The server receives the newly submitted billing data and verifies that the data format is correct.

[1485] Step 8:

[1486] The server saves the new data

[1487] The server stores the data in a database after format checking is complete.

[1488] Step 9:

[1489] The server parses the new billing data

[1490] The server retrieves the new billing data and parses the information.

[1491] For example, extract billing items and amounts.

[1492] Step 10:

[1493] The server generates inspection information using the learning model.

[1494] The server uses the trained model to automatically generate an acceptance or rejection decision for new billing data.

[1495] For example, if the purchase of equipment on January 1, 2023 costs 15,000 yen, it is determined that the equipment can be inspected.

[1496] Step 11:

[1497] The server stores the generated inspection results and provides them to the user.

[1498] The server stores the generated inspection information in a database.

[1499] At the same time, the inspection results are displayed to the user on the system interface.

[1500] Step 12:

[1501] Emotion engine recognizes user emotions

[1502] The emotion engine analyzes facial expressions and tone of voice through the user's camera and microphone.

[1503] This identifies the user's emotional state, such as whether they are stressed or confused.

[1504] Step 13:

[1505] The server adjusts the interface based on the emotional state.

[1506] The server adjusts the way the system interface is displayed based on the user's emotional state obtained from the emotion engine.

[1507] For example, if a user is stressed, provide a more friendly and understandable display or additional help options.

[1508] Step 14:

[1509] The user checks, corrects, and approves the inspection results

[1510] The user checks the inspection results through the system interface, makes corrections as necessary, and then approves them.

[1511] The emotion engine will display a notification to encourage reconsideration if the user is feeling anxious.

[1512] Example 2

[1513] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1514] In conventional inspection work, inspection information is generated manually, which requires a lot of time and effort. In addition, because the user's emotional state is not taken into consideration, users often feel stressed when using the system. This reduces work efficiency and increases the risk of mistakes. To solve these problems, a system that automatically generates inspection information and takes the user's emotional state into consideration is needed.

[1515] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1516] In this invention, the server includes a means for inputting or uploading past inspection data, a means for analyzing the stored past inspection data and learning data characteristics and patterns, and a means for inputting or uploading new billing data, which enables automatic generation of inspection information and flexible system display that takes into account the user's emotional state.

[1517] "Past inspection data" is information relating to inspection work carried out in the past, including the date of invoice, invoice item, amount, whether or not inspection was possible, and the like.

[1518] "Input or upload means" means an interface through which a user can manually enter data or upload it to the system in the form of a file.

[1519] The "means for analyzing the stored past inspection data" refers to an algorithm or program for analyzing the past inspection data received by the server and extracting characteristics and patterns of the data.

[1520] "Methods of learning features and patterns in data" is the process of using machine learning algorithms to automatically learn useful information and trends from past data.

[1521] "New billing data" refers to information about a bill that is newly entered or uploaded into the system, including the billing date, billing items, amount, etc.

[1522] The "means for generating inspection information" refers to an algorithm or program that uses learned patterns or models to automatically generate information such as whether or not an item can be inspected from new invoice data.

[1523] The "means for storing the generated acceptance information" is a process in which the server stores the automatically generated acceptance information in a persistent storage device such as a database.

[1524] The "means for providing to the user" is an interface that allows the user to check, correct, and approve the stored acceptance information.

[1525] "Means for recognizing the user's emotions and adjusting the system's behavior" refers to algorithms or programs that analyze the user's facial expressions, tone of voice, etc., and change the system's display method and behavior based on the results.

[1526] This system improves the efficiency of inspection work and adjusts the operation of the system by recognizing the user's emotions. A specific embodiment of this system will be described below.

[1527] First, the user inputs or uploads past inspection data into the terminal. Specifically, past billing data (billing date, billing item, amount, whether inspection was possible, etc.) can be manually input or uploaded in CSV file format. This data is sent to the server through the terminal interface.

[1528] The server receives the data sent from the device and performs a format check. Format check is a process to ensure the data is correct, and invalid data is logged. The data is then saved in a temporary storage area and then permanently stored in a database (e.g., MySQL).

[1529] Next, the server analyzes the stored past inspection data and learns the data's features and patterns. This process uses machine learning algorithms (such as scikit-learn or TensorFlow). The data is loaded into a pandas DataFrame, features are extracted, and the data is split into training datasets. The data is trained using scikit-learn's RandomForestClassifier, and the generated model is saved in pickle format.

[1530] Next, the user enters or uploads new billing data, just like the previous data. The new data includes the billing date, billing item, amount, etc. This is also sent to the server via the terminal.

[1531] The server receives the new data, checks the format again, and saves it to the database. It then uses the saved learning model to generate acceptance information from the new data. Specifically, it inputs the new data into the trained model and uses the model's .predict method to predict whether the data will be accepted or not. The generated acceptance information is saved in the database.

[1532] The generated inspection information is provided to the user as the inspection result. The terminal provides an interface that allows the user to check the inspection result and make corrections or approvals. The user checks the results and makes corrections or approvals as necessary.

[1533] The system also incorporates an "emotion engine" to recognize the user's emotions. The device captures the user's facial expressions and tone of voice using a camera and microphone and sends this data to a server. The server then uses the emotion engine to analyze the user's emotional state and adjusts the system's display accordingly. For example, facial expression recognition is performed using OpenCV and voice analysis is performed using Google Cloud Speech-to-Text. If the system determines that the user is stressed, it displays additional options such as "Would you like to check again?" or "Do you need support?"

[1534] Specific examples

[1535] Consider the case where a user uploads billing data from the past year to the system. The past data includes information such as "January 1, 2022, equipment purchase, 10,000 yen, accepted" and "February 1, 2022, conference fee, 20,000 yen, not accepted." The server analyzes this data and learns patterns such as "the smaller the amount, the higher the probability of acceptance" and "if the conference fee is high, it is not accepted."

[1536] When new billing data "January 1, 2023, Equipment Purchase, ¥15,000" and "February 1, 2023, Conference Expenses, ¥30,000" are entered, the server analyzes this and generates new acceptance information "January 1, 2023, Equipment Purchase, ¥15,000, Accepted" and "February 1, 2023, Conference Expenses, ¥30,000, Not Accepted."

[1537] To achieve this, the following prompts are given to the generative AI model:

[1538] "Analyze the following historical data and generate a prompt to predict whether the new billing data will be accepted: 'January 1, 2022, Equipment Purchase, ¥10,000, Acceptable'; 'February 1, 2022, Conference Expenses, ¥20,000, Unacceptable'. The new billing data is 'January 1, 2023, Equipment Purchase, ¥15,000'."

[1539] This system allows users to efficiently perform inspection work and significantly reduces the manual workload. In addition, by utilizing an emotion engine, it is possible to respond flexibly according to the user's emotional state, providing a seamless user experience.

[1540] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1541] Step 1: User enters or uploads historical data

[1542] Users can input past inspection data into the terminal or upload it in CSV file format. Input information includes the billing date, billing item, amount, and whether or not inspection was possible. Specifically, users can either manually input data using the keyboard or select a CSV file from the file selection dialog and upload it.

[1543] Input: Invoice date, invoice item, amount, and inspection availability data

[1544] Output: Data transfer from device to server

[1545] Step 2: Server receives data and checks format

[1546] The server receives the data sent from the terminal. After receiving it, it performs a format check on the data. The format check is a process to check whether the data is written correctly in JSON format, and invalid data is recorded in a log.

[1547] Input: Data sent from the terminal

[1548] Output: Normalized data after format check, logging of invalid data

[1549] Step 3: Temporarily save the data and store it in the database

[1550] The server saves data that passes the format check in a temporary storage area. It then stores it permanently in a database (such as MySQL). Specific operations include registering the data in the MySQL database with an INSERT statement.

[1551] Input: Normalized data after format check

[1552] Output: Inspection data stored in the database

[1553] Step 4: Data analysis and model training by the server

[1554] The server analyzes the stored past inspection data and uses machine learning algorithms to learn the data's features and patterns. This process involves loading the data into a pandas DataFrame and extracting features. Next, it runs training using scikit-learn's RandomForestClassifier and saves the trained model in pickle format.

[1555] Input: Past inspection data stored in the database

[1556] Output: A trained machine learning model

[1557] Step 5: User enters or uploads new claim data

[1558] The user inputs or uploads new billing data to the terminal in the same way as past data. Input information includes billing date, billing item, amount, etc. Specific operations include manually entering data using the keyboard or uploading a CSV file by selecting it from the file selection dialog.

[1559] Input: New invoice date, invoice item, and amount data

[1560] Output: New data transfer from device to server

[1561] Step 6: Server receives new data and checks format

[1562] The server receives the new billing data and performs another format check to ensure that the data is correctly formatted in JSON, and any invalid data is logged.

[1563] Input: New billing data sent from the terminal

[1564] Output: Normalized data after format check, logging of invalid data

[1565] Step 7: Save the new data to the database

[1566] The server saves the new data that passes the format check to the database, which specifically involves the process of registering the data in the MySQL database with an INSERT statement.

[1567] Input: Normalized data after format check

[1568] Output: New billing data stored in the database

[1569] Step 8: Server analyzes new data and generates acceptance information

[1570] The server analyzes newly received data using the existing learning model and automatically generates inspection information. The new data is input into the trained model and the .predict method of the model is used to predict whether the data can be inspected or not.

[1571] Input: New claims data, trained machine learning model

[1572] Output: Automatically generated inspection information

[1573] Step 9: Save the generated acceptance information and provide it to the user

[1574] The server stores the generated inspection information in a database and provides an interface for users to view it. Specifically, it stores the inspection results in a MySQL database and displays the results on the dashboard of the web application.

[1575] Input: Automatically generated acceptance information

[1576] Output: Inspection information stored in the database, user interface for checking inspection results

[1577] Step 10: Analyze user emotions using the emotion engine and adjust the interface

[1578] The device captures the user's facial expressions and tone of voice using a camera and microphone and sends this data to a server. The server then uses an emotion engine to analyze the user's emotional state and adjusts the system's display accordingly. Specifically, it uses OpenCV for facial recognition, Google Cloud Speech-to-Text for speech analysis, and changes the display of UI components based on the results.

[1579] Input: User's facial expression data, voice data

[1580] Output: Sentiment analysis results, adjusted interface display

[1581] (Application example 2)

[1582] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1583] Conventional inspection work is often done manually, which is time-consuming and labor-intensive, resulting in inefficiency. Therefore, there is a demand for automation of the generation of inspection information. However, to further improve work efficiency and reduce worker stress and confusion, conventional systems lack the functionality to consider the user's emotional state. Therefore, the objective of this invention is to provide a flexible operation interface that takes into account the worker's emotional state while improving the efficiency of inspection work.

[1584] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting or uploading past inspection data, means for analyzing the saved past inspection data and learning data characteristics and patterns, means for inputting or uploading new billing data, means for analyzing the new billing data and generating inspection information based on patterns learned from the past data, means for saving the generated inspection information and providing it to the user, and means for recognizing the user's emotional state and adjusting the interface. This makes it possible to improve the efficiency of inspection work and reduce stress and confusion for the worker by using a flexible operation interface that takes the worker's emotional state into consideration.

[1585] "Past inspection data" refers to data related to inspections, such as previously received billing information and inspection results.

[1586] "Input or Upload Means" refers to an interface or mechanism through which a User manually enters data or uploads data in the form of a file.

[1587] "Means for analyzing stored past inspection data" refers to algorithms or systems that use accumulated past data to recognize and analyze its features and patterns.

[1588] "Means of learning data features and patterns" refers to the process of applying machine learning algorithms to stored data to find criteria and patterns for acceptance or rejection and build a model.

[1589] "Means for analyzing new claims data" refers to algorithms or systems that analyze the latest claims data provided by users and generate acceptance information using models learned from past data.

[1590] "Means for generating inspection information" refers to a system that uses past learning results based on new billing data to determine whether or not inspection is possible, and automatically creates inspection information based on that.

[1591] "Means for storing the generated inspection information and providing it to the user" refers to an interface or mechanism for storing the generated inspection information in a database, etc., and providing that information so that the user can check, modify, and approve it.

[1592] "Means for recognizing the user's emotional state and adjusting the interface" refers to a mechanism that analyzes data such as the user's facial expressions and voice to recognize their emotions, and changes the interface display method and operation guide according to the results.

[1593] "System" refers to a program or device with a set of functions designed to streamline inspection work, including the above means.

[1594] The present invention is a system that improves the efficiency of inspection work at logistics centers and provides a flexible interface that takes into account the emotional state of the user. This system operates in conjunction with the server, terminals, and users.

[1595] First, an interface is provided on the terminal side for users to input or upload past inspection data. Users can input past inspection data manually or upload it in a format such as a CSV file. This data includes the billing date, billing item, amount, and whether or not the data was inspected. The server receives the data sent from the terminal, checks the format, and saves it in a temporary storage area. It is then stored in the database.

[1596] Next, the server analyzes the stored past inspection data and uses a machine learning algorithm to learn the characteristics and patterns of the data. Through this learning, the server builds criteria for determining whether or not an item can be inspected. For example, it finds patterns that indicate "acceptable" when the amount is small or when certain items are included.

[1597] The user enters or uploads new billing data through a similar interface. This new data also contains the same information as the past data (billing date, billing item, amount, etc.). The server receives this new data, checks the format, and saves it. Next, the server automatically generates inspection information based on the newly received billing data using a model learned from past data. For example, based on past learning results, the server can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be inspected." This judgment is made using a machine learning model, providing highly accurate results.

[1598] The generated inspection information is stored in a database on the server. An interface is then provided so that users can check these inspection results through their terminals. Users can check the results and make corrections or approvals as necessary.

[1599] Furthermore, the present invention integrates an emotion engine to recognize the user's emotions. The emotion engine analyzes data such as the user's facial expressions and tone of voice to identify the user's emotional state. If the user feels stressed or confused, the emotion engine can adjust the interface display to help the user perform their tasks more smoothly. For example, if the user is in a negative emotional state, the system can display an error message in a gentle tone and provide additional help options. Furthermore, if the user feels anxious, the system can send a notification prompting the user to reconfirm the inspection information.

[1600] As a concrete example, consider the case where a user uploads billing data and its inspection results from the past year to the system. This past data includes the billing date, billing item, amount, and whether or not inspection was possible. Data such as "January 1, 2022, equipment purchase, 10,000 yen, inspection possible" and "February 1, 2022, conference expenses, 20,000 yen, inspection impossible" are input. The server analyzes this data and later learns patterns such as "the smaller the amount, the higher the probability of inspection being possible" and "if conference expenses are expensive, inspection is impossible." When the user enters new billing data "January 1, 2023, equipment purchase, ¥15,000" and "February 1, 2023, conference fee, ¥30,000," the server analyzes this and generates new acceptance information "January 1, 2023, equipment purchase, ¥15,000, accepted" and "February 1, 2023, conference fee, ¥30,000, not accepted."

[1601] Furthermore, if the emotion engine analyzes the user's facial expression data and determines that the user is stressed, the system displays the inspection results in a more user-friendly and understandable format. For example, it provides additional options such as "Do you want to check again?" or "Do you need assistance?", reducing the burden on the user. This system allows users to perform inspection work efficiently and significantly reduces the burden of manual work. In addition, the integration of the emotion engine allows for flexible responses that take into account the user's emotional state, providing excellent flexibility that can be applied even in cases where it is difficult to change the inspection method.

[1602] An example of a prompt could be, "We are creating an application that recognizes user emotions. If a worker feels stressed or confused, how should the system change the interface? For example, by showing options such as 'Do you want to check again?' or 'Do you need help?'"

[1603] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1604] Step 1:

[1605] Users input or upload past inspection data. They can manually enter data using the terminal interface or upload data in a format such as a CSV file. This sends information such as the billing date, billing item, amount, and whether or not the inspection was successful to the server as input data.

[1606] Step 2:

[1607] The server receives past inspection data sent from the terminal and performs a format check. If the format is correct, it is saved in a temporary storage area and then stored in a database. In the format check process, the accuracy of the data format is confirmed and inaccurate data is excluded.

[1608] Step 3:

[1609] The server analyzes past inspection data stored in a database. It uses a machine learning algorithm to learn the characteristics and patterns of the data. For example, as criteria for determining whether an item can be inspected, it can find patterns such as "the smaller the amount, the higher the probability that it can be inspected" or "if certain items are included, it cannot be inspected." The input for this process is the stored data, and the output is a learning model.

[1610] Step 4:

[1611] The user also inputs or uploads new billing data through the terminal interface, which also includes the billing date, billing item, amount, etc. This becomes the input data and is sent to the server.

[1612] Step 5:

[1613] The server receives the new claim data, performs format checks and stores it in the database. Once the format check is complete, the new claim data is stored in the database. This data is used as input for subsequent analysis.

[1614] Step 6:

[1615] The server analyzes newly received billing data and automatically generates acceptance information using a machine learning model learned from past data. For example, it can reach a conclusion such as "If the purchase of equipment on January 1, 2023 costs 15,000 yen, it can be accepted for inspection." The input data is the new billing data, and the output is the generated acceptance information.

[1616] Step 7:

[1617] The server stores the generated inspection information in a database and then provides the information via a terminal interface for the user to review. The user can review the inspection results and make corrections or approvals as necessary. The input is the generated inspection information, and the output is the information provided to the user.

[1618] Step 8:

[1619] The emotion engine recognizes the user's emotional state in real time by analyzing the user's facial expressions and tone of voice using the smartphone's camera and microphone. The input data is camera footage and audio data, and the output is the identified emotional state.

[1620] Step 9:

[1621] The server adjusts the interface based on the emotional state recognized by the emotion engine. For example, if the user is determined to be stressed, the server displays a more user-friendly error message and offers additional help options. Alternatively, if the user is anxious, the server sends a notification prompting the user to reconfirm the acceptance information. The input is the identified emotional state, and the output is the adjusted interface display.

[1622] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1623] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1624] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1625] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1626] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1627] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1628] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1629] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1630] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1631] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1632] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1633] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1634] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1635] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1636] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1637] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1638] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1639] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1640] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1641] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1642] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1643] The following is further disclosed regarding the above embodiment.

[1644] (Claim 1)

[1645] A means to input or upload past inspection data;

[1646] A means of analyzing stored past inspection data and learning data characteristics and patterns;

[1647] A means to enter or upload new claim data;

[1648] means for analyzing new billing data and generating acceptance information based on patterns learned from past data;

[1649] A means for storing the generated acceptance information and providing it to a user;

[1650] A system including:

[1651] (Claim 2)

[1652] 2. The system according to claim 1, wherein the means for learning past inspection data uses an algorithm for extracting and classifying categorical data of whether or not the product is acceptable for inspection.

[1653] (Claim 3)

[1654] 2. The system of claim 1, wherein the means for analyzing new billing data and generating acceptance information automatically generates acceptance results using a machine learning model.

[1655] "Example 1"

[1656] (Claim 1)

[1657] A means for receiving past inspection data, performing a format check, and storing the data in a temporary storage area;

[1658] A means for analyzing the stored past inspection data and storing it in a database;

[1659] A means of analyzing stored past inspection data and learning data features and patterns using machine learning algorithms;

[1660] a means for receiving new claim data, format checking it, and storing it in a database;

[1661] means for analyzing new billing data and generating acceptance information based on patterns learned from past data;

[1662] A means for storing the generated inspection information in a database and providing it so that the user can check it through a terminal;

[1663] A system including:

[1664] (Claim 2)

[1665] 2. The system according to claim 1, wherein the means for learning past inspection data uses a machine learning algorithm to establish criteria for determining whether or not an inspection is acceptable.

[1666] (Claim 3)

[1667] 2. The system of claim 1, wherein the means for analyzing new billing data and generating acceptance information automatically generates acceptance results using a machine learning model.

[1668] "Application Example 1"

[1669] (Claim 1)

[1670] A means to input or upload past inspection data;

[1671] A means of analyzing stored past inspection data and learning data characteristics and patterns;

[1672] A means to enter or upload new claim data;

[1673] means for analyzing new billing data and generating acceptance information based on patterns learned from past data;

[1674] A means for storing the generated acceptance information and providing it to a user;

[1675] a means for using a robot at a logistics facility to scan designated products and generate inspection information;

[1676] A system including:

[1677] (Claim 2)

[1678] 2. The system according to claim 1, wherein the means for learning past inspection data uses an algorithm for extracting and classifying categorical data of whether or not the product is acceptable for inspection.

[1679] (Claim 3) 【1...

Claims

1. A means to input or upload past inspection data; A means of analyzing stored past inspection data and learning data characteristics and patterns; A means to enter or upload new claim data; means for analyzing new billing data and generating acceptance information based on patterns learned from past data; A means for storing the generated acceptance information and providing it to a user; A system including:

2. 2. The system according to claim 1, wherein the means for learning past inspection data uses an algorithm for extracting and classifying category data of whether or not the product is acceptable for inspection.

3. 2. The system of claim 1, wherein the means for analyzing new claim data and generating acceptance information automatically generates acceptance results using a machine learning model.

Images