system

Abstract

We provide the system. [Solution] A natural language processing means that analyzes user input information to generate structured data, A command generation means that generates appropriate inventory management commands based on analyzed data, A means of updating inventory information by updating a database, A notification mechanism for informing users of the generated results and reports, A voice recognition means that recognizes voice data and extracts operation commands, A video analysis means that analyzes video data of captured items and integrates it with inventory information, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Manual work in item management has problems such as many mistakes and low efficiency. In particular, duplicate purchases and losses of items frequently occur, and the inventory-taking work takes a great deal of time and cost, so it is required to solve these problems. Furthermore, real-time management of inventory and optimal processing proposals for long-unused items are also unsolved problems.

Means for Solving the Problems

[0005] This invention employs natural language processing means to analyze natural language input from users and extract necessary information as structured data. Furthermore, it includes command generation means to automate inventory management based on the analyzed data, and inventory update means to update the database, thereby improving work efficiency and accuracy. In addition, it provides notification means to immediately inform users of the generated results and reports, centrally managing all item management processes. This enables the prevention of item loss, deterrence of duplicate purchases, real-time inventory management, and provides appropriate methods for handling items that have been unused for extended periods.

[0006] "Natural language processing means" refers to technologies that analyze user input and convert linguistic data into structured data using machine learning models.

[0007] The "command generation means" is a component that generates appropriate commands for inventory management based on the analyzed input data.

[0008] A "stock update mechanism" is a function that operates a database according to instructions to keep the inventory information of goods up to date.

[0009] A "notification means" is a communication function that immediately conveys processing results or command details to the user or relevant parties.

[0010] A "visual data integration system" is a system that uses image analysis technology to analyze the visual information of items and import it into an inventory database. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0017] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] This invention provides an automated inventory system aimed at improving the efficiency of inventory management. The implementation method and specific operation of the system are described below.

[0033] First, when a user performs an action related to an item (e.g., new registration, borrowing, return), they input their request in natural language. In response, the terminal sends the user's instructions to the server. The server is equipped with natural language processing capabilities and analyzes the input natural language text, converting it into structured data. This data includes the item name, quantity, and action details.

[0034] Next, the server uses a command generation mechanism to generate specific commands for inventory management based on the analyzed information. These commands include, for example, updating inventory quantities, registering items on the loan list, and adding new items.

[0035] Subsequently, the inventory update mechanism updates the database inventory information based on the instructions. This ensures accurate inventory information in real time, avoiding manual errors and duplicate management.

[0036] Furthermore, results and, if necessary, reports are generated and returned to the user through notification channels. For example, users may be notified when a new item is registered or when a lending operation is completed.

[0037] As an additional feature, users can take pictures of items with their devices and send them to the server. The server uses visual data integration to analyze the images and integrate the item information into a database. This automates the visual management of items.

[0038] For example, if a user enters "I want to borrow two projectors," the server identifies the item and quantity as projectors, instructs the lending process, and appropriately reduces the inventory by two units. Furthermore, a report is generated as needed and notified to the user, allowing them to verify the entire operation.

[0039] The above describes the embodiments for carrying out this invention and the method for achieving the objective of the invention, which is to improve the efficiency and accuracy of inventory management.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The user uses a terminal to input requests for operations related to items in natural language. For example, they might input the instruction, "Add 20 notes."

[0043] Step 2:

[0044] The terminal sends the entered data to the server. The transmitted data retains the user's instructions exactly as they were given.

[0045] Step 3:

[0046] The server analyzes the received data using natural language processing. Based on the analysis, it extracts the item name "notebook" and quantity "20" from the instructions and saves this as structured data.

[0047] Step 4:

[0048] The server uses a command generation mechanism to generate specific inventory management commands based on the analyzed data. For example, a command such as "Add 20 notebooks to inventory" is generated.

[0049] Step 5:

[0050] The server, using the inventory update mechanism, accesses the database and updates the inventory information based on the generated instructions. In this case, the number of notes increases by 20.

[0051] Step 6:

[0052] The server checks the results after the update and generates a message to notify the user. This message includes confirmation that the operation was completed successfully.

[0053] Step 7:

[0054] The terminal receives a notification message from the server and displays the result to the user. The user confirms that their instructions were processed correctly.

[0055] Step 8:

[0056] When a user takes a picture of an item and provides it, the device sends the image data to the server.

[0057] Step 9:

[0058] The server utilizes visual data integration technology to analyze received image data and integrate relevant information into the inventory database. During this process, the appearance and model number of items are included as supplementary data.

[0059] (Example 1)

[0060] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0061] In inventory management, manual inventory management hinders operational efficiency and is prone to errors and duplicate entries. Furthermore, the lack of automated methods for visually managing inventory makes it difficult to effectively grasp inventory levels.

[0062] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0063] In this invention, the server includes means for analyzing input natural language information to generate structured data, means for generating specific inventory management commands based on the analyzed structured data, and means for updating the inventory information in the database based on the commands. This enables more efficient operations and accurate inventory management. Furthermore, by utilizing visual data, it becomes possible to make the management of goods more comprehensive and automated.

[0064] "Natural language information" refers to linguistic data entered by users in the language they use on a daily basis, and is information that does not depend on specific rules or codes.

[0065] "Structured data" refers to data that is organized according to a specific format and made easily accessible and processable.

[0066] An "inventory management directive" is a set of specific instructions generated to manage inventory levels, and its contents include matters such as increasing or decreasing inventory and updating records.

[0067] A "database" is a means of storing information designed to efficiently store, search, and update large amounts of data.

[0068] "Notification means" refers to methods or devices for communicating generated information or results to the user, and includes various forms such as alerts and messages.

[0069] "Visual data" refers to data in a format that humans can perceive through their vision, such as images and videos.

[0070] "Visual data integration means" refers to methods or devices for analyzing visual information and incorporating it into existing data systems.

[0071] This invention comprises an automated inventory system that improves the efficiency and accuracy of inventory management. At the core of this system is the collaboration between the user, terminal, and server. The user begins by inputting instructions regarding inventory management into the terminal using natural language. In this process, natural language processing utilizing a generative AI model plays a crucial role.

[0072] When a user's instructions are received by the terminal, the terminal sends them to the server. The server uses its internal natural language processing capabilities to analyze these instructions and convert them into structured data. This structured data includes the item name, quantity, and operation details. The software used by the server includes an AI model for natural language processing.

[0073] Based on structured data, the server generates inventory management directives. These directives are used to update the inventory information in the database in real time. Specific software and hardware components include database systems and inventory management software. The server thus avoids manual errors and duplicate management, ensuring accurate inventory data.

[0074] Furthermore, the server uses a visual data integration mechanism to analyze images of items taken by the user with their device and integrates the visual data into a database. This function also allows for the management of visual item information.

[0075] As a concrete example, when a user enters "I want to borrow two projectors" into the terminal, the server identifies the item name "projector" and the quantity, and then performs an operation to reduce the inventory by two. An example of a prompt in this system is natural language input such as "I want to borrow two projectors." The system can automatically proceed with a series of processes based on this prompt.

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] The user enters instructions for managing the items into the terminal using natural language. An example of input is a prompt message such as, "I would like to borrow two projectors." This input is received by the terminal's interface and sent to the server.

[0079] Step 2:

[0080] The server analyzes the user's instructions received using natural language processing. This process uses a generative AI model to analyze the input natural language and identify item names, quantities, and operation details. Once the analysis is complete, this information is output as structured data.

[0081] Step 3:

[0082] Based on the structured data obtained through analysis, the server generates specific inventory management commands using a command generation mechanism. For example, this might include a command such as "rent out two projectors." These generated commands are prepared as output for inventory updates.

[0083] Step 4:

[0084] The server uses an inventory update mechanism to update the database based on instructions. Specifically, it reduces the quantity of items available for loan and saves the updated inventory information to the database. This ensures that accurate inventory status is maintained in real time.

[0085] Step 5:

[0086] The server generates operation results and necessary reports, and notifies the user through a notification system. For example, a message such as "The loan of two projectors is complete" is sent to the terminal. The notification allows the user to confirm the completion of the operation.

[0087] Step 6:

[0088] The user takes an image of an item using a device and sends it to the server. This data is passed to the server as additional input. The server analyzes the image using a visual data integration system and integrates the item information into a database. This also achieves visual data management.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] Current inventory management systems require users to manually input data when issuing operation instructions, a process prone to errors. Furthermore, inventory information is often not updated in real time, making it difficult to accurately check inventory levels. Additionally, visual verification of items relies on manual processes, leading to reduced efficiency.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes a natural language processing means for analyzing user input information and generating structured data, a speech recognition means for recognizing voice data and extracting operation commands, and a video analysis means for analyzing video data of captured items and integrating it with inventory information. This makes it possible for users to issue operation commands by voice and for inventory information to be updated immediately. In addition, the visual confirmation of items is automated through video analysis, improving management efficiency.

[0094] "Natural language processing means" refers to a device or program that analyzes natural language input information from a user and converts it into structured data suitable for data processing.

[0095] A "command generation means" is a device or program for automatically generating commands necessary for inventory management based on structured data.

[0096] An "inventory update means" is a device or program that has the function of accurately updating the inventory information in a database based on the generated instructions.

[0097] "Notification means" refers to a device or program that has the function of communicating inventory information or the results of instructions to the user.

[0098] "Voice recognition means" refers to a device or program that analyzes voice data, converts its content into text data, and extracts operation commands based on this.

[0099] "Video analysis means" refers to a device or program that processes video data of an item that has been photographed, extracts information about the item based on that data, and integrates it with inventory information.

[0100] This invention provides a system that allows users to patrol a logistics center and issue voice commands using smart glasses. The server utilizes speech recognition technology to convert voice data from the user into text. This process uses speech recognition APIs such as Google® Cloud Speech-to-Text or Amazon Transcribe. The converted text is analyzed by a natural language processing system to generate structured data. This data extracts information about the type and quantity of items, and the command generation means creates inventory management commands. The commands are reflected in the database by the inventory update means, and the latest inventory status is updated in real time.

[0101] Furthermore, video data of items captured through the camera installed in the smart glasses is analyzed on the server side using video analysis APIs such as Google Cloud Vision and Amazon Rekognition. Based on this video analysis, the characteristics and condition of the items are understood and integrated with inventory information. Based on this integrated data, appropriate inventory status and reports are provided to the user through notification means. For example, if a user gives a voice command such as "Check the inventory of pallet A," the system can retrieve the inventory information for pallet A and display it on the glasses' display. As a result, inventory management in the logistics field can be performed quickly and accurately.

[0102] As a concrete example, consider a system that uses smart glasses to manage inventory using voice and images within a logistics center. For instance, if a user gives the instruction, "Take pictures of all inventory in section B and create a report," all items will be automatically listed, an inventory report will be generated, and displayed on the screen. This process can be streamlined using a generative AI model, and suggestions that are useful for log management and anomaly detection can also be made. An example of a prompt would be, "Please explain the benefits of a system that uses smart glasses to manage inventory using voice and images within a logistics center."

[0103] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0104] Step 1:

[0105] The user issues voice commands through smart glasses. The input is the user's voice data, acquired through the smart glasses' microphone. The output is voice data sent to the server.

[0106] Step 2:

[0107] The server uses a speech recognition API to convert the acquired audio data into text. The input is audio data, and the output is text data. Specifically, it uses Google Cloud Speech-to-Text for speech recognition.

[0108] Step 3:

[0109] Based on text data, a natural language processing system performs analysis to generate structured data. The input is text data, and the output is structured data containing information necessary for inventory management. Specifically, the system extracts information such as item names and quantities from the text.

[0110] Step 4:

[0111] The server uses a command generation mechanism to generate inventory management commands based on structured data. The input is structured data, and the output is command data for updating inventory. Specifically, it identifies the operations to be performed, such as adding or removing inventory.

[0112] Step 5:

[0113] The server updates the inventory information in the database through an inventory update mechanism. The input is command data, and the output is the updated inventory information. Specifically, this involves updating records in the inventory table using a database system (e.g., MySQL®).

[0114] Step 6:

[0115] The system transmits video data of objects captured by the user's smart glasses to a server. The input is the video data captured by the user, and the output is the video data sent to the server.

[0116] Step 7:

[0117] The server uses a video analysis API to extract information about items based on video data and integrate it into inventory information. The input is video data, and the output is integrated data including item information. Specifically, this includes the process of identifying item model numbers and quantities using Google Cloud Vision.

[0118] Step 8:

[0119] The server returns the final results to the user through a notification system. The input is integrated inventory information and reporting data, and the output is a display on the user's screen. Specifically, a display process is performed to provide visual feedback on the situation to the user's smart glasses.

[0120] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0121] This invention provides a system that enables user-based interaction by incorporating an emotion engine into item management. This system is configured to analyze the user's natural language input and simultaneously detect the user's emotions from that input, and reflect them in the item management operation flow.

[0122] First, the user enters a request for items in natural language on the terminal. For example, a request might be, "I'm in a bit of a hurry, please register 10 laptops right now." This input is immediately sent to the server and parsed by natural language processing.

[0123] In parallel, the server utilizes an emotion engine to extract emotions such as "urgent" or "confused" from user input. This emotion information influences inventory management commands generated by the system, for example, by setting priorities that allow for faster processing.

[0124] The server uses a command generation mechanism to create commands that take into account the user's requests and emotions, and updates the database inventory information using an inventory update mechanism. In this process, the user is notified with emotion-based, customized messages. For example, feedback such as, "You seem to be in a hurry. We have prioritized your laptop registration," is sent immediately.

[0125] As a concrete example, if a user instructs, "I have some leeway, so please prepare five projectors by next week," the server's emotion engine will recognize this as "leeway" and adjust the project accordingly. If inventory is insufficient, it will generate a suggestion-based notification and present the user with alternative solutions.

[0126] This enables inventory management tailored to the user's specific situation and psychological state, leading to increased operational efficiency and improved user satisfaction. In particular, this invention contributes to customizing the user experience and improving the accuracy of inventory management.

[0127] The following describes the processing flow.

[0128] Step 1:

[0129] The user uses a terminal to input requests for items in natural language. For example, they might input a request such as, "I would like to reserve 10 tablets for next week's meeting."

[0130] Step 2:

[0131] The terminal sends user input to the server. This data contains the user's request exactly as it was submitted.

[0132] Step 3:

[0133] The server analyzes the received data using natural language processing. This extracts structured information from the request, such as "tablet," "10 units," and "reservation."

[0134] Step 4:

[0135] The server uses an emotion engine to analyze emotional information from the user's input text. In this case, from the phrase "for the meeting," it infers the importance and necessity of the task and extracts emotions such as "expectation" and "sense of responsibility."

[0136] Step 5:

[0137] Based on extracted information and sentiment data, the server utilizes a command generation mechanism to create optimal inventory management commands. In this example, a command is created to check the inventory status of tablets and secure them.

[0138] Step 6:

[0139] The server updates inventory information in the database based on instructions using inventory update mechanisms. Specifically, it performs temporary reservation of tablets and future allocations.

[0140] Step 7:

[0141] The server generates a customized message based on the processing results and emotions, and notifies the user. For example, it might send a message such as, "We support the success of your meeting. Your tablet reservation has been secured."

[0142] Step 8:

[0143] The terminal receives notifications sent from the server and displays the results to the user. The user can then confirm that their request has been processed correctly and proceed with confidence.

[0144] Step 9:

[0145] If requested by the user, additional images of the items can be taken with the device and sent to the server.

[0146] Step 10:

[0147] The server uses a visual data integration mechanism to integrate the received image data with inventory information. If necessary, it records detailed information about the items in a database.

[0148] (Example 2)

[0149] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0150] Traditional inventory management systems have faced challenges in improving user experience and efficient inventory management because they operate without considering the user's emotions or psychological state. Furthermore, there was a lack of feedback tailored to individual user needs, leading to a desire for increased satisfaction.

[0151] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0152] In this invention, the server includes a natural language processing means for analyzing user input information and generating structured information, an emotion detection means for extracting emotions from the analyzed information, and a command generation means for generating inventory management commands based on the extracted emotions and input information. This enables efficient item management based on user emotions and the provision of individually customized feedback.

[0153] "Natural language processing means" refers to a function that analyzes user input information and converts it into easily understandable structured information.

[0154] An "emotion detection method" is a function that analyzes the emotions contained in the user's input text and extracts the emotional state.

[0155] The "command generation means" is a function that processes the generation of optimal inventory management commands based on the user's requests and extracted emotions.

[0156] The "update mechanism" refers to a function for updating and managing the inventory information of a data storage device in a timely manner.

[0157] A "notification method" is a function that communicates generated feedback and system processing results to the user.

[0158] "Visual information integration means" refers to a function that integrates visual information of items acquired using image analysis technology into inventory information.

[0159] This invention is a system for realizing dynamic item management based on user emotions. First, the user uses a terminal to input item management requests in natural language. This input is immediately sent to the server.

[0160] The server utilizes natural language processing (NLP) tools to analyze user input and generate structured information. This NLP can utilize open-source libraries or cloud-based NLP services. Examples include libraries such as SpaCy and NLTK.

[0161] Simultaneously, the server uses emotion detection tools to extract the user's emotions from the analyzed information. This process can utilize, for example, Google Cloud's Natural Language API. If the emotion is detected as "urgent," this information will influence the subsequent command generation.

[0162] The server uses a command generation mechanism to generate inventory management commands based on the analyzed data and extracted sentiment. These commands are used to update inventory information via a data storage device.

[0163] Furthermore, the server uses notification methods to communicate processing results and feedback to the user via the terminal. Personalized feedback based on emotions improves the user experience. For example, the user might be notified with a message such as, "It seems you're in a hurry. We've prioritized registering the 10 items you specified."

[0164] For example, if a user enters "I need 10 desktop PCs by the end of next month," the server will recognize that there is "plenty of time" and adjust the project accordingly. Another example of a prompt message could be "Please register 50 cables urgently."

[0165] In this way, it becomes possible to manage items based on user emotions, achieving both increased operational efficiency and improved user satisfaction.

[0166] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0167] Step 1:

[0168] Users input requests regarding inventory management in natural language through a terminal. For example, they might input, "I want to register five new printers as soon as possible." This input data is structured as natural language text.

[0169] Step 2:

[0170] The terminal receives input from the user and sends that raw data directly to the server. The output here is a direct relay of the user's text input.

[0171] Step 3:

[0172] The server uses natural language processing to analyze the received text data. The input is the user's natural language text, and the output is the structured information of that text. Specifically, tokenization and syntactic analysis are performed.

[0173] Step 4:

[0174] The server uses emotion detection to extract the user's emotions from structured information. The input is structured information, and the output is emotion tags such as "urgent" and "relieved." The process includes the specific operation of analyzing emotion indicators in the input text using an emotion engine.

[0175] Step 5:

[0176] The server uses a command generation mechanism to generate inventory management commands based on structured information and sentiment tags. The input is structured information and sentiment tags, and the output is specific command information for inventory updates. In terms of operation, it sets priorities and generates commands for high-priority tasks to be processed quickly.

[0177] Step 6:

[0178] Based on this instruction, the server updates the inventory information of the data storage device using an update mechanism. The input is the instruction information, and the output is the updated inventory information. Specific operations include connecting to the database and performing the modifications.

[0179] Step 7:

[0180] The server notifies the user's terminal of the processing results and sentiment-based feedback via a notification system. The input is message information based on the processing results and sentiment, and the output is the feedback message provided to the user. Specifically, it generates a message such as, "It seems you are in a hurry. We have registered the specified printer as a priority," and sends it to the user.

[0181] (Application Example 2)

[0182] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0183] Existing inventory management systems lack the functionality to adjust workflows based on user emotions, making it difficult to process urgent tasks quickly. Furthermore, the lack of customization for the user experience hinders efforts to improve user satisfaction.

[0184] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0185] In this invention, the server includes a natural language processing means that analyzes user input information to generate structured data, an emotion analysis means that detects the user's emotional state and adjusts the priority of the work flow based on that emotion, and an inventory update means that updates the database to manage inventory information. This enables faster work and improved user satisfaction.

[0186] "User input information" refers to information such as instructions and inquiries regarding goods and tasks, entered in natural language.

[0187] "Natural language processing means" refers to technical means that analyze natural language input by a user and formalize it as structured data.

[0188] The "command generation means" is a function that automatically generates necessary inventory management and work instructions based on the analyzed data.

[0189] A "database" is an information infrastructure that effectively organizes, stores, and communicates inventory information and related data as needed.

[0190] A "stock update mechanism" is a function that updates the inventory information in the database according to the generated command, accurately reflecting the current status of goods.

[0191] "Emotion analysis means" refers to technology that detects the emotional state from the user's natural language input and appropriately adjusts the priority of business processes based on that.

[0192] A "notification method" is a mechanism for communicating information such as generated analysis results, inventory management status, and user feedback to users.

[0193] The system used to implement this application primarily operates in a network environment including a server and mobile devices. The server instantly receives information input from the user in natural language and performs analysis using natural language processing libraries such as Python's spaCy and NLTK. This analysis identifies what kind of item the user's input relates to and generates the necessary instructions.

[0194] In parallel, the server uses sentiment analysis tools such as the Google Cloud Natural Language API and IBM Watson® Tone Analyzer to detect the user's emotional state. For example, it extracts emotional elements such as "urgent" or "request" from the input text and analyzes how these affect logistics priorities. Based on this sentiment analysis, the server dynamically adjusts the priorities of the workflow, processing tasks that require a quick response first.

[0195] The terminal functions as an interface with the user and receives feedback from the server. The terminal instantly displays customized notifications to the user, providing real-time information on work progress and problem-solving. This feedback includes sentiment-based messages tailored to the user's input, ensuring that the user is receiving appropriate responses to their needs and circumstances.

[0196] For example, if a staff member asks "Is there any way to make it work?" on a day when a delivery schedule for a large appliance is set, the system will provide feedback such as, "Yes, it seems urgent. We have readjusted the shipping schedule and arranged for pickup to be completed."

[0197] Furthermore, an example of a prompt message for the generating AI model is: "You are the inventory management system for a logistics center. Analyze user input, understand their emotions, and generate the most appropriate work instructions. For example, if given the dialog 'Please register 10 laptops as soon as possible,' set the priority high and provide a fast registration process." In this way, it is possible to realize emotion-based adaptive inventory management.

[0198] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0199] Step 1:

[0200] The user inputs information into the terminal using natural language. This information may include instructions or inquiries regarding the receiving, dispatching, or delivery of goods. This input serves as the initial data for proceeding to the next step.

[0201] Step 2:

[0202] The terminal sends input information to the server. The server uses Python's spaCy or NLTK to analyze the input natural language and generate structured data. The input is natural language that humans can understand, and the output is the analyzed structured data. This process is the foundation for identifying the items and quantities of the input.

[0203] Step 3:

[0204] The server uses the Google Cloud Natural Language API and IBM Watson Tone Analyzer to perform sentiment analysis based on the analyzed data. In this step, it detects the emotional state from the user's input and determines urgency and priority. The input is structured data, and the output is metadata containing sentiment information.

[0205] Step 4:

[0206] The server uses a command generation system to generate necessary inventory management commands based on analysis results and sentiment information. These commands are prioritized and clearly indicate tasks requiring immediate attention. The input is structured data with added sentiment information, and the output is specific work commands including priorities.

[0207] Step 5:

[0208] The database is updated on the server, keeping inventory information up-to-date. This process is carried out based on instructions and reflects the current location and status of items.

[0209] Step 6:

[0210] The server sends notifications to the terminal. These notifications include feedback based on the processing status of the commands received by the user and their emotions. The input is the generated commands and emotion feedback information, and the output is a customized notification for the user. The terminal displays the received information to the user, allowing the user to see the system's operation in real time.

[0211] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0212] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0213] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0214] [Second Embodiment]

[0215] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0216] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0217] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0218] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0219] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0220] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0221] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0222] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0223] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0224] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0225] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0226] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0227] This invention provides an automated inventory system aimed at improving the efficiency of inventory management. The implementation method and specific operation of the system are described below.

[0228] First, when a user performs an action related to an item (e.g., new registration, borrowing, return), they input their request in natural language. In response, the terminal sends the user's instructions to the server. The server is equipped with natural language processing capabilities and analyzes the input natural language text, converting it into structured data. This data includes the item name, quantity, and action details.

[0229] Next, the server uses a command generation mechanism to generate specific commands for inventory management based on the analyzed information. These commands include, for example, updating inventory quantities, registering items on the loan list, and adding new items.

[0230] Subsequently, the inventory update mechanism updates the database inventory information based on the instructions. This ensures accurate inventory information in real time, avoiding manual errors and duplicate management.

[0231] Furthermore, results and, if necessary, reports are generated and returned to the user through notification channels. For example, users may be notified when a new item is registered or when a lending operation is completed.

[0232] As an additional feature, users can take pictures of items with their devices and send them to the server. The server uses visual data integration to analyze the images and integrate the item information into a database. This automates the visual management of items.

[0233] For example, if a user enters "I want to borrow two projectors," the server identifies the item and quantity as projectors, instructs the lending process, and appropriately reduces the inventory by two units. Furthermore, a report is generated as needed and notified to the user, allowing them to verify the entire operation.

[0234] The above describes the embodiments for carrying out this invention and the method for achieving the objective of the invention, which is to improve the efficiency and accuracy of inventory management.

[0235] The following describes the processing flow.

[0236] Step 1:

[0237] The user uses a terminal to input requests for operations related to items in natural language. For example, they might input the instruction, "Add 20 notes."

[0238] Step 2:

[0239] The terminal sends the entered data to the server. The transmitted data retains the user's instructions exactly as they were given.

[0240] Step 3:

[0241] The server analyzes the received data using natural language processing. Based on the analysis, it extracts the item name "notebook" and quantity "20" from the instructions and saves this as structured data.

[0242] Step 4:

[0243] The server uses a command generation mechanism to generate specific inventory management commands based on the analyzed data. For example, a command such as "Add 20 notebooks to inventory" is generated.

[0244] Step 5:

[0245] The server, using the inventory update mechanism, accesses the database and updates the inventory information based on the generated instructions. In this case, the number of notes increases by 20.

[0246] Step 6:

[0247] The server checks the results after the update and generates a message to notify the user. This message includes confirmation that the operation was completed successfully.

[0248] Step 7:

[0249] The terminal receives a notification message from the server and displays the result to the user. The user confirms that their instructions were processed correctly.

[0250] Step 8:

[0251] When a user takes a picture of an item and provides it, the device sends the image data to the server.

[0252] Step 9:

[0253] The server utilizes visual data integration technology to analyze received image data and integrate relevant information into the inventory database. During this process, the appearance and model number of items are included as supplementary data.

[0254] (Example 1)

[0255] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0256] In inventory management, manual inventory management hinders operational efficiency and is prone to errors and duplicate entries. Furthermore, the lack of automated methods for visually managing inventory makes it difficult to effectively grasp inventory levels.

[0257] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0258] In this invention, the server includes means for analyzing input natural language information to generate structured data, means for generating specific inventory management commands based on the analyzed structured data, and means for updating the inventory information in the database based on the commands. This enables more efficient operations and accurate inventory management. Furthermore, by utilizing visual data, it becomes possible to make the management of goods more comprehensive and automated.

[0259] "Natural language information" refers to linguistic data entered by users in the language they use on a daily basis, and is information that does not depend on specific rules or codes.

[0260] "Structured data" refers to data that is organized according to a specific format and made easily accessible and processable.

[0261] An "inventory management directive" is a set of specific instructions generated to manage inventory levels, and its contents include matters such as increasing or decreasing inventory and updating records.

[0262] A "database" is a means of storing information designed to efficiently store, search, and update large amounts of data.

[0263] "Notification means" refers to methods or devices for communicating generated information or results to the user, and includes various forms such as alerts and messages.

[0264] "Visual data" refers to data in a format that humans can perceive through their vision, such as images and videos.

[0265] "Visual data integration means" refers to methods or devices for analyzing visual information and incorporating it into existing data systems.

[0266] This invention comprises an automated inventory system that improves the efficiency and accuracy of inventory management. At the core of this system is the collaboration between the user, terminal, and server. The user begins by inputting instructions regarding inventory management into the terminal using natural language. In this process, natural language processing utilizing a generative AI model plays a crucial role.

[0267] When a user's instructions are received by the terminal, the terminal sends them to the server. The server uses its internal natural language processing capabilities to analyze these instructions and convert them into structured data. This structured data includes the item name, quantity, and operation details. The software used by the server includes an AI model for natural language processing.

[0268] Based on structured data, the server generates inventory management directives. These directives are used to update the inventory information in the database in real time. Specific software and hardware components include database systems and inventory management software. The server thus avoids manual errors and duplicate management, ensuring accurate inventory data.

[0269] Furthermore, the server uses a visual data integration mechanism to analyze images of items taken by the user with their device and integrates the visual data into a database. This function also allows for the management of visual item information.

[0270] As a concrete example, when a user enters "I want to borrow two projectors" into the terminal, the server identifies the item name "projector" and the quantity, and then performs an operation to reduce the inventory by two. An example of a prompt in this system is natural language input such as "I want to borrow two projectors." The system can automatically proceed with a series of processes based on this prompt.

[0271] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0272] Step 1:

[0273] The user enters instructions for managing the items into the terminal using natural language. An example of input is a prompt message such as, "I would like to borrow two projectors." This input is received by the terminal's interface and sent to the server.

[0274] Step 2:

[0275] The server analyzes the user's instructions received using natural language processing. This process uses a generative AI model to analyze the input natural language and identify item names, quantities, and operation details. Once the analysis is complete, this information is output as structured data.

[0276] Step 3:

[0277] Based on the structured data obtained by the server through analysis, a specific inventory management instruction is generated using the instruction generation means. For example, it includes the content of "lend out 2 projectors". This generated instruction is prepared as an output for inventory update.

[0278] Step 4:

[0279] The server uses the inventory update means to update the database based on the instruction. Specifically, the quantity of the item to be lent out is decreased, and the updated inventory information is saved in the database. Thereby, an accurate inventory status is maintained in real time.

[0280] Step 5:

[0281] The server generates an operation result and a necessary report, and notifies the user through the notification means. For example, a message such as "The lending of 2 projectors has been completed" is sent to the terminal. Through the notification, the user can confirm the completion of the operation.

[0282] Step 6:

[0283] The user takes a picture of the item using the terminal and sends it to the server. This data is passed to the server as additional input. The server analyzes the image with the visual data integration means and integrates the item information into the database. Thereby, visual data management is also achieved.

[0284] (Application Example 1)

[0285] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0286] In the current inventory management system, manual input is required when users issue operation instructions, and errors are likely to occur during this process. Also, inventory information is often not updated in real time, making it difficult to check the accurate inventory status. Furthermore, visual confirmation of items also has to rely on manual work, resulting in the problem of reduced efficiency.

[0287] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means.

[0288] In this invention, the server includes natural language processing means for analyzing input information from users to generate structured data, voice recognition means for recognizing voice data to extract operation commands, and video analysis means for analyzing the video data of the photographed items and integrating it with inventory information. As a result, it becomes possible for users to issue operation instructions by voice and immediately update the inventory information. Also, visual confirmation of items is automated through video analysis, improving management efficiency.

[0289] The "natural language processing means" is a device or program that analyzes input information in natural language from users and converts it into structured data suitable for data processing.

[0290] The "command generation means" is a device or program for automatically generating commands necessary for inventory management based on the structured data.

[0291] The "inventory update means" is a device or program having the function of accurately updating the inventory information in the database based on the generated commands.

[0292] The "notification means" is a device or program having the function of transmitting inventory information and the results of commands to users.

[0293] The "voice recognition means" is a device or program that analyzes voice data, converts its content into text data, and extracts operation commands based on this.

[0294] "Video analysis means" refers to a device or program that processes video data of an item that has been photographed, extracts information about the item based on that data, and integrates it with inventory information.

[0295] This invention provides a system that allows users to patrol a logistics center and issue voice commands using smart glasses. The server utilizes speech recognition technology to convert voice data from the user into text. This process uses speech recognition APIs such as Google Cloud Speech-to-Text or Amazon Transcribe. The converted text is analyzed by a natural language processing system to generate structured data. This data extracts information about the type and quantity of items, and the command generation means creates inventory management commands. The commands are reflected in the database by the inventory update means, and the latest inventory status is updated in real time.

[0296] Furthermore, video data of items captured through the camera installed in the smart glasses is analyzed on the server side using video analysis APIs such as Google Cloud Vision and Amazon Rekognition. Based on this video analysis, the characteristics and condition of the items are understood and integrated with inventory information. Based on this integrated data, appropriate inventory status and reports are provided to the user through notification means. For example, if a user gives a voice command such as "Check the inventory of pallet A," the system can retrieve the inventory information for pallet A and display it on the glasses' display. As a result, inventory management in the logistics field can be performed quickly and accurately.

[0297] As a concrete example, consider a system that uses smart glasses to manage inventory using voice and images within a logistics center. For instance, if a user gives the instruction, "Take pictures of all inventory in section B and create a report," all items will be automatically listed, an inventory report will be generated, and displayed on the screen. This process can be streamlined using a generative AI model, and suggestions that are useful for log management and anomaly detection can also be made. An example of a prompt would be, "Please explain the benefits of a system that uses smart glasses to manage inventory using voice and images within a logistics center."

[0298] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0299] Step 1:

[0300] The user issues voice commands through smart glasses. The input is the user's voice data, acquired through the smart glasses' microphone. The output is voice data sent to the server.

[0301] Step 2:

[0302] The server uses a speech recognition API to convert the acquired audio data into text. The input is audio data, and the output is text data. Specifically, it uses Google Cloud Speech-to-Text for speech recognition.

[0303] Step 3:

[0304] Based on text data, a natural language processing system performs analysis to generate structured data. The input is text data, and the output is structured data containing information necessary for inventory management. Specifically, the system extracts information such as item names and quantities from the text.

[0305] Step 4:

[0306] The server uses command generation means to generate inventory management commands based on structured data. The input is structured data, and the output is command data for inventory update. Specifically, it identifies the operation content such as adding or reducing inventory.

[0307] Step 5:

[0308] The server updates the inventory information in the database through inventory update means. The input is command data, and the output is the updated inventory information. Specifically, it includes the operation of updating the records in the inventory table using a database system (such as MySQL).

[0309] Step 6:

[0310] Transmit the video data of the item photographed by the user's smart glasses to the server. The input is the video data photographed by the user, and the output is the video data sent to the server.

[0311] Step 7:

[0312] The server uses a video analysis API to extract item information based on the video data and integrate it into the inventory information. The input is the video data, and the output is integrated data including item information. Specifically, it includes the process of identifying the item model number and quantity using Google Cloud Vision.

[0313] Step 8:

[0314] The server returns the final result to the user through notification means. The input is the integrated inventory information and report data, and the output is the display on the user's display. Specifically, display processing is performed to visually feedback the situation to the user's smart glasses.

[0315] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform specific processing using the user's emotion.

[0316] This invention provides a system that enables user-based interaction by incorporating an emotion engine into item management. This system is configured to analyze the user's natural language input and simultaneously detect the user's emotions from that input, and reflect them in the item management operation flow.

[0317] First, the user enters a request for items in natural language on the terminal. For example, a request might be, "I'm in a bit of a hurry, please register 10 laptops right now." This input is immediately sent to the server and parsed by natural language processing.

[0318] In parallel, the server utilizes an emotion engine to extract emotions such as "urgent" or "confused" from user input. This emotion information influences inventory management commands generated by the system, for example, by setting priorities that allow for faster processing.

[0319] The server uses a command generation mechanism to create commands that take into account the user's requests and emotions, and updates the database inventory information using an inventory update mechanism. In this process, the user is notified with emotion-based, customized messages. For example, feedback such as, "You seem to be in a hurry. We have prioritized your laptop registration," is sent immediately.

[0320] As a concrete example, if a user instructs, "I have some leeway, so please prepare five projectors by next week," the server's emotion engine will recognize this as "leeway" and adjust the project accordingly. If inventory is insufficient, it will generate a suggestion-based notification and present the user with alternative solutions.

[0321] This enables inventory management tailored to the user's specific situation and psychological state, leading to increased operational efficiency and improved user satisfaction. In particular, this invention contributes to customizing the user experience and improving the accuracy of inventory management.

[0322] The following describes the processing flow.

[0323] Step 1:

[0324] The user uses a terminal to input requests for items in natural language. For example, they might input a request such as, "I would like to reserve 10 tablets for next week's meeting."

[0325] Step 2:

[0326] The terminal sends user input to the server. This data contains the user's request exactly as it was submitted.

[0327] Step 3:

[0328] The server analyzes the received data using natural language processing. This extracts structured information from the request, such as "tablet," "10 units," and "reservation."

[0329] Step 4:

[0330] The server uses an emotion engine to analyze emotional information from the user's input text. In this case, from the phrase "for the meeting," it infers the importance and necessity of the task and extracts emotions such as "expectation" and "sense of responsibility."

[0331] Step 5:

[0332] Based on extracted information and sentiment data, the server utilizes a command generation mechanism to create optimal inventory management commands. In this example, a command is created to check the inventory status of tablets and secure them.

[0333] Step 6:

[0334] The server updates inventory information in the database based on instructions using inventory update mechanisms. Specifically, it performs temporary reservation of tablets and future allocations.

[0335] Step 7:

[0336] The server generates a customized message based on the processing results and emotions, and notifies the user. For example, it might send a message such as, "We support the success of your meeting. Your tablet reservation has been secured."

[0337] Step 8:

[0338] The terminal receives notifications sent from the server and displays the results to the user. The user can then confirm that their request has been processed correctly and proceed with confidence.

[0339] Step 9:

[0340] If requested by the user, additional images of the items can be taken with the device and sent to the server.

[0341] Step 10:

[0342] The server uses a visual data integration mechanism to integrate the received image data with inventory information. If necessary, it records detailed information about the items in a database.

[0343] (Example 2)

[0344] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0345] Traditional inventory management systems have faced challenges in improving user experience and efficient inventory management because they operate without considering the user's emotions or psychological state. Furthermore, there was a lack of feedback tailored to individual user needs, leading to a desire for increased satisfaction.

[0346] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0347] In this invention, the server includes a natural language processing means for analyzing user input information and generating structured information, an emotion detection means for extracting emotions from the analyzed information, and a command generation means for generating inventory management commands based on the extracted emotions and input information. This enables efficient item management based on user emotions and the provision of individually customized feedback.

[0348] "Natural language processing means" refers to a function that analyzes user input information and converts it into easily understandable structured information.

[0349] An "emotion detection method" is a function that analyzes the emotions contained in the user's input text and extracts the emotional state.

[0350] The "command generation means" is a function that processes the generation of optimal inventory management commands based on the user's requests and extracted emotions.

[0351] The "update mechanism" refers to a function for updating and managing the inventory information of a data storage device in a timely manner.

[0352] A "notification method" is a function that communicates generated feedback and system processing results to the user.

[0353] "Visual information integration means" refers to a function that integrates visual information of items acquired using image analysis technology into inventory information.

[0354] This invention is a system for realizing dynamic item management based on user emotions. First, the user uses a terminal to input item management requests in natural language. This input is immediately sent to the server.

[0355] The server utilizes natural language processing (NLP) tools to analyze user input and generate structured information. This NLP can utilize open-source libraries or cloud-based NLP services. Examples include libraries such as SpaCy and NLTK.

[0356] Simultaneously, the server uses emotion detection tools to extract the user's emotions from the analyzed information. This process can utilize, for example, Google Cloud's Natural Language API. If the emotion is detected as "urgent," this information will influence the subsequent command generation.

[0357] The server uses a command generation mechanism to generate inventory management commands based on the analyzed data and extracted sentiment. These commands are used to update inventory information via a data storage device.

[0358] Furthermore, the server uses notification methods to communicate processing results and feedback to the user via the terminal. Personalized feedback based on emotions improves the user experience. For example, the user might be notified with a message such as, "It seems you're in a hurry. We've prioritized registering the 10 items you specified."

[0359] For example, if a user enters "I need 10 desktop PCs by the end of next month," the server will recognize that there is "plenty of time" and adjust the project accordingly. Another example of a prompt message could be "Please register 50 cables urgently."

[0360] In this way, it becomes possible to manage items based on user emotions, achieving both increased operational efficiency and improved user satisfaction.

[0361] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0362] Step 1:

[0363] Users input requests regarding inventory management in natural language through a terminal. For example, they might input, "I want to register five new printers as soon as possible." This input data is structured as natural language text.

[0364] Step 2:

[0365] The terminal receives input from the user and sends that raw data directly to the server. The output here is a direct relay of the user's text input.

[0366] Step 3:

[0367] The server uses natural language processing to analyze the received text data. The input is the user's natural language text, and the output is the structured information of that text. Specifically, tokenization and syntactic analysis are performed.

[0368] Step 4:

[0369] The server uses emotion detection to extract the user's emotions from structured information. The input is structured information, and the output is emotion tags such as "urgent" and "relieved." The process includes the specific operation of analyzing emotion indicators in the input text using an emotion engine.

[0370] Step 5:

[0371] The server uses a command generation mechanism to generate inventory management commands based on structured information and sentiment tags. The input is structured information and sentiment tags, and the output is specific command information for inventory updates. In terms of operation, it sets priorities and generates commands for high-priority tasks to be processed quickly.

[0372] Step 6:

[0373] Based on this instruction, the server updates the inventory information of the data storage device using an update mechanism. The input is the instruction information, and the output is the updated inventory information. Specific operations include connecting to the database and performing the modifications.

[0374] Step 7:

[0375] The server notifies the user's terminal of the processing results and sentiment-based feedback via a notification system. The input is message information based on the processing results and sentiment, and the output is the feedback message provided to the user. Specifically, it generates a message such as, "It seems you are in a hurry. We have registered the specified printer as a priority," and sends it to the user.

[0376] (Application Example 2)

[0377] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0378] Existing inventory management systems lack the functionality to adjust workflows based on user emotions, making it difficult to process urgent tasks quickly. Furthermore, the lack of customization for the user experience hinders efforts to improve user satisfaction.

[0379] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0380] In this invention, the server includes a natural language processing means that analyzes user input information to generate structured data, an emotion analysis means that detects the user's emotional state and adjusts the priority of the work flow based on that emotion, and an inventory update means that updates the database to manage inventory information. This enables faster work and improved user satisfaction.

[0381] "User input information" refers to information such as instructions and inquiries regarding goods and tasks, entered in natural language.

[0382] "Natural language processing means" refers to technical means that analyze natural language input by a user and formalize it as structured data.

[0383] The "command generation means" is a function that automatically generates necessary inventory management and work instructions based on the analyzed data.

[0384] A "database" is an information infrastructure that effectively organizes, stores, and communicates inventory information and related data as needed.

[0385] A "stock update mechanism" is a function that updates the inventory information in the database according to the generated command, accurately reflecting the current status of goods.

[0386] "Emotion analysis means" refers to technology that detects the emotional state from the user's natural language input and appropriately adjusts the priority of business processes based on that.

[0387] A "notification method" is a mechanism for communicating information such as generated analysis results, inventory management status, and user feedback to users.

[0388] The system used to implement this application primarily operates in a network environment including a server and mobile devices. The server instantly receives information input from the user in natural language and performs analysis using natural language processing libraries such as Python's spaCy and NLTK. This analysis identifies what kind of item the user's input relates to and generates the necessary instructions.

[0389] In parallel, the server uses sentiment analysis tools such as the Google Cloud Natural Language API and IBM Watson Tone Analyzer to detect the user's emotional state. For example, it extracts emotional elements such as "urgent" or "request" from the input text and analyzes how these affect logistics priorities. Based on this sentiment analysis, the server dynamically adjusts the priorities of the workflow, processing tasks that require a quick response first.

[0390] The terminal functions as an interface with the user and receives feedback from the server. The terminal instantly displays customized notifications to the user, providing real-time information on work progress and problem-solving. This feedback includes sentiment-based messages tailored to the user's input, ensuring that the user is receiving appropriate responses to their needs and circumstances.

[0391] For example, if a staff member asks "Is there any way to make it work?" on a day when a delivery schedule for a large appliance is set, the system will provide feedback such as, "Yes, it seems urgent. We have readjusted the shipping schedule and arranged for pickup to be completed."

[0392] Furthermore, an example of a prompt message for the generating AI model is: "You are the inventory management system for a logistics center. Analyze user input, understand their emotions, and generate the most appropriate work instructions. For example, if given the dialog 'Please register 10 laptops as soon as possible,' set the priority high and provide a fast registration process." In this way, it is possible to realize emotion-based adaptive inventory management.

[0393] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0394] Step 1:

[0395] The user inputs information into the terminal using natural language. This information may include instructions or inquiries regarding the receiving, dispatching, or delivery of goods. This input serves as the initial data for proceeding to the next step.

[0396] Step 2:

[0397] The terminal sends input information to the server. The server uses Python's spaCy or NLTK to analyze the input natural language and generate structured data. The input is natural language that humans can understand, and the output is the analyzed structured data. This process is the foundation for identifying the items and quantities of the input.

[0398] Step 3:

[0399] The server uses the Google Cloud Natural Language API and IBM Watson Tone Analyzer to perform sentiment analysis based on the analyzed data. In this step, it detects the emotional state from the user's input and determines urgency and priority. The input is structured data, and the output is metadata containing sentiment information.

[0400] Step 4:

[0401] The server uses a command generation system to generate necessary inventory management commands based on analysis results and sentiment information. These commands are prioritized and clearly indicate tasks requiring immediate attention. The input is structured data with added sentiment information, and the output is specific work commands including priorities.

[0402] Step 5:

[0403] The database is updated on the server, keeping inventory information up-to-date. This process is carried out based on instructions and reflects the current location and status of items.

[0404] Step 6:

[0405] The server sends notifications to the terminal. These notifications include feedback based on the processing status of the commands received by the user and their emotions. The input is the generated commands and emotion feedback information, and the output is a customized notification for the user. The terminal displays the received information to the user, allowing the user to see the system's operation in real time.

[0406] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0407] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0408] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0409] [Third Embodiment]

[0410] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0411] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0412] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0413] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0414] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0415] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0416] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0417] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0418] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0419] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0420] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0421] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0422] This invention provides an automated inventory system aimed at improving the efficiency of inventory management. The implementation method and specific operation of the system are described below.

[0423] First, when a user performs an action related to an item (e.g., new registration, borrowing, return), they input their request in natural language. In response, the terminal sends the user's instructions to the server. The server is equipped with natural language processing capabilities and analyzes the input natural language text, converting it into structured data. This data includes the item name, quantity, and action details.

[0424] Next, the server uses a command generation mechanism to generate specific commands for inventory management based on the analyzed information. These commands include, for example, updating inventory quantities, registering items on the loan list, and adding new items.

[0425] Subsequently, the inventory update mechanism updates the database inventory information based on the instructions. This ensures accurate inventory information in real time, avoiding manual errors and duplicate management.

[0426] Furthermore, results and, if necessary, reports are generated and returned to the user through notification channels. For example, users may be notified when a new item is registered or when a lending operation is completed.

[0427] As an additional feature, users can take pictures of items with their devices and send them to the server. The server uses visual data integration to analyze the images and integrate the item information into a database. This automates the visual management of items.

[0428] For example, if a user enters "I want to borrow two projectors," the server identifies the item and quantity as projectors, instructs the lending process, and appropriately reduces the inventory by two units. Furthermore, a report is generated as needed and notified to the user, allowing them to verify the entire operation.

[0429] The above describes the embodiments for carrying out this invention and the method for achieving the objective of the invention, which is to improve the efficiency and accuracy of inventory management.

[0430] The following describes the processing flow.

[0431] Step 1:

[0432] The user uses a terminal to input requests for operations related to items in natural language. For example, they might input the instruction, "Add 20 notes."

[0433] Step 2:

[0434] The terminal sends the entered data to the server. The transmitted data retains the user's instructions exactly as they were given.

[0435] Step 3:

[0436] The server analyzes the received data using natural language processing. Based on the analysis, it extracts the item name "notebook" and quantity "20" from the instructions and saves this as structured data.

[0437] Step 4:

[0438] The server uses a command generation mechanism to generate specific inventory management commands based on the analyzed data. For example, a command such as "Add 20 notebooks to inventory" is generated.

[0439] Step 5:

[0440] The server, using the inventory update mechanism, accesses the database and updates the inventory information based on the generated instructions. In this case, the number of notes increases by 20.

[0441] Step 6:

[0442] The server checks the results after the update and generates a message to notify the user. This message includes confirmation that the operation was completed successfully.

[0443] Step 7:

[0444] The terminal receives a notification message from the server and displays the result to the user. The user confirms that their instructions were processed correctly.

[0445] Step 8:

[0446] When a user takes a picture of an item and provides it, the device sends the image data to the server.

[0447] Step 9:

[0448] The server utilizes visual data integration technology to analyze received image data and integrate relevant information into the inventory database. During this process, the appearance and model number of items are included as supplementary data.

[0449] (Example 1)

[0450] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0451] In inventory management, manual inventory management hinders operational efficiency and is prone to errors and duplicate entries. Furthermore, the lack of automated methods for visually managing inventory makes it difficult to effectively grasp inventory levels.

[0452] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0453] In this invention, the server includes means for analyzing input natural language information to generate structured data, means for generating specific inventory management commands based on the analyzed structured data, and means for updating the inventory information in the database based on the commands. This enables more efficient operations and accurate inventory management. Furthermore, by utilizing visual data, it becomes possible to make the management of goods more comprehensive and automated.

[0454] "Natural language information" refers to linguistic data entered by users in the language they use on a daily basis, and is information that does not depend on specific rules or codes.

[0455] "Structured data" refers to data that is organized according to a specific format and made easily accessible and processable.

[0456] An "inventory management directive" is a set of specific instructions generated to manage inventory levels, and its contents include matters such as increasing or decreasing inventory and updating records.

[0457] A "database" is a means of storing information designed to efficiently store, search, and update large amounts of data.

[0458] "Notification means" refers to methods or devices for communicating generated information or results to the user, and includes various forms such as alerts and messages.

[0459] "Visual data" refers to data in a format that humans can perceive through their vision, such as images and videos.

[0460] "Visual data integration means" refers to methods or devices for analyzing visual information and incorporating it into existing data systems.

[0461] This invention comprises an automated inventory system that improves the efficiency and accuracy of inventory management. At the core of this system is the collaboration between the user, terminal, and server. The user begins by inputting instructions regarding inventory management into the terminal using natural language. In this process, natural language processing utilizing a generative AI model plays a crucial role.

[0462] When a user's instructions are received by the terminal, the terminal sends them to the server. The server uses its internal natural language processing capabilities to analyze these instructions and convert them into structured data. This structured data includes the item name, quantity, and operation details. The software used by the server includes an AI model for natural language processing.

[0463] Based on structured data, the server generates inventory management directives. These directives are used to update the inventory information in the database in real time. Specific software and hardware components include database systems and inventory management software. The server thus avoids manual errors and duplicate management, ensuring accurate inventory data.

[0464] Furthermore, the server uses a visual data integration mechanism to analyze images of items taken by the user with their device and integrates the visual data into a database. This function also allows for the management of visual item information.

[0465] As a concrete example, when a user enters "I want to borrow two projectors" into the terminal, the server identifies the item name "projector" and the quantity, and then performs an operation to reduce the inventory by two. An example of a prompt in this system is natural language input such as "I want to borrow two projectors." The system can automatically proceed with a series of processes based on this prompt.

[0466] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0467] Step 1:

[0468] The user enters instructions for managing the items into the terminal using natural language. An example of input is a prompt message such as, "I would like to borrow two projectors." This input is received by the terminal's interface and sent to the server.

[0469] Step 2:

[0470] The server analyzes the user's instructions received using natural language processing. This process uses a generative AI model to analyze the input natural language and identify item names, quantities, and operation details. Once the analysis is complete, this information is output as structured data.

[0471] Step 3:

[0472] Based on the structured data obtained through analysis, the server generates specific inventory management commands using a command generation mechanism. For example, this might include a command such as "rent out two projectors." These generated commands are prepared as output for inventory updates.

[0473] Step 4:

[0474] The server uses an inventory update mechanism to update the database based on instructions. Specifically, it reduces the quantity of items available for loan and saves the updated inventory information to the database. This ensures that accurate inventory status is maintained in real time.

[0475] Step 5:

[0476] The server generates operation results and necessary reports, and notifies the user through a notification system. For example, a message such as "The loan of two projectors is complete" is sent to the terminal. The notification allows the user to confirm the completion of the operation.

[0477] Step 6:

[0478] The user takes an image of an item using a device and sends it to the server. This data is passed to the server as additional input. The server analyzes the image using a visual data integration system and integrates the item information into a database. This also achieves visual data management.

[0479] (Application Example 1)

[0480] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0481] Current inventory management systems require users to manually input data when issuing operation instructions, a process prone to errors. Furthermore, inventory information is often not updated in real time, making it difficult to accurately check inventory levels. Additionally, visual verification of items relies on manual processes, leading to reduced efficiency.

[0482] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0483] In this invention, the server includes a natural language processing means for analyzing user input information and generating structured data, a speech recognition means for recognizing voice data and extracting operation commands, and a video analysis means for analyzing video data of captured items and integrating it with inventory information. This makes it possible for users to issue operation commands by voice and for inventory information to be updated immediately. In addition, the visual confirmation of items is automated through video analysis, improving management efficiency.

[0484] "Natural language processing means" refers to a device or program that analyzes natural language input information from a user and converts it into structured data suitable for data processing.

[0485] A "command generation means" is a device or program for automatically generating commands necessary for inventory management based on structured data.

[0486] An "inventory update means" is a device or program that has the function of accurately updating the inventory information in a database based on the generated instructions.

[0487] "Notification means" refers to a device or program that has the function of communicating inventory information or the results of instructions to the user.

[0488] "Voice recognition means" refers to a device or program that analyzes voice data, converts its content into text data, and extracts operation commands based on this.

[0489] "Video analysis means" refers to a device or program that processes video data of an item that has been photographed, extracts information about the item based on that data, and integrates it with inventory information.

[0490] This invention provides a system that allows users to patrol a logistics center and issue voice commands using smart glasses. The server utilizes speech recognition technology to convert voice data from the user into text. This process uses speech recognition APIs such as Google Cloud Speech-to-Text or Amazon Transcribe. The converted text is analyzed by a natural language processing system to generate structured data. This data extracts information about the type and quantity of items, and the command generation means creates inventory management commands. The commands are reflected in the database by the inventory update means, and the latest inventory status is updated in real time.

[0491] Furthermore, video data of items captured through the camera installed in the smart glasses is analyzed on the server side using video analysis APIs such as Google Cloud Vision and Amazon Rekognition. Based on this video analysis, the characteristics and condition of the items are understood and integrated with inventory information. Based on this integrated data, appropriate inventory status and reports are provided to the user through notification means. For example, if a user gives a voice command such as "Check the inventory of pallet A," the system can retrieve the inventory information for pallet A and display it on the glasses' display. As a result, inventory management in the logistics field can be performed quickly and accurately.

[0492] As a concrete example, consider a system that uses smart glasses to manage inventory using voice and images within a logistics center. For instance, if a user gives the instruction, "Take pictures of all inventory in section B and create a report," all items will be automatically listed, an inventory report will be generated, and displayed on the screen. This process can be streamlined using a generative AI model, and suggestions that are useful for log management and anomaly detection can also be made. An example of a prompt would be, "Please explain the benefits of a system that uses smart glasses to manage inventory using voice and images within a logistics center."

[0493] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0494] Step 1:

[0495] The user issues voice commands through smart glasses. The input is the user's voice data, acquired through the smart glasses' microphone. The output is voice data sent to the server.

[0496] Step 2:

[0497] The server uses a speech recognition API to convert the acquired audio data into text. The input is audio data, and the output is text data. Specifically, it uses Google Cloud Speech-to-Text for speech recognition.

[0498] Step 3:

[0499] Based on text data, a natural language processing system performs analysis to generate structured data. The input is text data, and the output is structured data containing information necessary for inventory management. Specifically, the system extracts information such as item names and quantities from the text.

[0500] Step 4:

[0501] The server uses a command generation mechanism to generate inventory management commands based on structured data. The input is structured data, and the output is command data for updating inventory. Specifically, it identifies the operations to be performed, such as adding or removing inventory.

[0502] Step 5:

[0503] The server updates the inventory information in the database through an inventory update mechanism. The input is instruction data, and the output is the updated inventory information. Specifically, this involves updating records in the inventory table using a database system (e.g., MySQL).

[0504] Step 6:

[0505] The system transmits video data of objects captured by the user's smart glasses to a server. The input is the video data captured by the user, and the output is the video data sent to the server.

[0506] Step 7:

[0507] The server uses a video analysis API to extract information about items based on video data and integrate it into inventory information. The input is video data, and the output is integrated data including item information. Specifically, this includes the process of identifying item model numbers and quantities using Google Cloud Vision.

[0508] Step 8:

[0509] The server returns the final results to the user through a notification system. The input is integrated inventory information and reporting data, and the output is a display on the user's screen. Specifically, a display process is performed to provide visual feedback on the situation to the user's smart glasses.

[0510] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0511] This invention provides a system that enables user-based interaction by incorporating an emotion engine into item management. This system is configured to analyze the user's natural language input and simultaneously detect the user's emotions from that input, and reflect them in the item management operation flow.

[0512] First, the user enters a request for items in natural language on the terminal. For example, a request might be, "I'm in a bit of a hurry, please register 10 laptops right now." This input is immediately sent to the server and parsed by natural language processing.

[0513] In parallel, the server utilizes an emotion engine to extract emotions such as "urgent" or "confused" from user input. This emotion information influences inventory management commands generated by the system, for example, by setting priorities that allow for faster processing.

[0514] The server uses a command generation mechanism to create commands that take into account the user's requests and emotions, and updates the database inventory information using an inventory update mechanism. In this process, the user is notified with emotion-based, customized messages. For example, feedback such as, "You seem to be in a hurry. We have prioritized your laptop registration," is sent immediately.

[0515] As a concrete example, if a user instructs, "I have some leeway, so please prepare five projectors by next week," the server's emotion engine will recognize this as "leeway" and adjust the project accordingly. If inventory is insufficient, it will generate a suggestion-based notification and present the user with alternative solutions.

[0516] This enables inventory management tailored to the user's specific situation and psychological state, leading to increased operational efficiency and improved user satisfaction. In particular, this invention contributes to customizing the user experience and improving the accuracy of inventory management.

[0517] The following describes the processing flow.

[0518] Step 1:

[0519] The user uses a terminal to input requests for items in natural language. For example, they might input a request such as, "I would like to reserve 10 tablets for next week's meeting."

[0520] Step 2:

[0521] The terminal sends user input to the server. This data contains the user's request exactly as it was submitted.

[0522] Step 3:

[0523] The server analyzes the received data using natural language processing. This extracts structured information from the request, such as "tablet," "10 units," and "reservation."

[0524] Step 4:

[0525] The server uses an emotion engine to analyze emotional information from the user's input text. In this case, from the phrase "for the meeting," it infers the importance and necessity of the task and extracts emotions such as "expectation" and "sense of responsibility."

[0526] Step 5:

[0527] Based on extracted information and sentiment data, the server utilizes a command generation mechanism to create optimal inventory management commands. In this example, a command is created to check the inventory status of tablets and secure them.

[0528] Step 6:

[0529] The server updates inventory information in the database based on instructions using inventory update mechanisms. Specifically, it performs temporary reservation of tablets and future allocations.

[0530] Step 7:

[0531] The server generates a customized message based on the processing results and emotions, and notifies the user. For example, it might send a message such as, "We support the success of your meeting. Your tablet reservation has been secured."

[0532] Step 8:

[0533] The terminal receives notifications sent from the server and displays the results to the user. The user can then confirm that their request has been processed correctly and proceed with confidence.

[0534] Step 9:

[0535] If requested by the user, additional images of the items can be taken with the device and sent to the server.

[0536] Step 10:

[0537] The server uses a visual data integration mechanism to integrate the received image data with inventory information. If necessary, it records detailed information about the items in a database.

[0538] (Example 2)

[0539] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0540] Traditional inventory management systems have faced challenges in improving user experience and efficient inventory management because they operate without considering the user's emotions or psychological state. Furthermore, there was a lack of feedback tailored to individual user needs, leading to a desire for increased satisfaction.

[0541] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0542] In this invention, the server includes a natural language processing means for analyzing user input information and generating structured information, an emotion detection means for extracting emotions from the analyzed information, and a command generation means for generating inventory management commands based on the extracted emotions and input information. This enables efficient item management based on user emotions and the provision of individually customized feedback.

[0543] "Natural language processing means" refers to a function that analyzes user input information and converts it into easily understandable structured information.

[0544] An "emotion detection method" is a function that analyzes the emotions contained in the user's input text and extracts the emotional state.

[0545] The "command generation means" is a function that processes the generation of optimal inventory management commands based on the user's requests and extracted emotions.

[0546] The "update mechanism" refers to a function for updating and managing the inventory information of a data storage device in a timely manner.

[0547] A "notification method" is a function that communicates generated feedback and system processing results to the user.

[0548] "Visual information integration means" refers to a function that integrates visual information of items acquired using image analysis technology into inventory information.

[0549] This invention is a system for realizing dynamic item management based on user emotions. First, the user uses a terminal to input item management requests in natural language. This input is immediately sent to the server.

[0550] The server utilizes natural language processing (NLP) tools to analyze user input and generate structured information. This NLP can utilize open-source libraries or cloud-based NLP services. Examples include libraries such as SpaCy and NLTK.

[0551] Simultaneously, the server uses emotion detection tools to extract the user's emotions from the analyzed information. This process can utilize, for example, Google Cloud's Natural Language API. If the emotion is detected as "urgent," this information will influence the subsequent command generation.

[0552] The server uses a command generation mechanism to generate inventory management commands based on the analyzed data and extracted sentiment. These commands are used to update inventory information via a data storage device.

[0553] Furthermore, the server uses notification methods to communicate processing results and feedback to the user via the terminal. Personalized feedback based on emotions improves the user experience. For example, the user might be notified with a message such as, "It seems you're in a hurry. We've prioritized registering the 10 items you specified."

[0554] For example, if a user enters "I need 10 desktop PCs by the end of next month," the server will recognize that there is "plenty of time" and adjust the project accordingly. Another example of a prompt message could be "Please register 50 cables urgently."

[0555] In this way, it becomes possible to manage items based on user emotions, achieving both increased operational efficiency and improved user satisfaction.

[0556] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0557] Step 1:

[0558] Users input requests regarding inventory management in natural language through a terminal. For example, they might input, "I want to register five new printers as soon as possible." This input data is structured as natural language text.

[0559] Step 2:

[0560] The terminal receives input from the user and sends that raw data directly to the server. The output here is a direct relay of the user's text input.

[0561] Step 3:

[0562] The server uses natural language processing to analyze the received text data. The input is the user's natural language text, and the output is the structured information of that text. Specifically, tokenization and syntactic analysis are performed.

[0563] Step 4:

[0564] The server uses emotion detection to extract the user's emotions from structured information. The input is structured information, and the output is emotion tags such as "urgent" and "relieved." The process includes the specific operation of analyzing emotion indicators in the input text using an emotion engine.

[0565] Step 5:

[0566] The server uses a command generation mechanism to generate inventory management commands based on structured information and sentiment tags. The input is structured information and sentiment tags, and the output is specific command information for inventory updates. In terms of operation, it sets priorities and generates commands for high-priority tasks to be processed quickly.

[0567] Step 6:

[0568] Based on this instruction, the server updates the inventory information of the data storage device using an update mechanism. The input is the instruction information, and the output is the updated inventory information. Specific operations include connecting to the database and performing the modifications.

[0569] Step 7:

[0570] The server notifies the user's terminal of the processing results and sentiment-based feedback via a notification system. The input is message information based on the processing results and sentiment, and the output is the feedback message provided to the user. Specifically, it generates a message such as, "It seems you are in a hurry. We have registered the specified printer as a priority," and sends it to the user.

[0571] (Application Example 2)

[0572] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0573] Existing inventory management systems lack the functionality to adjust workflows based on user emotions, making it difficult to process urgent tasks quickly. Furthermore, the lack of customization for the user experience hinders efforts to improve user satisfaction.

[0574] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0575] In this invention, the server includes a natural language processing means that analyzes user input information to generate structured data, an emotion analysis means that detects the user's emotional state and adjusts the priority of the work flow based on that emotion, and an inventory update means that updates the database to manage inventory information. This enables faster work and improved user satisfaction.

[0576] "User input information" refers to information such as instructions and inquiries regarding goods and tasks, entered in natural language.

[0577] "Natural language processing means" refers to technical means that analyze natural language input by a user and formalize it as structured data.

[0578] The "command generation means" is a function that automatically generates necessary inventory management and work instructions based on the analyzed data.

[0579] A "database" is an information infrastructure that effectively organizes, stores, and communicates inventory information and related data as needed.

[0580] A "stock update mechanism" is a function that updates the inventory information in the database according to the generated command, accurately reflecting the current status of goods.

[0581] "Emotion analysis means" refers to technology that detects the emotional state from the user's natural language input and appropriately adjusts the priority of business processes based on that.

[0582] A "notification method" is a mechanism for communicating information such as generated analysis results, inventory management status, and user feedback to users.

[0583] The system used to implement this application primarily operates in a network environment including a server and mobile devices. The server instantly receives information input from the user in natural language and performs analysis using natural language processing libraries such as Python's spaCy and NLTK. This analysis identifies what kind of item the user's input relates to and generates the necessary instructions.

[0584] In parallel, the server uses sentiment analysis tools such as the Google Cloud Natural Language API and IBM Watson Tone Analyzer to detect the user's emotional state. For example, it extracts emotional elements such as "urgent" or "request" from the input text and analyzes how these affect logistics priorities. Based on this sentiment analysis, the server dynamically adjusts the priorities of the workflow, processing tasks that require a quick response first.

[0585] The terminal functions as an interface with the user and receives feedback from the server. The terminal instantly displays customized notifications to the user, providing real-time information on work progress and problem-solving. This feedback includes sentiment-based messages tailored to the user's input, ensuring that the user is receiving appropriate responses to their needs and circumstances.

[0586] For example, if a staff member asks "Is there any way to make it work?" on a day when a delivery schedule for a large appliance is set, the system will provide feedback such as, "Yes, it seems urgent. We have readjusted the shipping schedule and arranged for pickup to be completed."

[0587] Furthermore, an example of a prompt message for the generating AI model is: "You are the inventory management system for a logistics center. Analyze user input, understand their emotions, and generate the most appropriate work instructions. For example, if given the dialog 'Please register 10 laptops as soon as possible,' set the priority high and provide a fast registration process." In this way, it is possible to realize emotion-based adaptive inventory management.

[0588] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0589] Step 1:

[0590] The user inputs information into the terminal using natural language. This information may include instructions or inquiries regarding the receiving, dispatching, or delivery of goods. This input serves as the initial data for proceeding to the next step.

[0591] Step 2:

[0592] The terminal sends input information to the server. The server uses Python's spaCy or NLTK to analyze the input natural language and generate structured data. The input is natural language that humans can understand, and the output is the analyzed structured data. This process is the foundation for identifying the items and quantities of the input.

[0593] Step 3:

[0594] The server uses the Google Cloud Natural Language API and IBM Watson Tone Analyzer to perform sentiment analysis based on the analyzed data. In this step, it detects the emotional state from the user's input and determines urgency and priority. The input is structured data, and the output is metadata containing sentiment information.

[0595] Step 4:

[0596] The server uses a command generation system to generate necessary inventory management commands based on analysis results and sentiment information. These commands are prioritized and clearly indicate tasks requiring immediate attention. The input is structured data with added sentiment information, and the output is specific work commands including priorities.

[0597] Step 5:

[0598] The database is updated on the server, keeping inventory information up-to-date. This process is carried out based on instructions and reflects the current location and status of items.

[0599] Step 6:

[0600] The server sends notifications to the terminal. These notifications include feedback based on the processing status of the commands received by the user and their emotions. The input is the generated commands and emotion feedback information, and the output is a customized notification for the user. The terminal displays the received information to the user, allowing the user to see the system's operation in real time.

[0601] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0602] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0603] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0604] [Fourth Embodiment]

[0605] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0606] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0607] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0608] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0609] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0610] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0611] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0612] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0613] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0614] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0615] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0616] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0617] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0618] This invention provides an automated inventory system aimed at improving the efficiency of inventory management. The implementation method and specific operation of the system are described below.

[0619] First, when a user performs an action related to an item (e.g., new registration, borrowing, return), they input their request in natural language. In response, the terminal sends the user's instructions to the server. The server is equipped with natural language processing capabilities and analyzes the input natural language text, converting it into structured data. This data includes the item name, quantity, and action details.

[0620] Next, the server uses a command generation mechanism to generate specific commands for inventory management based on the analyzed information. These commands include, for example, updating inventory quantities, registering items on the loan list, and adding new items.

[0621] Subsequently, the inventory update mechanism updates the database inventory information based on the instructions. This ensures accurate inventory information in real time, avoiding manual errors and duplicate management.

[0622] Furthermore, results and, if necessary, reports are generated and returned to the user through notification channels. For example, users may be notified when a new item is registered or when a lending operation is completed.

[0623] As an additional feature, users can take pictures of items with their devices and send them to the server. The server uses visual data integration to analyze the images and integrate the item information into a database. This automates the visual management of items.

[0624] For example, if a user enters "I want to borrow two projectors," the server identifies the item and quantity as projectors, instructs the lending process, and appropriately reduces the inventory by two units. Furthermore, a report is generated as needed and notified to the user, allowing them to verify the entire operation.

[0625] The above describes the embodiments for carrying out this invention and the method for achieving the objective of the invention, which is to improve the efficiency and accuracy of inventory management.

[0626] The following describes the processing flow.

[0627] Step 1:

[0628] The user uses a terminal to input requests for operations related to items in natural language. For example, they might input the instruction, "Add 20 notes."

[0629] Step 2:

[0630] The terminal sends the entered data to the server. The transmitted data retains the user's instructions exactly as they were given.

[0631] Step 3:

[0632] The server analyzes the received data using natural language processing. Based on the analysis, it extracts the item name "notebook" and quantity "20" from the instructions and saves this as structured data.

[0633] Step 4:

[0634] The server uses a command generation mechanism to generate specific inventory management commands based on the analyzed data. For example, a command such as "Add 20 notebooks to inventory" is generated.

[0635] Step 5:

[0636] The server, using the inventory update mechanism, accesses the database and updates the inventory information based on the generated instructions. In this case, the number of notes increases by 20.

[0637] Step 6:

[0638] The server checks the results after the update and generates a message to notify the user. This message includes confirmation that the operation was completed successfully.

[0639] Step 7:

[0640] The terminal receives a notification message from the server and displays the result to the user. The user confirms that their instructions were processed correctly.

[0641] Step 8:

[0642] When a user takes a picture of an item and provides it, the device sends the image data to the server.

[0643] Step 9:

[0644] The server utilizes visual data integration technology to analyze received image data and integrate relevant information into the inventory database. During this process, the appearance and model number of items are included as supplementary data.

[0645] (Example 1)

[0646] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0647] In inventory management, manual inventory management hinders operational efficiency and is prone to errors and duplicate entries. Furthermore, the lack of automated methods for visually managing inventory makes it difficult to effectively grasp inventory levels.

[0648] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0649] In this invention, the server includes means for analyzing input natural language information to generate structured data, means for generating specific inventory management commands based on the analyzed structured data, and means for updating the inventory information in the database based on the commands. This enables more efficient operations and accurate inventory management. Furthermore, by utilizing visual data, it becomes possible to make the management of goods more comprehensive and automated.

[0650] "Natural language information" refers to linguistic data entered by users in the language they use on a daily basis, and is information that does not depend on specific rules or codes.

[0651] "Structured data" refers to data that is organized according to a specific format and made easily accessible and processable.

[0652] An "inventory management directive" is a set of specific instructions generated to manage inventory levels, and its contents include matters such as increasing or decreasing inventory and updating records.

[0653] A "database" is a means of storing information designed to efficiently store, search, and update large amounts of data.

[0654] "Notification means" refers to methods or devices for communicating generated information or results to the user, and includes various forms such as alerts and messages.

[0655] "Visual data" refers to data in a format that humans can perceive through their vision, such as images and videos.

[0656] "Visual data integration means" refers to methods or devices for analyzing visual information and incorporating it into existing data systems.

[0657] This invention comprises an automated inventory system that improves the efficiency and accuracy of inventory management. At the core of this system is the collaboration between the user, terminal, and server. The user begins by inputting instructions regarding inventory management into the terminal using natural language. In this process, natural language processing utilizing a generative AI model plays a crucial role.

[0658] When a user's instructions are received by the terminal, the terminal sends them to the server. The server uses its internal natural language processing capabilities to analyze these instructions and convert them into structured data. This structured data includes the item name, quantity, and operation details. The software used by the server includes an AI model for natural language processing.

[0659] Based on structured data, the server generates inventory management directives. These directives are used to update the inventory information in the database in real time. Specific software and hardware components include database systems and inventory management software. The server thus avoids manual errors and duplicate management, ensuring accurate inventory data.

[0660] Furthermore, the server uses a visual data integration mechanism to analyze images of items taken by the user with their device and integrates the visual data into a database. This function also allows for the management of visual item information.

[0661] As a concrete example, when a user enters "I want to borrow two projectors" into the terminal, the server identifies the item name "projector" and the quantity, and then performs an operation to reduce the inventory by two. An example of a prompt in this system is natural language input such as "I want to borrow two projectors." The system can automatically proceed with a series of processes based on this prompt.

[0662] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0663] Step 1:

[0664] The user enters instructions for managing the items into the terminal using natural language. An example of input is a prompt message such as, "I would like to borrow two projectors." This input is received by the terminal's interface and sent to the server.

[0665] Step 2:

[0666] The server analyzes the user's instructions received using natural language processing. This process uses a generative AI model to analyze the input natural language and identify item names, quantities, and operation details. Once the analysis is complete, this information is output as structured data.

[0667] Step 3:

[0668] Based on the structured data obtained through analysis, the server generates specific inventory management commands using a command generation mechanism. For example, this might include a command such as "rent out two projectors." These generated commands are prepared as output for inventory updates.

[0669] Step 4:

[0670] The server uses an inventory update mechanism to update the database based on instructions. Specifically, it reduces the quantity of items available for loan and saves the updated inventory information to the database. This ensures that accurate inventory status is maintained in real time.

[0671] Step 5:

[0672] The server generates operation results and necessary reports, and notifies the user through a notification system. For example, a message such as "The loan of two projectors is complete" is sent to the terminal. The notification allows the user to confirm the completion of the operation.

[0673] Step 6:

[0674] The user takes an image of an item using a device and sends it to the server. This data is passed to the server as additional input. The server analyzes the image using a visual data integration system and integrates the item information into a database. This also achieves visual data management.

[0675] (Application Example 1)

[0676] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0677] Current inventory management systems require users to manually input data when issuing operation instructions, a process prone to errors. Furthermore, inventory information is often not updated in real time, making it difficult to accurately check inventory levels. Additionally, visual verification of items relies on manual processes, leading to reduced efficiency.

[0678] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0679] In this invention, the server includes a natural language processing means for analyzing user input information and generating structured data, a speech recognition means for recognizing voice data and extracting operation commands, and a video analysis means for analyzing video data of captured items and integrating it with inventory information. This makes it possible for users to issue operation commands by voice and for inventory information to be updated immediately. In addition, the visual confirmation of items is automated through video analysis, improving management efficiency.

[0680] "Natural language processing means" refers to a device or program that analyzes natural language input information from a user and converts it into structured data suitable for data processing.

[0681] A "command generation means" is a device or program for automatically generating commands necessary for inventory management based on structured data.

[0682] An "inventory update means" is a device or program that has the function of accurately updating the inventory information in a database based on the generated instructions.

[0683] "Notification means" refers to a device or program that has the function of communicating inventory information or the results of instructions to the user.

[0684] "Voice recognition means" refers to a device or program that analyzes voice data, converts its content into text data, and extracts operation commands based on this.

[0685] "Video analysis means" refers to a device or program that processes video data of an item that has been photographed, extracts information about the item based on that data, and integrates it with inventory information.

[0686] This invention provides a system that allows users to patrol a logistics center and issue voice commands using smart glasses. The server utilizes speech recognition technology to convert voice data from the user into text. This process uses speech recognition APIs such as Google Cloud Speech-to-Text or Amazon Transcribe. The converted text is analyzed by a natural language processing system to generate structured data. This data extracts information about the type and quantity of items, and the command generation means creates inventory management commands. The commands are reflected in the database by the inventory update means, and the latest inventory status is updated in real time.

[0687] Furthermore, video data of items captured through the camera installed in the smart glasses is analyzed on the server side using video analysis APIs such as Google Cloud Vision and Amazon Rekognition. Based on this video analysis, the characteristics and condition of the items are understood and integrated with inventory information. Based on this integrated data, appropriate inventory status and reports are provided to the user through notification means. For example, if a user gives a voice command such as "Check the inventory of pallet A," the system can retrieve the inventory information for pallet A and display it on the glasses' display. As a result, inventory management in the logistics field can be performed quickly and accurately.

[0688] As a concrete example, consider a system that uses smart glasses to manage inventory using voice and images within a logistics center. For instance, if a user gives the instruction, "Take pictures of all inventory in section B and create a report," all items will be automatically listed, an inventory report will be generated, and displayed on the screen. This process can be streamlined using a generative AI model, and suggestions that are useful for log management and anomaly detection can also be made. An example of a prompt would be, "Please explain the benefits of a system that uses smart glasses to manage inventory using voice and images within a logistics center."

[0689] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0690] Step 1:

[0691] The user issues voice commands through smart glasses. The input is the user's voice data, acquired through the smart glasses' microphone. The output is voice data sent to the server.

[0692] Step 2:

[0693] The server uses a speech recognition API to convert the acquired audio data into text. The input is audio data, and the output is text data. Specifically, it uses Google Cloud Speech-to-Text for speech recognition.

[0694] Step 3:

[0695] Based on text data, a natural language processing system performs analysis to generate structured data. The input is text data, and the output is structured data containing information necessary for inventory management. Specifically, the system extracts information such as item names and quantities from the text.

[0696] Step 4:

[0697] The server uses a command generation mechanism to generate inventory management commands based on structured data. The input is structured data, and the output is command data for updating inventory. Specifically, it identifies the operations to be performed, such as adding or removing inventory.

[0698] Step 5:

[0699] The server updates the inventory information in the database through an inventory update mechanism. The input is instruction data, and the output is the updated inventory information. Specifically, this involves updating records in the inventory table using a database system (e.g., MySQL).

[0700] Step 6:

[0701] The system transmits video data of objects captured by the user's smart glasses to a server. The input is the video data captured by the user, and the output is the video data sent to the server.

[0702] Step 7:

[0703] The server uses a video analysis API to extract information about items based on video data and integrate it into inventory information. The input is video data, and the output is integrated data including item information. Specifically, this includes the process of identifying item model numbers and quantities using Google Cloud Vision.

[0704] Step 8:

[0705] The server returns the final results to the user through a notification system. The input is integrated inventory information and reporting data, and the output is a display on the user's screen. Specifically, a display process is performed to provide visual feedback on the situation to the user's smart glasses.

[0706] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0707] This invention provides a system that enables user-based interaction by incorporating an emotion engine into item management. This system is configured to analyze the user's natural language input and simultaneously detect the user's emotions from that input, and reflect them in the item management operation flow.

[0708] First, the user enters a request for items in natural language on the terminal. For example, a request might be, "I'm in a bit of a hurry, please register 10 laptops right now." This input is immediately sent to the server and parsed by natural language processing.

[0709] In parallel, the server utilizes an emotion engine to extract emotions such as "urgent" or "confused" from user input. This emotion information influences inventory management commands generated by the system, for example, by setting priorities that allow for faster processing.

[0710] The server uses a command generation mechanism to create commands that take into account the user's requests and emotions, and updates the database inventory information using an inventory update mechanism. In this process, the user is notified with emotion-based, customized messages. For example, feedback such as, "You seem to be in a hurry. We have prioritized your laptop registration," is sent immediately.

[0711] As a concrete example, if a user instructs, "I have some leeway, so please prepare five projectors by next week," the server's emotion engine will recognize this as "leeway" and adjust the project accordingly. If inventory is insufficient, it will generate a suggestion-based notification and present the user with alternative solutions.

[0712] This enables inventory management tailored to the user's specific situation and psychological state, leading to increased operational efficiency and improved user satisfaction. In particular, this invention contributes to customizing the user experience and improving the accuracy of inventory management.

[0713] The following describes the processing flow.

[0714] Step 1:

[0715] The user uses a terminal to input requests for items in natural language. For example, they might input a request such as, "I would like to reserve 10 tablets for next week's meeting."

[0716] Step 2:

[0717] The terminal sends user input to the server. This data contains the user's request exactly as it was submitted.

[0718] Step 3:

[0719] The server analyzes the received data using natural language processing. This extracts structured information from the request, such as "tablet," "10 units," and "reservation."

[0720] Step 4:

[0721] The server uses an emotion engine to analyze emotional information from the user's input text. In this case, from the phrase "for the meeting," it infers the importance and necessity of the task and extracts emotions such as "expectation" and "sense of responsibility."

[0722] Step 5:

[0723] Based on extracted information and sentiment data, the server utilizes a command generation mechanism to create optimal inventory management commands. In this example, a command is created to check the inventory status of tablets and secure them.

[0724] Step 6:

[0725] The server updates inventory information in the database based on instructions using inventory update mechanisms. Specifically, it performs temporary reservation of tablets and future allocations.

[0726] Step 7:

[0727] The server generates a customized message based on the processing results and emotions, and notifies the user. For example, it might send a message such as, "We support the success of your meeting. Your tablet reservation has been secured."

[0728] Step 8:

[0729] The terminal receives notifications sent from the server and displays the results to the user. The user can then confirm that their request has been processed correctly and proceed with confidence.

[0730] Step 9:

[0731] If requested by the user, additional images of the items can be taken with the device and sent to the server.

[0732] Step 10:

[0733] The server uses a visual data integration mechanism to integrate the received image data with inventory information. If necessary, it records detailed information about the items in a database.

[0734] (Example 2)

[0735] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0736] Traditional inventory management systems have faced challenges in improving user experience and efficient inventory management because they operate without considering the user's emotions or psychological state. Furthermore, there was a lack of feedback tailored to individual user needs, leading to a desire for increased satisfaction.

[0737] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0738] In this invention, the server includes a natural language processing means for analyzing user input information and generating structured information, an emotion detection means for extracting emotions from the analyzed information, and a command generation means for generating inventory management commands based on the extracted emotions and input information. This enables efficient item management based on user emotions and the provision of individually customized feedback.

[0739] "Natural language processing means" refers to a function that analyzes user input information and converts it into easily understandable structured information.

[0740] An "emotion detection method" is a function that analyzes the emotions contained in the user's input text and extracts the emotional state.

[0741] The "command generation means" is a function that processes the generation of optimal inventory management commands based on the user's requests and extracted emotions.

[0742] The "update mechanism" refers to a function for updating and managing the inventory information of a data storage device in a timely manner.

[0743] A "notification method" is a function that communicates generated feedback and system processing results to the user.

[0744] "Visual information integration means" refers to a function that integrates visual information of items acquired using image analysis technology into inventory information.

[0745] This invention is a system for realizing dynamic item management based on user emotions. First, the user uses a terminal to input item management requests in natural language. This input is immediately sent to the server.

[0746] The server utilizes natural language processing (NLP) tools to analyze user input and generate structured information. This NLP can utilize open-source libraries or cloud-based NLP services. Examples include libraries such as SpaCy and NLTK.

[0747] Simultaneously, the server uses emotion detection tools to extract the user's emotions from the analyzed information. This process can utilize, for example, Google Cloud's Natural Language API. If the emotion is detected as "urgent," this information will influence the subsequent command generation.

[0748] The server uses a command generation mechanism to generate inventory management commands based on the analyzed data and extracted sentiment. These commands are used to update inventory information via a data storage device.

[0749] Furthermore, the server uses notification methods to communicate processing results and feedback to the user via the terminal. Personalized feedback based on emotions improves the user experience. For example, the user might be notified with a message such as, "It seems you're in a hurry. We've prioritized registering the 10 items you specified."

[0750] For example, if a user enters "I need 10 desktop PCs by the end of next month," the server will recognize that there is "plenty of time" and adjust the project accordingly. Another example of a prompt message could be "Please register 50 cables urgently."

[0751] In this way, it becomes possible to manage items based on user emotions, achieving both increased operational efficiency and improved user satisfaction.

[0752] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0753] Step 1:

[0754] Users input requests regarding inventory management in natural language through a terminal. For example, they might input, "I want to register five new printers as soon as possible." This input data is structured as natural language text.

[0755] Step 2:

[0756] The terminal receives input from the user and sends that raw data directly to the server. The output here is a direct relay of the user's text input.

[0757] Step 3:

[0758] The server uses natural language processing to analyze the received text data. The input is the user's natural language text, and the output is the structured information of that text. Specifically, tokenization and syntactic analysis are performed.

[0759] Step 4:

[0760] The server uses emotion detection to extract the user's emotions from structured information. The input is structured information, and the output is emotion tags such as "urgent" and "relieved." The process includes the specific operation of analyzing emotion indicators in the input text using an emotion engine.

[0761] Step 5:

[0762] The server uses a command generation mechanism to generate inventory management commands based on structured information and sentiment tags. The input is structured information and sentiment tags, and the output is specific command information for inventory updates. In terms of operation, it sets priorities and generates commands for high-priority tasks to be processed quickly.

[0763] Step 6:

[0764] Based on this instruction, the server updates the inventory information of the data storage device using an update mechanism. The input is the instruction information, and the output is the updated inventory information. Specific operations include connecting to the database and performing the modifications.

[0765] Step 7:

[0766] The server notifies the user's terminal of the processing results and sentiment-based feedback via a notification system. The input is message information based on the processing results and sentiment, and the output is the feedback message provided to the user. Specifically, it generates a message such as, "It seems you are in a hurry. We have registered the specified printer as a priority," and sends it to the user.

[0767] (Application Example 2)

[0768] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0769] Existing inventory management systems lack the functionality to adjust workflows based on user emotions, making it difficult to process urgent tasks quickly. Furthermore, the lack of customization for the user experience hinders efforts to improve user satisfaction.

[0770] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0771] In this invention, the server includes a natural language processing means that analyzes user input information to generate structured data, an emotion analysis means that detects the user's emotional state and adjusts the priority of the work flow based on that emotion, and an inventory update means that updates the database to manage inventory information. This enables faster work and improved user satisfaction.

[0772] "User input information" refers to information such as instructions and inquiries regarding goods and tasks, entered in natural language.

[0773] "Natural language processing means" refers to technical means that analyze natural language input by a user and formalize it as structured data.

[0774] The "command generation means" is a function that automatically generates necessary inventory management and work instructions based on the analyzed data.

[0775] A "database" is an information infrastructure that effectively organizes, stores, and communicates inventory information and related data as needed.

[0776] A "stock update mechanism" is a function that updates the inventory information in the database according to the generated command, accurately reflecting the current status of goods.

[0777] "Emotion analysis means" refers to technology that detects the emotional state from the user's natural language input and appropriately adjusts the priority of business processes based on that.

[0778] A "notification method" is a mechanism for communicating information such as generated analysis results, inventory management status, and user feedback to users.

[0779] The system used to implement this application primarily operates in a network environment including a server and mobile devices. The server instantly receives information input from the user in natural language and performs analysis using natural language processing libraries such as Python's spaCy and NLTK. This analysis identifies what kind of item the user's input relates to and generates the necessary instructions.

[0780] In parallel, the server uses sentiment analysis tools such as the Google Cloud Natural Language API and IBM Watson Tone Analyzer to detect the user's emotional state. For example, it extracts emotional elements such as "urgent" or "request" from the input text and analyzes how these affect logistics priorities. Based on this sentiment analysis, the server dynamically adjusts the priorities of the workflow, processing tasks that require a quick response first.

[0781] The terminal functions as an interface with the user and receives feedback from the server. The terminal instantly displays customized notifications to the user, providing real-time information on work progress and problem-solving. This feedback includes sentiment-based messages tailored to the user's input, ensuring that the user is receiving appropriate responses to their needs and circumstances.

[0782] For example, if a staff member asks "Is there any way to make it work?" on a day when a delivery schedule for a large appliance is set, the system will provide feedback such as, "Yes, it seems urgent. We have readjusted the shipping schedule and arranged for pickup to be completed."

[0783] Furthermore, an example of a prompt message for the generating AI model is: "You are the inventory management system for a logistics center. Analyze user input, understand their emotions, and generate the most appropriate work instructions. For example, if given the dialog 'Please register 10 laptops as soon as possible,' set the priority high and provide a fast registration process." In this way, it is possible to realize emotion-based adaptive inventory management.

[0784] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0785] Step 1:

[0786] The user inputs information into the terminal using natural language. This information may include instructions or inquiries regarding the receiving, dispatching, or delivery of goods. This input serves as the initial data for proceeding to the next step.

[0787] Step 2:

[0788] The terminal sends input information to the server. The server uses Python's spaCy or NLTK to analyze the input natural language and generate structured data. The input is natural language that humans can understand, and the output is the analyzed structured data. This process is the foundation for identifying the items and quantities of the input.

[0789] Step 3:

[0790] The server uses the Google Cloud Natural Language API and IBM Watson Tone Analyzer to perform sentiment analysis based on the analyzed data. In this step, it detects the emotional state from the user's input and determines urgency and priority. The input is structured data, and the output is metadata containing sentiment information.

[0791] Step 4:

[0792] The server uses a command generation system to generate necessary inventory management commands based on analysis results and sentiment information. These commands are prioritized and clearly indicate tasks requiring immediate attention. The input is structured data with added sentiment information, and the output is specific work commands including priorities.

[0793] Step 5:

[0794] The database is updated on the server, keeping inventory information up-to-date. This process is carried out based on instructions and reflects the current location and status of items.

[0795] Step 6:

[0796] The server sends notifications to the terminal. These notifications include feedback based on the processing status of the commands received by the user and their emotions. The input is the generated commands and emotion feedback information, and the output is a customized notification for the user. The terminal displays the received information to the user, allowing the user to see the system's operation in real time.

[0797] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0798] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0799] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0800] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0801] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0802] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0803] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0804] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0805] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0806] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0807] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0808] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0809] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0810] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0811] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0812] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0813] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0814] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0815] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0816] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0817] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0818] The following is further disclosed regarding the embodiments described above.

[0819] (Claim 1)

[0820] A natural language processing means that analyzes user input information to generate structured data,

[0821] A command generation means that generates appropriate inventory management commands based on analyzed data,

[0822] A means of updating inventory information by updating a database,

[0823] A notification mechanism for informing users of the generated results and reports,

[0824] A system that includes this.

[0825] (Claim 2)

[0826] The system according to claim 1, comprising means for automatically detecting items from a database that have not been used for a long period of time and providing a recommended disposal method.

[0827] (Claim 3)

[0828] The system according to claim 1, comprising a visual data integration means for analyzing the visual data of an item using image analysis technology and integrating it into inventory information.

[0829] "Example 1"

[0830] (Claim 1)

[0831] A means for analyzing input natural language information and generating structured data,

[0832] A means of generating specific inventory management instructions based on analyzed structured data,

[0833] A means of updating the database inventory information based on a command,

[0834] A means of notifying the user of the generated operation results and reports,

[0835] A means of analyzing visual data from users and integrating it into inventory information,

[0836] A system that includes this.

[0837] (Claim 2)

[0838] The system according to claim 1, comprising means for providing a method for processing infrequently used items by referring to information in a database.

[0839] (Claim 3)

[0840] The system according to claim 1, comprising means for analyzing input visual data and integrating item information into a database.

[0841] "Application Example 1"

[0842] (Claim 1)

[0843] A natural language processing means that analyzes user input information to generate structured data,

[0844] A command generation means that generates appropriate inventory management commands based on analyzed data,

[0845] A means of updating inventory information by updating a database,

[0846] A notification mechanism for informing users of the generated results and reports,

[0847] A voice recognition means that recognizes voice data and extracts operation commands,

[0848] A video analysis means that analyzes video data of captured items and integrates it with inventory information,

[0849] A system that includes this.

[0850] (Claim 2)

[0851] The system according to claim 1, comprising means for automatically detecting items from a database that have not been used for a long period of time and providing a recommended disposal method.

[0852] (Claim 3)

[0853] The system according to claim 1, comprising: a visual data integration means for analyzing the visual data of an item using image analysis technology and integrating it into inventory information; and a means for acquiring inventory information in real time based on voice instructions and generating a report.

[0854] "Example 2 of combining an emotion engine"

[0855] (Claim 1)

[0856] A natural language processing means that analyzes user input information to generate structured information,

[0857] An emotion detection method for extracting emotions from analyzed information,

[0858] A command generation means that generates inventory management commands based on extracted emotions and input information,

[0859] A means for updating a data storage device to manage inventory information,

[0860] A notification method for informing users of emotion-based feedback,

[0861] A system that includes this.

[0862] (Claim 2)

[0863] The system according to claim 1, comprising means for automatically detecting items that have not been used for a long period of time from a data storage device and providing a recommended processing method.

[0864] (Claim 3)

[0865] The system according to claim 1, comprising a visual information integration means for analyzing the visual information of an item using image analysis technology and integrating it into inventory information.

[0866] "Application example 2 when combining with an emotional engine"

[0867] (Claim 1)

[0868] A natural language processing means that analyzes user input information to generate structured data,

[0869] A command generation means that generates appropriate inventory management commands based on analyzed data,

[0870] A means of updating inventory information by updating a database,

[0871] An emotion analysis means that detects the user's emotional state and adjusts the priority of the workflow based on that emotion,

[0872] A notification mechanism for informing users of the generated results and reports,

[0873] A system that includes this.

[0874] (Claim 2)

[0875] The system according to claim 1, comprising means for automatically detecting items from a database that have not been used for a long period of time and providing a recommended disposal method.

[0876] (Claim 3)

[0877] The system according to claim 1, comprising a visual data integration means for analyzing the visual data of an item using image analysis technology and integrating it into inventory information. [Explanation of symbols]

[0878] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A natural language processing means that analyzes user input information to generate structured data, A command generation means that generates appropriate inventory management commands based on analyzed data, A means of updating inventory information by updating a database, A notification mechanism for informing users of the generated results and reports, A voice recognition means that recognizes voice data and extracts operation commands, A video analysis means that analyzes video data of captured items and integrates it with inventory information, A system that includes this.

2. The system according to claim 1, comprising means for automatically detecting items that have not been used for a long period of time from a database and providing a recommended processing method.

3. The system according to claim 1, comprising: a visual data integration means for analyzing the visual data of an item using image analysis technology and integrating it into inventory information; and a means for acquiring inventory information in real time based on voice instructions and generating a report.

Images