system

Abstract

We provide the system. [Solution] A means for analyzing user input information using natural language processing and evaluating its accuracy, A means for analyzing image data of an object using visual information processing means and extracting information about the object, A means of generating reports based on inventory information using an automated generation method and communicating them to the manager, A means for generating an inventory management policy using optimization means and realizing efficient inventory management, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional article management relies on manual work, and problems such as incorrect input, duplicate registration, and loss of articles caused by human errors occur frequently. In addition, the accompanying wasteful costs and increased working hours have become a burden on the organization. In particular, the physical inventory of articles requires a great deal of labor and time, and it is difficult to say that it is efficient. Therefore, there is a need to develop a system that solves these problems and realizes more efficient and accurate article management.

Means for Solving the Problems

[0005] This invention improves the efficiency and accuracy of inventory management by providing a system that includes natural language processing means, visual information processing means, automatic generation means, and optimization means. Specifically, it prevents input errors by analyzing user input information with natural language processing means and evaluating its accuracy. It also ensures data accuracy by automatically extracting inventory information from image data using visual information processing means. In addition, it enables efficient communication to managers by automatically generating reports based on inventory information using automatic generation means. Furthermore, it optimizes inventory management across the entire organization by generating inventory management policies using optimization means and automating the reduction of excess inventory and the handling of lost items.

[0006] "Natural language processing means" refers to technology that analyzes text input from users, understands its meaning, and evaluates its accuracy.

[0007] "Visual information processing means" refers to technologies that perform object identification and information extraction by analyzing image data.

[0008] "Automatic generation methods" refer to technologies that automatically create necessary reports and documents from data such as inventory information.

[0009] "Optimization means" refers to technologies that generate inventory management policies and support the efficient use of goods and the optimization of inventory.

[0010] "Asset management" refers to the processes within a company or organization, including purchasing, registering, using, returning, inventory management, stocktaking, and handling lost items.

[0011] A "report" refers to a document created to organize and communicate information such as inventory and usage status of goods to the manager. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention relates to a system that highly automates the inventory management process using AI technology. This system includes natural language processing means, visual information processing means, automatic generation means, and optimization means. The following describes how each component of the system functions.

[0034] The server is the core of the system. The server accesses the database for managing items and updates and manages the data in real time. First, the user enters instructions for registering new items, using them, or returning them via a terminal. The terminal sends the entered information to the server, which then processes it.

[0035] Specifically, when a user registers a new item, the server uses natural language processing to analyze the input information and returns instructions prompting correction if there are any errors. Furthermore, the user takes an image of the item to be registered using their device and simultaneously sends the image data to the server. The server utilizes visual information processing to automatically extract item information from the image and accurately records the information in the database.

[0036] Furthermore, regarding inventory management, the server periodically collects inventory information, uses an automated generation system to create inventory reports, and submits them to the administrator. These reports contain detailed information on missing items and items that have been unused for a long time.

[0037] By applying optimization techniques, the server generates inventory management policies based on data and proposes to the user, for example, ways to effectively utilize items that have been unused for a long period. In this way, the system reduces errors in inventory management and achieves efficient resource utilization.

[0038] For example, if a user enters "I want to reserve a projector for a meeting" into the terminal, the server uses natural language processing to analyze the instruction and check inventory. If a projector is available, the server approves the request and updates the reservation information in the database. This series of processes significantly improves the accuracy and efficiency of inventory management.

[0039] The following describes the processing flow.

[0040] Step 1:

[0041] The user enters a request to register a new item via their terminal. This includes entering the item name, category, quantity, and image.

[0042] Step 2:

[0043] The terminal receives input data from the user and sends the data to the server.

[0044] Step 3:

[0045] The server receives the data, analyzes the text information using natural language processing tools, and verifies that there are no errors in the input data.

[0046] Step 4:

[0047] If the server analyzes the input and finds any errors, it will return instructions to the user for correction. If the input is correct, it will proceed to the next step.

[0048] Step 5:

[0049] The user takes a picture of an item with their device and sends the image data to the server.

[0050] Step 6:

[0051] The server uses visual information processing to analyze image data and automatically extracts item information.

[0052] Step 7:

[0053] If the extracted information is deemed correct, the server records the new item information in the database.

[0054] Step 8:

[0055] When a user requests the use of an item, they enter instructions in natural language, such as "Use the projector for the meeting."

[0056] Step 9:

[0057] The terminal sends this request to the server, which uses LLM to analyze the instructions and understand the requirements.

[0058] Step 10:

[0059] The server checks the database to see if the requested item is in stock, updates the reservation information if available, and sends an availability notification to the user.

[0060] Step 11:

[0061] The server periodically performs inventory checks, organizes inventory information using an automated generation system, and sends reports to the administrator.

[0062] Step 12:

[0063] The server plans the appropriate disposal of unused items and makes suggestions to the administrator regarding items that have not been used for a long period of time.

[0064] (Example 1)

[0065] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0066] Traditional inventory management systems face challenges such as incorrect data entry, timely inventory updates, and a lack of effective ways to utilize unused resources. This reduces management efficiency and hinders the efficient use of resources. It is necessary to address these issues and achieve automation and optimization of inventory management.

[0067] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0068] This invention includes a server that analyzes user input information using an AI model and provides feedback for error detection and correction, a server that analyzes image data of items using visual information processing and extracts features, and an automated generation means that creates an inventory report based on inventory information and provides it to the administrator. This enables the automation of the item management process, efficient management, and effective use of resources.

[0069] "Generating AI models" refers to artificial intelligence technology that interprets natural language instructions from users and performs error detection and correction.

[0070] "Visual information processing" is a technology that extracts features from image data of objects and updates databases based on that information.

[0071] "Automatic generation means" refers to a system function that periodically collects inventory information, creates inventory reports, and provides them to administrators.

[0072] "Optimization" refers to the design of inventory management policies and methods for efficiently utilizing resources.

[0073] A "terminal" refers to an input / output device used by users to input instructions for managing items.

[0074] "Administrator" refers to the entity responsible for operating the system and receiving the results of inventory management.

[0075] A "database" is a digital repository that systematically stores information and inventory data about goods, and allows read and write access as needed.

[0076] This invention relates to a system that automates and optimizes inventory management using AI technology. The system is operated by a server, terminals, and users. The server significantly improves the efficiency of inventory management by utilizing technologies such as AI models, visual information processing, automatic data generation, and optimization.

[0077] The server receives natural language instructions entered by the user through the terminal. The input information is analyzed using a generative AI model, which has the function of detecting errors and providing feedback for correction. For example, if the user enters "I want to register a new projector for the conference room," the server analyzes this instruction and requests the necessary item information.

[0078] Next, the user takes a picture of the item using a device. The captured image data is sent to the server, which extracts the item's features from the image using visual information processing. This allows accurate item information to be registered in the database from the image data.

[0079] Furthermore, the server uses automated generation methods to create inventory reports from periodic inventory data and submit them to the administrator. These reports include detailed information on insufficient stock and unused items. As a result, administrators can make appropriate inventory decisions.

[0080] The server also optimizes based on accumulated data and suggests efficient ways to use resources to users. For example, it can suggest reallocating resources that have been unused for a long period for meetings. This suggestion promotes the effective use of resources throughout the organization.

[0081] An example of a prompt generated by the AI ​​model might be, "Please enter the information of the item you wish to register in natural language. Then, take a picture of the item and send it." This allows users to intuitively proceed through the item management process.

[0082] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0083] Step 1:

[0084] The user uses a terminal to input instructions in natural language regarding the registration, use, or return of an item. Specifically, the user inputs text information such as "I want to register a new projector for meeting room 2." This serves as initial input, informing the server of which item the user wants to use and how.

[0085] Step 2:

[0086] The terminal sends the information entered by the user to the server. Specifically, the entered natural language text data is passed directly to the server via the communication channel. The server receives this data and prepares for the next processing step.

[0087] Step 3:

[0088] The server analyzes the received natural language input using a generative AI model. The AI ​​model understands the user's intent from the text data and detects errors or omissions as needed. Based on this analysis, the server generates feedback if corrections are necessary and returns it to the user via the terminal. This process involves syntactic analysis of natural language and extraction of information about the type and use of items.

[0089] Step 4:

[0090] The user takes pictures of an object with their device. For example, they might take pictures of the front and back of a projector. The captured images are then sent back to the server by the device. This image data is stored on the server for further processing in the next step.

[0091] Step 5:

[0092] The server analyzes the received image data using visual information processing tools. It extracts characteristic information about the items from the images (for example, shape, manufacturer name, model number, etc.). This role is performed by an image processing algorithm, which automatically recognizes specific features and compares them with an existing database.

[0093] Step 6:

[0094] The server updates or registers new information in the inventory management database based on the analyzed text information and visually extracted information. The updated results are stored as accurate inventory information, which can then be used for future inventory management and usage monitoring.

[0095] Step 7:

[0096] The server uses an automated generation system to create an inventory report based on the latest inventory information and provides it to the administrator. This report can be output as a PDF or spreadsheet, allowing the administrator to verify the inventory. The report includes information such as inventory shortages and items that have been unused for a long period.

[0097] Step 8:

[0098] The server analyzes inventory data using optimization techniques and generates inventory management policies. For example, it might notify users of suggestions for efficient use of projectors. Such suggestions are presented to users via terminals and can be used to help them decide on effective resource utilization policies.

[0099] (Application Example 1)

[0100] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0101] Managing goods in a logistics center is extremely complex, and errors and data delays are particularly serious problems in inventory management. This can prevent the development of appropriate supply plans, potentially resulting in excess or shortages of inventory. Furthermore, managing lost items and long-term unused goods requires considerable effort, so there is a need for efficient solutions to these challenges.

[0102] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0103] In this invention, the server includes means for analyzing user input information using natural language processing technology and evaluating its accuracy, means for analyzing image data of items using visual information processing technology and extracting item information, and means for creating a report based on inventory information using automatic generation technology and transmitting it to the manager. As a result, inventory checks at the logistics center can be performed accurately in real time, and supply plans can be optimized by predicting excess or insufficient inventory in advance.

[0104] "Natural language processing technology" is a technology that analyzes human language, understands its meaning, and processes it.

[0105] "Visual information processing technology" is a technology that analyzes images and videos and extracts useful information from them.

[0106] "Automated generation technology" refers to technology that automatically creates reports and documents based on data.

[0107] "Optimization techniques" are techniques that involve calculations and adjustments to obtain the best possible results under given conditions.

[0108] A "resource management policy" is a guideline that outlines the most efficient methods for managing goods and inventory.

[0109] "Excess inventory" refers to a situation where more inventory is held than is needed.

[0110] "Inventory shortage" refers to a situation where there is insufficient inventory to meet demand.

[0111] A "supply plan" is the plan for how to supply goods and inventory.

[0112] This invention aims to realize an inventory management system in a logistics center. The server uses natural language processing technology to analyze voice instructions and text input from the user, and extracts and evaluates the necessary information. Images of items taken with a smartphone are then sent to the server, and item information is extracted via visual information processing technology. Specifically, image analysis is performed using software such as OpenCV and TENSORFLOW®.

[0113] The generated data is documented using automated generation technology as management reports and inventory forecast reports, and communicated to administrators. This primarily involves storing and managing data using a database management system located on a cloud server, such as MySQL®.

[0114] The server uses AI models to analyze inventory data, predicting excess and shortages, and creating optimal supply plans. By leveraging optimization technology, efficient resource management becomes possible. In this way, precise inventory management is achieved in the logistics center, promoting the effective use of resources.

[0115] For example, if a user enters the prompt "Please run the process to have the AI ​​calculate the appropriate purchase quantity of product A for the next month" into the terminal, the server will analyze the inventory data for the relevant product and provide a recommended purchase quantity. This process reduces the risk of excess or insufficient inventory and optimizes supply.

[0116] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0117] Step 1:

[0118] The user inputs information about an item via voice or text from their smartphone. Using natural language processing technology, the server analyzes this input and interprets its intent. The input consists of user instructions, such as reserving a projector or checking its availability. The analysis results in the output of specific operation commands and item IDs.

[0119] Step 2:

[0120] The server retrieves existing data about an item from the database according to the user's instructions. The input consists of the item ID and instructions, and the server uses a database system on the cloud server to retrieve the corresponding inventory information and usage history. The output is data indicating the item's current status and availability.

[0121] Step 3:

[0122] The user takes a picture of an item with their smartphone and sends it to the server. Using visual information processing technology, the server extracts identification information of the item from the image. The input is the captured image, which is then analyzed using software such as OpenCV. The output includes the type of item and its individual identifier.

[0123] Step 4:

[0124] The server integrates the analyzed linguistic and visual information to perform item registration and update processes. This includes adding new information to the database and updating existing information. The input to this process is the analyzed linguistic and visual information, and the output is the updated database entry.

[0125] Step 5:

[0126] The server analyzes collected inventory data and uses an AI model to predict future inventory trends. Inputs include data such as inventory levels and usage history, and the AI ​​model is used to forecast demand. Outputs include future demand trends and suggestions for appropriate procurement quantities.

[0127] Step 6:

[0128] The server uses automated generation technology to create an optimized supply plan based on the analysis results from the AI ​​model, and communicates it to the administrator as a report. The input is demand forecast data, and the output is a supply plan document that the administrator can use. This supports appropriate decision-making and action.

[0129] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0130] This invention combines an emotion engine with an item management system to enable flexible responses tailored to the user's emotional state. This system includes natural language processing, visual information processing, automatic generation, and optimization means, in addition to the emotion engine. The specific operation of this system is described below.

[0131] The core of the system is the server. The server works in conjunction with a database that holds various types of data. Users use terminals to perform operations such as registering, using, and returning items. Instructions entered by the user are received by the terminal and sent to the server.

[0132] In particular, the emotion engine analyzes the emotions a user displays during operation in real time. For example, if a user becomes frustrated when their input is not correctly received while registering an item, the emotion engine detects their stress level. This allows the server to present the user with optimized support messages and help resolve the problem.

[0133] For example, if a user expresses dissatisfaction upon receiving a notification that "the wireless mouse is out of stock," the emotion engine recognizes that emotion and presents appropriate alternatives, such as "suggesting the use of a similar product" or "setting up a notification when it is back in stock."

[0134] Furthermore, the emotion engine is involved in supplementing periodic inventory reports. Based on managers' reactions to past reports, the process is automated to identify areas for improvement and adjust new reports to better meet the managers' needs.

[0135] Thus, the present invention not only improves the accuracy and efficiency of inventory management, but also enhances the user experience and realizes a more flexible and adaptive inventory management environment.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] Users input requests for the use of items in natural language through their terminals. For example, "I would like to use the laptop for next week's meeting."

[0139] Step 2:

[0140] The terminal sends the input information received from the user to the server.

[0141] Step 3:

[0142] The server uses natural language processing to analyze the user's request information and accurately understand their intent.

[0143] Step 4:

[0144] The server checks the database to verify the inventory status and availability of the requested item.

[0145] Step 5:

[0146] Based on the verification results, the server activates an emotion engine to recognize the user's emotions and predict the user's response.

[0147] Step 6:

[0148] When a user receives a request, the system uses the device's camera and microphone to collect emotional data from the user's facial expressions and voice, and sends it to the server.

[0149] Step 7:

[0150] The server uses an emotion engine to analyze the received emotion data and evaluate the user's emotional state. For example, it identifies feelings such as joy, frustration, and anxiety.

[0151] Step 8:

[0152] Based on the evaluation results of the emotion engine, the server determines the optimal course of action to improve the user experience. For example, it may provide comforting messages or alternatives if an option is unavailable.

[0153] Step 9:

[0154] The server sends the sentiment engine's analysis results and a corresponding message to the user's device, which then displays them.

[0155] Step 10:

[0156] The inventory management process proceeds smoothly by allowing the user to select the next action based on the response provided and to re-enter any necessary requests or instructions on the terminal.

[0157] (Example 2)

[0158] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0159] In asset management systems, flexible support that takes into account the emotional state of users is not provided. Therefore, there is a need to achieve efficient asset management while reducing user stress. Furthermore, it is necessary to effectively utilize user feedback and continuously improve the system.

[0160] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0161] In this invention, the server includes means for analyzing user input information using natural language processing and evaluating its accuracy, means for detecting the user's emotional state and providing optimized support information based on that state, and means for collecting user feedback data and using it to improve the overall system. This enables more effective and adaptive item management for users.

[0162] "Natural language processing means" refers to technologies that analyze user input information and accurately understand its intent and meaning.

[0163] "Visual information processing means" refers to a technology for analyzing image data of an object and extracting useful information about that object from it.

[0164] An "automatic generation method" is a technology that automatically creates documents such as reports based on predefined information and conditions, and presents them to relevant parties.

[0165] "Optimization methods" refer to techniques that analyze data and propose and implement optimal management methods based on that analysis, in order to efficiently operate policies and processes in inventory management.

[0166] "Means for detecting emotional states" refers to technologies that recognize users' emotions from their actions and inputs, and then quantify or classify them.

[0167] "Means of providing support information" refers to technologies that provide information and suggestions tailored to the user's emotional state in a convenient format.

[0168] "Means of collecting feedback data" refers to technologies that systematically collect data based on user behavior and reactions, and use it for subsequent analysis and system improvement.

[0169] This item management system is server-centric and can flexibly respond to user operations and requests. The server uses natural language processing technology to understand instructions and questions sent by users from their terminals. This technology, for example, utilizes natural language processing libraries to analyze information and perform processing corresponding to the instructions. Furthermore, it uses visual information processing to extract item information from image data, enabling accurate item registration and management. This includes image recognition technology and integration with databases.

[0170] An emotion engine is built in, and the server detects the user's emotional state in real time based on their input. Based on this data, a generative AI model can be used to provide support information optimized for the user's emotions. In this process, for example, a language model is used as the AI ​​model to generate appropriate support messages and suggestions that match the user's emotions.

[0171] The terminal visually displays information provided by the server, allowing users to easily register, use, and return items. Feedback data is collected via the terminal and analyzed on the server. This contributes to improving the overall system performance and user experience.

[0172] For example, if a user expresses dissatisfaction upon receiving a notification on their device that "the wireless mouse is out of stock," the emotion engine recognizes this emotion, and the server presents appropriate alternatives, such as "suggesting the use of a similar product" or "recommending a restock notification." Such suggestions are made by inputting a prompt into the generative AI model that instructs it to "analyze the emotional response when the user checks the stock and generate relevant support messages."

[0173] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0174] Step 1:

[0175] The user uses a terminal to input information regarding the registration, use, and return of items. The entered data is received by the terminal and prepared to be sent to the server. Here, the user is instructed to check the availability of a wireless mouse. The terminal formats this instruction as an HTTP request and sends it to the server.

[0176] Step 2:

[0177] The server receives HTTP requests sent from terminals and parses the instructions using natural language processing. It processes the user's instructions received as input using text analysis techniques and converts the instructions into a format suitable for database searches. This analysis allows the server to quickly identify the necessary information.

[0178] Step 3:

[0179] Based on the analysis results, the server retrieves inventory information for the target item from the database. In this case, the number of wireless mice in stock is searched from the database. The inventory information is obtained as output and used for the next process. A database query is used to retrieve the current status of the item and prepare to inform the user.

[0180] Step 4:

[0181] The server sends input information to the emotion engine to detect the user's emotional state. The emotion engine analyzes the user's emotional response to the input (e.g., frustration or confusion) and returns the results to the server. This emotional data is used as a basis for determining when providing support information.

[0182] Step 5:

[0183] The server generates support messages using a generative AI model based on the output of the emotion engine. The input is the result of the emotion analysis, and the output is the generated support message. For example, if "wireless mouse is out of stock," it will generate suggestions for similar products and suggested settings for restock notifications. The prompt used for this generation is "Analyze the emotional response when the user checks the stock and generate relevant support messages."

[0184] Step 6:

[0185] The server sends the generated support message to the terminal. The terminal receives this message and displays it in an easy-to-understand format for the user. Suggestions and information are presented visually and clearly through the user interface.

[0186] Step 7:

[0187] The user reviews the support message displayed on the device and selects the next action. For example, they might decide to consider using a similar product or set up a restock notification. The device then sends the user's selection back to the server as feedback, and the server uses this data to improve the system.

[0188] (Application Example 2)

[0189] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0190] In inventory management systems, simply managing inventory and items without considering user emotions can lead to user dissatisfaction and stress. This can result in decreased customer satisfaction and inefficient management operations. Furthermore, to provide appropriate responses, a system is needed that can analyze users' emotional states in real time and automatically propose appropriate response strategies.

[0191] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0192] In this invention, the server includes means for analyzing user input information using natural language processing means and evaluating its accuracy, means for analyzing image data of objects using visual information processing means and extracting object information, means for generating reports based on inventory information using automatic generation means and transmitting them to the administrator, and means for analyzing the user's emotional state in real time using emotion analysis means and automatically proposing a response plan. This enables flexible and efficient responses that take into account the user's emotional state.

[0193] "Natural language processing means" refers to technologies that analyze input information such as speech and text from users and understand its accuracy and content.

[0194] "Visual information processing means" refers to a technology that analyzes image data of objects acquired from cameras or image sensors and extracts useful information from it.

[0195] "Automated generation means" refers to technology that automatically generates reports and proposals based on inventory information and user requests, and then communicates them to relevant parties.

[0196] An "optimization method" is a technology that generates an object management policy based on acquired information and makes adjustments to achieve efficient object management.

[0197] "Emotion analysis tools" are technologies that analyze the emotions expressed by users in real time and automatically suggest response strategies and methods of addressing them.

[0198] The system that realizes this invention consists of a server, a terminal used by the user, and a suite of software that supports them. The server is equipped with natural language processing means, visual information processing means, automatic generation means, optimization means, and sentiment analysis means, and uses these to respond to various requests from the user.

[0199] The servers run on cloud services such as Amazon Web Services and Google Cloud Platform. For natural language processing, the Google Cloud Natural Language API is used to analyze user voice and text data. For visual information processing, Microsoft Azure's Computer Vision API is used to extract necessary information from image data input by the user.

[0200] The automated generation method utilizes a TensorFlow-based AI model to automatically create reports and proposals based on inventory information and user preferences. Furthermore, the optimization method generates efficient system operation policies based on the acquired data.

[0201] The emotion analysis system analyzes the user's facial expressions and input in real time to detect the stress and dissatisfaction they are experiencing. This analysis allows the server to quickly provide an ideal response tailored to the user's emotional state. This enables store employees and managers using the terminals to achieve smoother communication with customers.

[0202] A concrete example is its use in a busy store. For instance, when a store employee uses Applia to process a customer's question, "Is this product out of stock?", the system immediately suggests similar products and recommends setting up restock notifications. Furthermore, by providing prompts to the generating AI model such as, "The user is dissatisfied with this product. Please suggest similar products and explain how to set up restock notifications," the AI ​​can design an appropriate response.

[0203] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0204] Step 1:

[0205] Users use their devices to make inquiries about products via voice or text. This input is sent from the device to the server. The server uses the Google Cloud Natural Language API to process this input data in natural language and parse the inquiry. It receives voice or text data as input and obtains the product inquiry as output.

[0206] Step 2:

[0207] The server queries the product inventory database to check the stock status of the queried product. If it is found that the product is out of stock during this process, the server uses a generative AI model powered by TensorFlow to automatically generate suggestions for similar alternative products. It receives information about the queried product as input and generates suggestions for alternative products as output.

[0208] Step 3:

[0209] Using emotion analysis tools, the server analyzes the user's emotional state at the time of their inquiry. It uses Microsoft Azure's Computer Vision API to determine the user's emotional state from their facial expressions and voice. It receives user image and audio data as input and outputs information about the user's emotional state.

[0210] Step 4:

[0211] The server automatically generates the optimal response based on the user's emotional state and inventory status. This response includes suggestions for similar products and instructions on how to set up restock notifications. The generated suggestion text is adjusted according to the user's emotional state. It takes emotional state and inventory information as input and generates a customized suggestion text as output.

[0212] Step 5:

[0213] The generated response is sent from the server to the terminal and presented to the user. Based on this, the user can choose the next action. Here, the suggestion sent from the server is displayed on the terminal. The generated suggestion is received as input and visually presented to the user as output.

[0214] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0215] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0216] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0217] [Second Embodiment]

[0218] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0219] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0220] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0221] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0222] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0223] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0224] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0225] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0226] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0227] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0228] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0229] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0230] This invention relates to a system that highly automates the inventory management process using AI technology. This system includes natural language processing means, visual information processing means, automatic generation means, and optimization means. The following describes how each component of the system functions.

[0231] The server is the core of the system. The server accesses the database for managing items and updates and manages the data in real time. First, the user enters instructions for registering new items, using them, or returning them via a terminal. The terminal sends the entered information to the server, which then processes it.

[0232] Specifically, when a user registers a new item, the server uses natural language processing to analyze the input information and returns instructions prompting correction if there are any errors. Furthermore, the user takes an image of the item to be registered using their device and simultaneously sends the image data to the server. The server utilizes visual information processing to automatically extract item information from the image and accurately records the information in the database.

[0233] Furthermore, regarding inventory management, the server periodically collects inventory information, uses an automated generation system to create inventory reports, and submits them to the administrator. These reports contain detailed information on missing items and items that have been unused for a long time.

[0234] By applying optimization techniques, the server generates inventory management policies based on data and proposes to the user, for example, ways to effectively utilize items that have been unused for a long period. In this way, the system reduces errors in inventory management and achieves efficient resource utilization.

[0235] For example, if a user enters "I want to reserve a projector for a meeting" into the terminal, the server uses natural language processing to analyze the instruction and check inventory. If a projector is available, the server approves the request and updates the reservation information in the database. This series of processes significantly improves the accuracy and efficiency of inventory management.

[0236] The following describes the processing flow.

[0237] Step 1:

[0238] The user enters a request to register a new item via their terminal. This includes entering the item name, category, quantity, and image.

[0239] Step 2:

[0240] The terminal receives input data from the user and sends the data to the server.

[0241] Step 3:

[0242] The server receives the data, analyzes the text information using natural language processing tools, and verifies that there are no errors in the input data.

[0243] Step 4:

[0244] If the server analyzes the input and finds any errors, it will return instructions to the user for correction. If the input is correct, it will proceed to the next step.

[0245] Step 5:

[0246] The user takes a picture of an item with their device and sends the image data to the server.

[0247] Step 6:

[0248] The server uses visual information processing to analyze image data and automatically extracts item information.

[0249] Step 7:

[0250] If the extracted information is deemed correct, the server records the new item information in the database.

[0251] Step 8:

[0252] When a user requests the use of an item, they enter instructions in natural language, such as "Use the projector for the meeting."

[0253] Step 9:

[0254] The terminal sends this request to the server, which uses LLM to analyze the instructions and understand the requirements.

[0255] Step 10:

[0256] The server checks the database to see if the requested item is in stock, updates the reservation information if available, and sends an availability notification to the user.

[0257] Step 11:

[0258] The server periodically performs inventory checks, organizes inventory information using an automated generation system, and sends reports to the administrator.

[0259] Step 12:

[0260] The server plans the appropriate disposal of unused items and makes suggestions to the administrator regarding items that have not been used for a long period of time.

[0261] (Example 1)

[0262] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0263] Traditional inventory management systems face challenges such as incorrect data entry, timely inventory updates, and a lack of effective ways to utilize unused resources. This reduces management efficiency and hinders the efficient use of resources. It is necessary to address these issues and achieve automation and optimization of inventory management.

[0264] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0265] This invention includes a server that analyzes user input information using an AI model and provides feedback for error detection and correction, a server that analyzes image data of items using visual information processing and extracts features, and an automated generation means that creates an inventory report based on inventory information and provides it to the administrator. This enables the automation of the item management process, efficient management, and effective use of resources.

[0266] "Generating AI models" refers to artificial intelligence technology that interprets natural language instructions from users and performs error detection and correction.

[0267] "Visual information processing" is a technology that extracts features from image data of objects and updates databases based on that information.

[0268] "Automatic generation means" refers to a system function that periodically collects inventory information, creates inventory reports, and provides them to administrators.

[0269] "Optimization" refers to the design of inventory management policies and methods for efficiently utilizing resources.

[0270] A "terminal" refers to an input / output device used by users to input instructions for managing items.

[0271] "Administrator" refers to the entity responsible for operating the system and receiving the results of inventory management.

[0272] A "database" is a digital repository that systematically stores information and inventory data about goods, and allows read and write access as needed.

[0273] This invention relates to a system that automates and optimizes inventory management using AI technology. The system is operated by a server, terminals, and users. The server significantly improves the efficiency of inventory management by utilizing technologies such as AI models, visual information processing, automatic data generation, and optimization.

[0274] The server receives natural language instructions entered by the user through the terminal. The input information is analyzed using a generative AI model, which has the function of detecting errors and providing feedback for correction. For example, if the user enters "I want to register a new projector for the conference room," the server analyzes this instruction and requests the necessary item information.

[0275] Next, the user takes a picture of the item using a device. The captured image data is sent to the server, which extracts the item's features from the image using visual information processing. This allows accurate item information to be registered in the database from the image data.

[0276] Furthermore, the server uses automated generation methods to create inventory reports from periodic inventory data and submit them to the administrator. These reports include detailed information on insufficient stock and unused items. As a result, administrators can make appropriate inventory decisions.

[0277] The server also optimizes based on accumulated data and suggests efficient ways to use resources to users. For example, it can suggest reallocating resources that have been unused for a long period for meetings. This suggestion promotes the effective use of resources throughout the organization.

[0278] An example of a prompt generated by the AI ​​model might be, "Please enter the information of the item you wish to register in natural language. Then, take a picture of the item and send it." This allows users to intuitively proceed through the item management process.

[0279] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0280] Step 1:

[0281] The user uses a terminal to input instructions in natural language regarding the registration, use, or return of an item. Specifically, the user inputs text information such as "I want to register a new projector for meeting room 2." This serves as initial input, informing the server of which item the user wants to use and how.

[0282] Step 2:

[0283] The terminal sends the information entered by the user to the server. Specifically, the input text data in natural language is directly passed to the server through the communication channel. The server receives this and prepares for the next processing.

[0284] Step 3:

[0285] The server analyzes the received natural language input using a generative AI model. The AI model understands the user's intention from the text data and detects errors and deficiencies if necessary. Based on this analysis, if corrections are needed, the server generates feedback and returns it to the user through the terminal. In this process, the natural language is parsed and information about the type and usage of the item is extracted.

[0286] Step 4:

[0287] The user takes a photo of the item with the terminal. As a specific operation for taking the photo, for example, images of the front and back of the projector are taken. The taken image is sent to the server again by the terminal. This image data is saved on the server for further processing in the next step.

[0288] Step 5:

[0289] The server analyzes the received image data using visual information processing means. Feature information of the item (such as shape, manufacturer name, model number, etc.) is extracted from the image. This role is played by an image processing algorithm, which automatically recognizes specific features and compares them with an existing database.

[0290] Step 6:

[0291] Based on the analyzed text information and visually extracted information, the server updates or newly registers information in the item management database. The updated result is saved as accurate inventory information. This is then utilized for future item management and monitoring of usage status.

[0292] Step 7:

[0293] The server uses an automated generation system to create an inventory report based on the latest inventory information and provides it to the administrator. This report can be output as a PDF or spreadsheet, allowing the administrator to verify the inventory. The report includes information such as inventory shortages and items that have been unused for a long period.

[0294] Step 8:

[0295] The server analyzes inventory data using optimization techniques and generates inventory management policies. For example, it might notify users of suggestions for efficient use of projectors. Such suggestions are presented to users via terminals and can be used to help them decide on effective resource utilization policies.

[0296] (Application Example 1)

[0297] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0298] Managing goods in a logistics center is extremely complex, and errors and data delays are particularly serious problems in inventory management. This can prevent the development of appropriate supply plans, potentially resulting in excess or shortages of inventory. Furthermore, managing lost items and long-term unused goods requires considerable effort, so there is a need for efficient solutions to these challenges.

[0299] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0300] In this invention, the server includes means for analyzing input information from a user using natural language processing technology and evaluating its accuracy, means for analyzing image data of an article using visual information processing technology and extracting article information, and means for creating a report based on inventory information using an automatic generation technology and transmitting it to an administrator. As a result, inventory verification at the logistics center can be accurately performed in real time, and it becomes possible to predict excess inventory and shortage inventory in advance and optimize the supply plan.

[0301] "Natural language processing technology" is a technology for analyzing human language, understanding its meaning, and processing it.

[0302] "Visual information processing technology" is a technology for analyzing images and videos and extracting useful information therefrom.

[0303] "Automatic generation technology" is a technology for automatically creating reports and documents based on data.

[0304] "Optimization technology" is a technology for performing calculations and adjustments to obtain the best results under given conditions.

[0305] "Resource management policy" is a guideline indicating the most efficient method for managing articles and inventory.

[0306] "Excess inventory" is a state in which more inventory is held than the demand.

[0307] "Shortage inventory" is a state in which there is insufficient inventory to meet the demand.

[0308] "Supply plan" is to plan how to supply articles and inventory.

[0309] This invention aims to realize an inventory management system in a logistics center. The server uses natural language processing technology to analyze voice instructions and text input from the user, and extracts and evaluates the necessary information. Images of items taken with a smartphone are then sent to the server, and item information is extracted via visual information processing technology. Specifically, image analysis is performed using software such as OpenCV and TensorFlow.

[0310] The generated data is documented using automated generation technology as management reports and inventory forecast reports, and communicated to administrators. This primarily involves storing and managing data using a database management system located on a cloud server, such as MySQL.

[0311] The server uses AI models to analyze inventory data, predicting excess and shortages, and creating optimal supply plans. By leveraging optimization technology, efficient resource management becomes possible. In this way, precise inventory management is achieved in the logistics center, promoting the effective use of resources.

[0312] For example, if a user enters the prompt "Please run the process to have the AI ​​calculate the appropriate purchase quantity of product A for the next month" into the terminal, the server will analyze the inventory data for the relevant product and provide a recommended purchase quantity. This process reduces the risk of excess or insufficient inventory and optimizes supply.

[0313] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0314] Step 1:

[0315] The user inputs information about an item via voice or text from their smartphone. Using natural language processing technology, the server analyzes this input and interprets its intent. The input consists of user instructions, such as reserving a projector or checking its availability. The analysis results in the output of specific operation commands and item IDs.

[0316] Step 2:

[0317] The server retrieves existing data about an item from the database according to the user's instructions. The input consists of the item ID and instructions, and the server uses a database system on the cloud server to retrieve the corresponding inventory information and usage history. The output is data indicating the item's current status and availability.

[0318] Step 3:

[0319] The user takes a picture of an item with their smartphone and sends it to the server. Using visual information processing technology, the server extracts identification information of the item from the image. The input is the captured image, which is then analyzed using software such as OpenCV. The output includes the type of item and its individual identifier.

[0320] Step 4:

[0321] The server integrates the analyzed linguistic and visual information to perform item registration and update processes. This includes adding new information to the database and updating existing information. The input to this process is the analyzed linguistic and visual information, and the output is the updated database entry.

[0322] Step 5:

[0323] The server analyzes collected inventory data and uses an AI model to predict future inventory trends. Inputs include data such as inventory levels and usage history, and the AI ​​model is used to forecast demand. Outputs include future demand trends and suggestions for appropriate procurement quantities.

[0324] Step 6:

[0325] The server uses automated generation technology to create an optimized supply plan based on the analysis results from the AI ​​model, and communicates it to the administrator as a report. The input is demand forecast data, and the output is a supply plan document that the administrator can use. This supports appropriate decision-making and action.

[0326] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0327] This invention combines an emotion engine with an item management system to enable flexible responses tailored to the user's emotional state. This system includes natural language processing, visual information processing, automatic generation, and optimization means, in addition to the emotion engine. The specific operation of this system is described below.

[0328] The core of the system is the server. The server works in conjunction with a database that holds various types of data. Users use terminals to perform operations such as registering, using, and returning items. Instructions entered by the user are received by the terminal and sent to the server.

[0329] In particular, the emotion engine analyzes the emotions a user displays during operation in real time. For example, if a user becomes frustrated when their input is not correctly received while registering an item, the emotion engine detects their stress level. This allows the server to present the user with optimized support messages and help resolve the problem.

[0330] For example, if a user expresses dissatisfaction upon receiving a notification that "the wireless mouse is out of stock," the emotion engine recognizes that emotion and presents appropriate alternatives, such as "suggesting the use of a similar product" or "setting up a notification when it is back in stock."

[0331] Furthermore, the emotion engine is involved in supplementing periodic inventory reports. Based on managers' reactions to past reports, the process is automated to identify areas for improvement and adjust new reports to better meet the managers' needs.

[0332] Thus, the present invention not only improves the accuracy and efficiency of inventory management, but also enhances the user experience and realizes a more flexible and adaptive inventory management environment.

[0333] The following describes the processing flow.

[0334] Step 1:

[0335] Users input requests for the use of items in natural language through their terminals. For example, "I would like to use the laptop for next week's meeting."

[0336] Step 2:

[0337] The terminal sends the input information received from the user to the server.

[0338] Step 3:

[0339] The server uses natural language processing to analyze the user's request information and accurately understand their intent.

[0340] Step 4:

[0341] The server checks the database to verify the inventory status and availability of the requested item.

[0342] Step 5:

[0343] Based on the verification results, the server activates an emotion engine to recognize the user's emotions and predict the user's response.

[0344] Step 6:

[0345] When a user receives a request, the system uses the device's camera and microphone to collect emotional data from the user's facial expressions and voice, and sends it to the server.

[0346] Step 7:

[0347] The server uses an emotion engine to analyze the received emotion data and evaluate the user's emotional state. For example, it identifies feelings such as joy, frustration, and anxiety.

[0348] Step 8:

[0349] Based on the evaluation results of the emotion engine, the server determines the optimal course of action to improve the user experience. For example, it may provide comforting messages or alternatives if an option is unavailable.

[0350] Step 9:

[0351] The server sends the sentiment engine's analysis results and a corresponding message to the user's device, which then displays them.

[0352] Step 10:

[0353] The inventory management process proceeds smoothly by allowing the user to select the next action based on the response provided and to re-enter any necessary requests or instructions on the terminal.

[0354] (Example 2)

[0355] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0356] In asset management systems, flexible support that takes into account the emotional state of users is not provided. Therefore, there is a need to achieve efficient asset management while reducing user stress. Furthermore, it is necessary to effectively utilize user feedback and continuously improve the system.

[0357] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0358] In this invention, the server includes means for analyzing user input information using natural language processing and evaluating its accuracy, means for detecting the user's emotional state and providing optimized support information based on that state, and means for collecting user feedback data and using it to improve the overall system. This enables more effective and adaptive item management for users.

[0359] "Natural language processing means" refers to technologies that analyze user input information and accurately understand its intent and meaning.

[0360] "Visual information processing means" refers to a technology for analyzing image data of an object and extracting useful information about that object from it.

[0361] An "automatic generation method" is a technology that automatically creates documents such as reports based on predefined information and conditions, and presents them to relevant parties.

[0362] "Optimization methods" refer to techniques that analyze data and propose and implement optimal management methods based on that analysis, in order to efficiently operate policies and processes in inventory management.

[0363] "Means for detecting emotional states" refers to technologies that recognize users' emotions from their actions and inputs, and then quantify or classify them.

[0364] "Means of providing support information" refers to technologies that provide information and suggestions tailored to the user's emotional state in a convenient format.

[0365] "Means of collecting feedback data" refers to technologies that systematically collect data based on user behavior and reactions, and use it for subsequent analysis and system improvement.

[0366] This item management system is server-centric and can flexibly respond to user operations and requests. The server uses natural language processing technology to understand instructions and questions sent by users from their terminals. This technology, for example, utilizes natural language processing libraries to analyze information and perform processing corresponding to the instructions. Furthermore, it uses visual information processing to extract item information from image data, enabling accurate item registration and management. This includes image recognition technology and integration with databases.

[0367] An emotion engine is built in, and the server detects the user's emotional state in real time based on their input. Based on this data, a generative AI model can be used to provide support information optimized for the user's emotions. In this process, for example, a language model is used as the AI ​​model to generate appropriate support messages and suggestions that match the user's emotions.

[0368] The terminal visually displays information provided by the server, allowing users to easily register, use, and return items. Feedback data is collected via the terminal and analyzed on the server. This contributes to improving the overall system performance and user experience.

[0369] For example, if a user expresses dissatisfaction upon receiving a notification on their device that "the wireless mouse is out of stock," the emotion engine recognizes this emotion, and the server presents appropriate alternatives, such as "suggesting the use of a similar product" or "recommending a restock notification." Such suggestions are made by inputting a prompt into the generative AI model that instructs it to "analyze the emotional response when the user checks the stock and generate relevant support messages."

[0370] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0371] Step 1:

[0372] The user uses a terminal to input information regarding the registration, use, and return of items. The entered data is received by the terminal and prepared to be sent to the server. Here, the user is instructed to check the availability of a wireless mouse. The terminal formats this instruction as an HTTP request and sends it to the server.

[0373] Step 2:

[0374] The server receives HTTP requests sent from terminals and parses the instructions using natural language processing. It processes the user's instructions received as input using text analysis techniques and converts the instructions into a format suitable for database searches. This analysis allows the server to quickly identify the necessary information.

[0375] Step 3:

[0376] Based on the analysis results, the server retrieves inventory information for the target item from the database. In this case, the number of wireless mice in stock is searched from the database. The inventory information is obtained as output and used for the next process. A database query is used to retrieve the current status of the item and prepare to inform the user.

[0377] Step 4:

[0378] The server sends input information to the emotion engine to detect the user's emotional state. The emotion engine analyzes the user's emotional response to the input (e.g., frustration or confusion) and returns the results to the server. This emotional data is used as a basis for determining when providing support information.

[0379] Step 5:

[0380] The server generates support messages using a generative AI model based on the output of the emotion engine. The input is the result of the emotion analysis, and the output is the generated support message. For example, if "wireless mouse is out of stock," it will generate suggestions for similar products and suggested settings for restock notifications. The prompt used for this generation is "Analyze the emotional response when the user checks the stock and generate relevant support messages."

[0381] Step 6:

[0382] The server sends the generated support message to the terminal. The terminal receives this message and displays it in an easy-to-understand format for the user. Suggestions and information are presented visually and clearly through the user interface.

[0383] Step 7:

[0384] The user reviews the support message displayed on the device and selects the next action. For example, they might decide to consider using a similar product or set up a restock notification. The device then sends the user's selection back to the server as feedback, and the server uses this data to improve the system.

[0385] (Application Example 2)

[0386] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0387] In inventory management systems, simply managing inventory and items without considering user emotions can lead to user dissatisfaction and stress. This can result in decreased customer satisfaction and inefficient management operations. Furthermore, to provide appropriate responses, a system is needed that can analyze users' emotional states in real time and automatically propose appropriate response strategies.

[0388] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0389] In this invention, the server includes means for analyzing user input information using natural language processing means and evaluating its accuracy, means for analyzing image data of objects using visual information processing means and extracting object information, means for generating reports based on inventory information using automatic generation means and transmitting them to the administrator, and means for analyzing the user's emotional state in real time using emotion analysis means and automatically proposing a response plan. This enables flexible and efficient responses that take into account the user's emotional state.

[0390] "Natural language processing means" refers to technologies that analyze input information such as speech and text from users and understand its accuracy and content.

[0391] "Visual information processing means" refers to a technology that analyzes image data of objects acquired from cameras or image sensors and extracts useful information from it.

[0392] "Automated generation means" refers to technology that automatically generates reports and proposals based on inventory information and user requests, and then communicates them to relevant parties.

[0393] An "optimization method" is a technology that generates an object management policy based on acquired information and makes adjustments to achieve efficient object management.

[0394] "Emotion analysis tools" are technologies that analyze the emotions expressed by users in real time and automatically suggest response strategies and methods of addressing them.

[0395] The system that realizes this invention consists of a server, a terminal used by the user, and a suite of software that supports them. The server is equipped with natural language processing means, visual information processing means, automatic generation means, optimization means, and sentiment analysis means, and uses these to respond to various requests from the user.

[0396] The servers run on cloud services such as Amazon Web Services and Google Cloud Platform. For natural language processing, the Google Cloud Natural Language API is used to analyze user voice and text data. For visual information processing, Microsoft Azure's Computer Vision API is used to extract necessary information from image data input by the user.

[0397] The automated generation method utilizes a TensorFlow-based AI model to automatically create reports and proposals based on inventory information and user preferences. Furthermore, the optimization method generates efficient system operation policies based on the acquired data.

[0398] The emotion analysis system analyzes the user's facial expressions and input in real time to detect the stress and dissatisfaction they are experiencing. This analysis allows the server to quickly provide an ideal response tailored to the user's emotional state. This enables store employees and managers using the terminals to achieve smoother communication with customers.

[0399] A concrete example is its use in a busy store. For instance, when a store employee uses Applia to process a customer's question, "Is this product out of stock?", the system immediately suggests similar products and recommends setting up restock notifications. Furthermore, by providing prompts to the generating AI model such as, "The user is dissatisfied with this product. Please suggest similar products and explain how to set up restock notifications," the AI ​​can design an appropriate response.

[0400] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0401] Step 1:

[0402] Users use their devices to make inquiries about products via voice or text. This input is sent from the device to the server. The server uses the Google Cloud Natural Language API to process this input data in natural language and parse the inquiry. It receives voice or text data as input and obtains the product inquiry as output.

[0403] Step 2:

[0404] The server queries the product inventory database to check the stock status of the queried product. If it is found that the product is out of stock during this process, the server uses a generative AI model powered by TensorFlow to automatically generate suggestions for similar alternative products. It receives information about the queried product as input and generates suggestions for alternative products as output.

[0405] Step 3:

[0406] Using emotion analysis tools, the server analyzes the user's emotional state at the time of their inquiry. It uses Microsoft Azure's Computer Vision API to determine the user's emotional state from their facial expressions and voice. It receives user image and audio data as input and outputs information about the user's emotional state.

[0407] Step 4:

[0408] The server automatically generates the optimal response based on the user's emotional state and inventory status. This response includes suggestions for similar products and instructions on how to set up restock notifications. The generated suggestion text is adjusted according to the user's emotional state. It takes emotional state and inventory information as input and generates a customized suggestion text as output.

[0409] Step 5:

[0410] The generated response is sent from the server to the terminal and presented to the user. Based on this, the user can choose the next action. Here, the suggestion sent from the server is displayed on the terminal. The generated suggestion is received as input and visually presented to the user as output.

[0411] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0412] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0413] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0414] [Third Embodiment]

[0415] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0416] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0417] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0418] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0419] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0420] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0421] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0422] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0423] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0424] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0425] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0426] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0427] This invention relates to a system that highly automates the inventory management process using AI technology. This system includes natural language processing means, visual information processing means, automatic generation means, and optimization means. The following describes how each component of the system functions.

[0428] The server is the core of the system. The server accesses the database for managing items and updates and manages the data in real time. First, the user enters instructions for registering new items, using them, or returning them via a terminal. The terminal sends the entered information to the server, which then processes it.

[0429] Specifically, when a user registers a new item, the server uses natural language processing to analyze the input information and returns instructions prompting correction if there are any errors. Furthermore, the user takes an image of the item to be registered using their device and simultaneously sends the image data to the server. The server utilizes visual information processing to automatically extract item information from the image and accurately records the information in the database.

[0430] Furthermore, regarding inventory management, the server periodically collects inventory information, uses an automated generation system to create inventory reports, and submits them to the administrator. These reports contain detailed information on missing items and items that have been unused for a long time.

[0431] By applying optimization techniques, the server generates inventory management policies based on data and proposes to the user, for example, ways to effectively utilize items that have been unused for a long period. In this way, the system reduces errors in inventory management and achieves efficient resource utilization.

[0432] For example, if a user enters "I want to reserve a projector for a meeting" into the terminal, the server uses natural language processing to analyze the instruction and check inventory. If a projector is available, the server approves the request and updates the reservation information in the database. This series of processes significantly improves the accuracy and efficiency of inventory management.

[0433] The following describes the processing flow.

[0434] Step 1:

[0435] The user enters a request to register a new item via their terminal. This includes entering the item name, category, quantity, and image.

[0436] Step 2:

[0437] The terminal receives input data from the user and sends the data to the server.

[0438] Step 3:

[0439] The server receives the data, analyzes the text information using natural language processing tools, and verifies that there are no errors in the input data.

[0440] Step 4:

[0441] If the server analyzes the input and finds any errors, it will return instructions to the user for correction. If the input is correct, it will proceed to the next step.

[0442] Step 5:

[0443] The user takes a picture of an item with their device and sends the image data to the server.

[0444] Step 6:

[0445] The server uses visual information processing to analyze image data and automatically extracts item information.

[0446] Step 7:

[0447] If the extracted information is deemed correct, the server records the new item information in the database.

[0448] Step 8:

[0449] When a user requests the use of an item, they enter instructions in natural language, such as "Use the projector for the meeting."

[0450] Step 9:

[0451] The terminal sends this request to the server, which uses LLM to analyze the instructions and understand the requirements.

[0452] Step 10:

[0453] The server checks the database to see if the requested item is in stock, updates the reservation information if available, and sends an availability notification to the user.

[0454] Step 11:

[0455] The server periodically performs inventory checks, organizes inventory information using an automated generation system, and sends reports to the administrator.

[0456] Step 12:

[0457] The server plans the appropriate disposal of unused items and makes suggestions to the administrator regarding items that have not been used for a long period of time.

[0458] (Example 1)

[0459] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0460] Traditional inventory management systems face challenges such as incorrect data entry, timely inventory updates, and a lack of effective ways to utilize unused resources. This reduces management efficiency and hinders the efficient use of resources. It is necessary to address these issues and achieve automation and optimization of inventory management.

[0461] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0462] This invention includes a server that analyzes user input information using an AI model and provides feedback for error detection and correction, a server that analyzes image data of items using visual information processing and extracts features, and an automated generation means that creates an inventory report based on inventory information and provides it to the administrator. This enables the automation of the item management process, efficient management, and effective use of resources.

[0463] "Generating AI models" refers to artificial intelligence technology that interprets natural language instructions from users and performs error detection and correction.

[0464] "Visual information processing" is a technology that extracts features from image data of objects and updates databases based on that information.

[0465] "Automatic generation means" refers to a system function that periodically collects inventory information, creates inventory reports, and provides them to administrators.

[0466] "Optimization" refers to the design of inventory management policies and methods for efficiently utilizing resources.

[0467] A "terminal" refers to an input / output device used by users to input instructions for managing items.

[0468] "Administrator" refers to the entity responsible for operating the system and receiving the results of inventory management.

[0469] A "database" is a digital repository that systematically stores information and inventory data about goods, and allows read and write access as needed.

[0470] This invention relates to a system that automates and optimizes inventory management using AI technology. The system is operated by a server, terminals, and users. The server significantly improves the efficiency of inventory management by utilizing technologies such as AI models, visual information processing, automatic data generation, and optimization.

[0471] The server receives natural language instructions entered by the user through the terminal. The input information is analyzed using a generative AI model, which has the function of detecting errors and providing feedback for correction. For example, if the user enters "I want to register a new projector for the conference room," the server analyzes this instruction and requests the necessary item information.

[0472] Next, the user takes a picture of the item using a device. The captured image data is sent to the server, which extracts the item's features from the image using visual information processing. This allows accurate item information to be registered in the database from the image data.

[0473] Furthermore, the server uses automated generation methods to create inventory reports from periodic inventory data and submit them to the administrator. These reports include detailed information on insufficient stock and unused items. As a result, administrators can make appropriate inventory decisions.

[0474] The server also optimizes based on accumulated data and suggests efficient ways to use resources to users. For example, it can suggest reallocating resources that have been unused for a long period for meetings. This suggestion promotes the effective use of resources throughout the organization.

[0475] An example of a prompt generated by the AI ​​model might be, "Please enter the information of the item you wish to register in natural language. Then, take a picture of the item and send it." This allows users to intuitively proceed through the item management process.

[0476] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0477] Step 1:

[0478] The user uses a terminal to input instructions in natural language regarding the registration, use, or return of an item. Specifically, the user inputs text information such as "I want to register a new projector for meeting room 2." This serves as initial input, informing the server of which item the user wants to use and how.

[0479] Step 2:

[0480] The terminal sends the information entered by the user to the server. Specifically, the entered natural language text data is passed directly to the server via the communication channel. The server receives this data and prepares for the next processing step.

[0481] Step 3:

[0482] The server analyzes the received natural language input using a generative AI model. The AI ​​model understands the user's intent from the text data and detects errors or omissions as needed. Based on this analysis, the server generates feedback if corrections are necessary and returns it to the user via the terminal. This process involves syntactic analysis of natural language and extraction of information about the type and use of items.

[0483] Step 4:

[0484] The user takes pictures of an object with their device. For example, they might take pictures of the front and back of a projector. The captured images are then sent back to the server by the device. This image data is stored on the server for further processing in the next step.

[0485] Step 5:

[0486] The server analyzes the received image data using visual information processing tools. It extracts characteristic information about the items from the images (for example, shape, manufacturer name, model number, etc.). This role is performed by an image processing algorithm, which automatically recognizes specific features and compares them with an existing database.

[0487] Step 6:

[0488] The server updates or registers new information in the inventory management database based on the analyzed text information and visually extracted information. The updated results are stored as accurate inventory information, which can then be used for future inventory management and usage monitoring.

[0489] Step 7:

[0490] The server uses an automated generation system to create an inventory report based on the latest inventory information and provides it to the administrator. This report can be output as a PDF or spreadsheet, allowing the administrator to verify the inventory. The report includes information such as inventory shortages and items that have been unused for a long period.

[0491] Step 8:

[0492] The server analyzes inventory data using optimization techniques and generates inventory management policies. For example, it might notify users of suggestions for efficient use of projectors. Such suggestions are presented to users via terminals and can be used to help them decide on effective resource utilization policies.

[0493] (Application Example 1)

[0494] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0495] Managing goods in a logistics center is extremely complex, and errors and data delays are particularly serious problems in inventory management. This can prevent the development of appropriate supply plans, potentially resulting in excess or shortages of inventory. Furthermore, managing lost items and long-term unused goods requires considerable effort, so there is a need for efficient solutions to these challenges.

[0496] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0497] In this invention, the server includes means for analyzing user input information using natural language processing technology and evaluating its accuracy, means for analyzing image data of items using visual information processing technology and extracting item information, and means for creating a report based on inventory information using automatic generation technology and transmitting it to the manager. As a result, inventory checks at the logistics center can be performed accurately in real time, and supply plans can be optimized by predicting excess or insufficient inventory in advance.

[0498] "Natural language processing technology" is a technology that analyzes human language, understands its meaning, and processes it.

[0499] "Visual information processing technology" is a technology that analyzes images and videos and extracts useful information from them.

[0500] "Automated generation technology" refers to technology that automatically creates reports and documents based on data.

[0501] "Optimization techniques" are techniques that involve calculations and adjustments to obtain the best possible results under given conditions.

[0502] A "resource management policy" is a guideline that outlines the most efficient methods for managing goods and inventory.

[0503] "Excess inventory" refers to a situation where more inventory is held than is needed.

[0504] "Inventory shortage" refers to a situation where there is insufficient inventory to meet demand.

[0505] A "supply plan" is the plan for how to supply goods and inventory.

[0506] This invention aims to realize an inventory management system in a logistics center. The server uses natural language processing technology to analyze voice instructions and text input from the user, and extracts and evaluates the necessary information. Images of items taken with a smartphone are then sent to the server, and item information is extracted via visual information processing technology. Specifically, image analysis is performed using software such as OpenCV and TensorFlow.

[0507] The generated data is documented using automated generation technology as management reports and inventory forecast reports, and communicated to administrators. This primarily involves storing and managing data using a database management system located on a cloud server, such as MySQL.

[0508] The server uses AI models to analyze inventory data, predicting excess and shortages, and creating optimal supply plans. By leveraging optimization technology, efficient resource management becomes possible. In this way, precise inventory management is achieved in the logistics center, promoting the effective use of resources.

[0509] For example, if a user enters the prompt "Please run the process to have the AI ​​calculate the appropriate purchase quantity of product A for the next month" into the terminal, the server will analyze the inventory data for the relevant product and provide a recommended purchase quantity. This process reduces the risk of excess or insufficient inventory and optimizes supply.

[0510] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0511] Step 1:

[0512] The user inputs information about an item via voice or text from their smartphone. Using natural language processing technology, the server analyzes this input and interprets its intent. The input consists of user instructions, such as reserving a projector or checking its availability. The analysis results in the output of specific operation commands and item IDs.

[0513] Step 2:

[0514] The server retrieves existing data about an item from the database according to the user's instructions. The input consists of the item ID and instructions, and the server uses a database system on the cloud server to retrieve the corresponding inventory information and usage history. The output is data indicating the item's current status and availability.

[0515] Step 3:

[0516] The user takes a picture of an item with their smartphone and sends it to the server. Using visual information processing technology, the server extracts identification information of the item from the image. The input is the captured image, which is then analyzed using software such as OpenCV. The output includes the type of item and its individual identifier.

[0517] Step 4:

[0518] The server integrates the analyzed linguistic and visual information to perform item registration and update processes. This includes adding new information to the database and updating existing information. The input to this process is the analyzed linguistic and visual information, and the output is the updated database entry.

[0519] Step 5:

[0520] The server analyzes collected inventory data and uses an AI model to predict future inventory trends. Inputs include data such as inventory levels and usage history, and the AI ​​model is used to forecast demand. Outputs include future demand trends and suggestions for appropriate procurement quantities.

[0521] Step 6:

[0522] The server uses automated generation technology to create an optimized supply plan based on the analysis results from the AI ​​model, and communicates it to the administrator as a report. The input is demand forecast data, and the output is a supply plan document that the administrator can use. This supports appropriate decision-making and action.

[0523] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0524] This invention combines an emotion engine with an item management system to enable flexible responses tailored to the user's emotional state. This system includes natural language processing, visual information processing, automatic generation, and optimization means, in addition to the emotion engine. The specific operation of this system is described below.

[0525] The core of the system is the server. The server works in conjunction with a database that holds various types of data. Users use terminals to perform operations such as registering, using, and returning items. Instructions entered by the user are received by the terminal and sent to the server.

[0526] In particular, the emotion engine analyzes the emotions a user displays during operation in real time. For example, if a user becomes frustrated when their input is not correctly received while registering an item, the emotion engine detects their stress level. This allows the server to present the user with optimized support messages and help resolve the problem.

[0527] For example, if a user expresses dissatisfaction upon receiving a notification that "the wireless mouse is out of stock," the emotion engine recognizes that emotion and presents appropriate alternatives, such as "suggesting the use of a similar product" or "setting up a notification when it is back in stock."

[0528] Furthermore, the emotion engine is involved in supplementing periodic inventory reports. Based on managers' reactions to past reports, the process is automated to identify areas for improvement and adjust new reports to better meet the managers' needs.

[0529] Thus, the present invention not only improves the accuracy and efficiency of inventory management, but also enhances the user experience and realizes a more flexible and adaptive inventory management environment.

[0530] The following describes the processing flow.

[0531] Step 1:

[0532] Users input requests for the use of items in natural language through their terminals. For example, "I would like to use the laptop for next week's meeting."

[0533] Step 2:

[0534] The terminal sends the input information received from the user to the server.

[0535] Step 3:

[0536] The server uses natural language processing to analyze the user's request information and accurately understand their intent.

[0537] Step 4:

[0538] The server checks the database to verify the inventory status and availability of the requested item.

[0539] Step 5:

[0540] Based on the verification results, the server activates an emotion engine to recognize the user's emotions and predict the user's response.

[0541] Step 6:

[0542] When a user receives a request, the system uses the device's camera and microphone to collect emotional data from the user's facial expressions and voice, and sends it to the server.

[0543] Step 7:

[0544] The server uses an emotion engine to analyze the received emotion data and evaluate the user's emotional state. For example, it identifies feelings such as joy, frustration, and anxiety.

[0545] Step 8:

[0546] Based on the evaluation results of the emotion engine, the server determines the optimal course of action to improve the user experience. For example, it may provide comforting messages or alternatives if an option is unavailable.

[0547] Step 9:

[0548] The server sends the sentiment engine's analysis results and a corresponding message to the user's device, which then displays them.

[0549] Step 10:

[0550] The inventory management process proceeds smoothly by allowing the user to select the next action based on the response provided and to re-enter any necessary requests or instructions on the terminal.

[0551] (Example 2)

[0552] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0553] In asset management systems, flexible support that takes into account the emotional state of users is not provided. Therefore, there is a need to achieve efficient asset management while reducing user stress. Furthermore, it is necessary to effectively utilize user feedback and continuously improve the system.

[0554] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0555] In this invention, the server includes means for analyzing user input information using natural language processing and evaluating its accuracy, means for detecting the user's emotional state and providing optimized support information based on that state, and means for collecting user feedback data and using it to improve the overall system. This enables more effective and adaptive item management for users.

[0556] "Natural language processing means" refers to technologies that analyze user input information and accurately understand its intent and meaning.

[0557] "Visual information processing means" refers to a technology for analyzing image data of an object and extracting useful information about that object from it.

[0558] An "automatic generation method" is a technology that automatically creates documents such as reports based on predefined information and conditions, and presents them to relevant parties.

[0559] "Optimization methods" refer to techniques that analyze data and propose and implement optimal management methods based on that analysis, in order to efficiently operate policies and processes in inventory management.

[0560] "Means for detecting emotional states" refers to technologies that recognize users' emotions from their actions and inputs, and then quantify or classify them.

[0561] "Means of providing support information" refers to technologies that provide information and suggestions tailored to the user's emotional state in a convenient format.

[0562] "Means of collecting feedback data" refers to technologies that systematically collect data based on user behavior and reactions, and use it for subsequent analysis and system improvement.

[0563] This item management system is server-centric and can flexibly respond to user operations and requests. The server uses natural language processing technology to understand instructions and questions sent by users from their terminals. This technology, for example, utilizes natural language processing libraries to analyze information and perform processing corresponding to the instructions. Furthermore, it uses visual information processing to extract item information from image data, enabling accurate item registration and management. This includes image recognition technology and integration with databases.

[0564] An emotion engine is built in, and the server detects the user's emotional state in real time based on their input. Based on this data, a generative AI model can be used to provide support information optimized for the user's emotions. In this process, for example, a language model is used as the AI ​​model to generate appropriate support messages and suggestions that match the user's emotions.

[0565] The terminal visually displays information provided by the server, allowing users to easily register, use, and return items. Feedback data is collected via the terminal and analyzed on the server. This contributes to improving the overall system performance and user experience.

[0566] For example, if a user expresses dissatisfaction upon receiving a notification on their device that "the wireless mouse is out of stock," the emotion engine recognizes this emotion, and the server presents appropriate alternatives, such as "suggesting the use of a similar product" or "recommending a restock notification." Such suggestions are made by inputting a prompt into the generative AI model that instructs it to "analyze the emotional response when the user checks the stock and generate relevant support messages."

[0567] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0568] Step 1:

[0569] The user uses a terminal to input information regarding the registration, use, and return of items. The entered data is received by the terminal and prepared to be sent to the server. Here, the user is instructed to check the availability of a wireless mouse. The terminal formats this instruction as an HTTP request and sends it to the server.

[0570] Step 2:

[0571] The server receives HTTP requests sent from terminals and parses the instructions using natural language processing. It processes the user's instructions received as input using text analysis techniques and converts the instructions into a format suitable for database searches. This analysis allows the server to quickly identify the necessary information.

[0572] Step 3:

[0573] Based on the analysis results, the server retrieves inventory information for the target item from the database. In this case, the number of wireless mice in stock is searched from the database. The inventory information is obtained as output and used for the next process. A database query is used to retrieve the current status of the item and prepare to inform the user.

[0574] Step 4:

[0575] The server sends input information to the emotion engine to detect the user's emotional state. The emotion engine analyzes the user's emotional response to the input (e.g., frustration or confusion) and returns the results to the server. This emotional data is used as a basis for determining when providing support information.

[0576] Step 5:

[0577] The server generates support messages using a generative AI model based on the output of the emotion engine. The input is the result of the emotion analysis, and the output is the generated support message. For example, if "wireless mouse is out of stock," it will generate suggestions for similar products and suggested settings for restock notifications. The prompt used for this generation is "Analyze the emotional response when the user checks the stock and generate relevant support messages."

[0578] Step 6:

[0579] The server sends the generated support message to the terminal. The terminal receives this message and displays it in an easy-to-understand format for the user. Suggestions and information are presented visually and clearly through the user interface.

[0580] Step 7:

[0581] The user reviews the support message displayed on the device and selects the next action. For example, they might decide to consider using a similar product or set up a restock notification. The device then sends the user's selection back to the server as feedback, and the server uses this data to improve the system.

[0582] (Application Example 2)

[0583] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0584] In inventory management systems, simply managing inventory and items without considering user emotions can lead to user dissatisfaction and stress. This can result in decreased customer satisfaction and inefficient management operations. Furthermore, to provide appropriate responses, a system is needed that can analyze users' emotional states in real time and automatically propose appropriate response strategies.

[0585] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0586] In this invention, the server includes means for analyzing user input information using natural language processing means and evaluating its accuracy, means for analyzing image data of objects using visual information processing means and extracting object information, means for generating reports based on inventory information using automatic generation means and transmitting them to the administrator, and means for analyzing the user's emotional state in real time using emotion analysis means and automatically proposing a response plan. This enables flexible and efficient responses that take into account the user's emotional state.

[0587] "Natural language processing means" refers to technologies that analyze input information such as speech and text from users and understand its accuracy and content.

[0588] "Visual information processing means" refers to a technology that analyzes image data of objects acquired from cameras or image sensors and extracts useful information from it.

[0589] "Automated generation means" refers to technology that automatically generates reports and proposals based on inventory information and user requests, and then communicates them to relevant parties.

[0590] An "optimization method" is a technology that generates an object management policy based on acquired information and makes adjustments to achieve efficient object management.

[0591] "Emotion analysis tools" are technologies that analyze the emotions expressed by users in real time and automatically suggest response strategies and methods of addressing them.

[0592] The system that realizes this invention consists of a server, a terminal used by the user, and a suite of software that supports them. The server is equipped with natural language processing means, visual information processing means, automatic generation means, optimization means, and sentiment analysis means, and uses these to respond to various requests from the user.

[0593] The servers run on cloud services such as Amazon Web Services and Google Cloud Platform. For natural language processing, the Google Cloud Natural Language API is used to analyze user voice and text data. For visual information processing, Microsoft Azure's Computer Vision API is used to extract necessary information from image data input by the user.

[0594] The automated generation method utilizes a TensorFlow-based AI model to automatically create reports and proposals based on inventory information and user preferences. Furthermore, the optimization method generates efficient system operation policies based on the acquired data.

[0595] The emotion analysis system analyzes the user's facial expressions and input in real time to detect the stress and dissatisfaction they are experiencing. This analysis allows the server to quickly provide an ideal response tailored to the user's emotional state. This enables store employees and managers using the terminals to achieve smoother communication with customers.

[0596] A concrete example is its use in a busy store. For instance, when a store employee uses Applia to process a customer's question, "Is this product out of stock?", the system immediately suggests similar products and recommends setting up restock notifications. Furthermore, by providing prompts to the generating AI model such as, "The user is dissatisfied with this product. Please suggest similar products and explain how to set up restock notifications," the AI ​​can design an appropriate response.

[0597] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0598] Step 1:

[0599] Users use their devices to make inquiries about products via voice or text. This input is sent from the device to the server. The server uses the Google Cloud Natural Language API to process this input data in natural language and parse the inquiry. It receives voice or text data as input and obtains the product inquiry as output.

[0600] Step 2:

[0601] The server queries the product inventory database to check the stock status of the queried product. If it is found that the product is out of stock during this process, the server uses a generative AI model powered by TensorFlow to automatically generate suggestions for similar alternative products. It receives information about the queried product as input and generates suggestions for alternative products as output.

[0602] Step 3:

[0603] Using emotion analysis tools, the server analyzes the user's emotional state at the time of their inquiry. It uses Microsoft Azure's Computer Vision API to determine the user's emotional state from their facial expressions and voice. It receives user image and audio data as input and outputs information about the user's emotional state.

[0604] Step 4:

[0605] The server automatically generates the optimal response based on the user's emotional state and inventory status. This response includes suggestions for similar products and instructions on how to set up restock notifications. The generated suggestion text is adjusted according to the user's emotional state. It takes emotional state and inventory information as input and generates a customized suggestion text as output.

[0606] Step 5:

[0607] The generated response is sent from the server to the terminal and presented to the user. Based on this, the user can choose the next action. Here, the suggestion sent from the server is displayed on the terminal. The generated suggestion is received as input and visually presented to the user as output.

[0608] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0609] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0610] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0611] [Fourth Embodiment]

[0612] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0613] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0614] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0615] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0616] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0617] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0618] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0619] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0620] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0621] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0622] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0623] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0624] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0625] This invention relates to a system that highly automates the inventory management process using AI technology. This system includes natural language processing means, visual information processing means, automatic generation means, and optimization means. The following describes how each component of the system functions.

[0626] The server is the core of the system. The server accesses the database for managing items and updates and manages the data in real time. First, the user enters instructions for registering new items, using them, or returning them via a terminal. The terminal sends the entered information to the server, which then processes it.

[0627] Specifically, when a user registers a new item, the server uses natural language processing to analyze the input information and returns instructions prompting correction if there are any errors. Furthermore, the user takes an image of the item to be registered using their device and simultaneously sends the image data to the server. The server utilizes visual information processing to automatically extract item information from the image and accurately records the information in the database.

[0628] Furthermore, regarding inventory management, the server periodically collects inventory information, uses an automated generation system to create inventory reports, and submits them to the administrator. These reports contain detailed information on missing items and items that have been unused for a long time.

[0629] By applying optimization techniques, the server generates inventory management policies based on data and proposes to the user, for example, ways to effectively utilize items that have been unused for a long period. In this way, the system reduces errors in inventory management and achieves efficient resource utilization.

[0630] For example, if a user enters "I want to reserve a projector for a meeting" into the terminal, the server uses natural language processing to analyze the instruction and check inventory. If a projector is available, the server approves the request and updates the reservation information in the database. This series of processes significantly improves the accuracy and efficiency of inventory management.

[0631] The following describes the processing flow.

[0632] Step 1:

[0633] The user enters a request to register a new item via their terminal. This includes entering the item name, category, quantity, and image.

[0634] Step 2:

[0635] The terminal receives input data from the user and sends the data to the server.

[0636] Step 3:

[0637] The server receives the data, analyzes the text information using natural language processing tools, and verifies that there are no errors in the input data.

[0638] Step 4:

[0639] If the server analyzes the input and finds any errors, it will return instructions to the user for correction. If the input is correct, it will proceed to the next step.

[0640] Step 5:

[0641] The user takes a picture of an item with their device and sends the image data to the server.

[0642] Step 6:

[0643] The server uses visual information processing to analyze image data and automatically extracts item information.

[0644] Step 7:

[0645] If the extracted information is deemed correct, the server records the new item information in the database.

[0646] Step 8:

[0647] When a user requests the use of an item, they enter instructions in natural language, such as "Use the projector for the meeting."

[0648] Step 9:

[0649] The terminal sends this request to the server, which uses LLM to analyze the instructions and understand the requirements.

[0650] Step 10:

[0651] The server checks the database to see if the requested item is in stock, updates the reservation information if available, and sends an availability notification to the user.

[0652] Step 11:

[0653] The server periodically performs inventory checks, organizes inventory information using an automated generation system, and sends reports to the administrator.

[0654] Step 12:

[0655] The server plans the appropriate disposal of unused items and makes suggestions to the administrator regarding items that have not been used for a long period of time.

[0656] (Example 1)

[0657] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0658] Traditional inventory management systems face challenges such as incorrect data entry, timely inventory updates, and a lack of effective ways to utilize unused resources. This reduces management efficiency and hinders the efficient use of resources. It is necessary to address these issues and achieve automation and optimization of inventory management.

[0659] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0660] This invention includes a server that analyzes user input information using an AI model and provides feedback for error detection and correction, a server that analyzes image data of items using visual information processing and extracts features, and an automated generation means that creates an inventory report based on inventory information and provides it to the administrator. This enables the automation of the item management process, efficient management, and effective use of resources.

[0661] "Generating AI models" refers to artificial intelligence technology that interprets natural language instructions from users and performs error detection and correction.

[0662] "Visual information processing" is a technology that extracts features from image data of objects and updates databases based on that information.

[0663] "Automatic generation means" refers to a system function that periodically collects inventory information, creates inventory reports, and provides them to administrators.

[0664] "Optimization" refers to the design of inventory management policies and methods for efficiently utilizing resources.

[0665] A "terminal" refers to an input / output device used by users to input instructions for managing items.

[0666] "Administrator" refers to the entity responsible for operating the system and receiving the results of inventory management.

[0667] A "database" is a digital repository that systematically stores information and inventory data about goods, and allows read and write access as needed.

[0668] This invention relates to a system that automates and optimizes inventory management using AI technology. The system is operated by a server, terminals, and users. The server significantly improves the efficiency of inventory management by utilizing technologies such as AI models, visual information processing, automatic data generation, and optimization.

[0669] The server receives natural language instructions entered by the user through the terminal. The input information is analyzed using a generative AI model, which has the function of detecting errors and providing feedback for correction. For example, if the user enters "I want to register a new projector for the conference room," the server analyzes this instruction and requests the necessary item information.

[0670] Next, the user takes a picture of the item using a device. The captured image data is sent to the server, which extracts the item's features from the image using visual information processing. This allows accurate item information to be registered in the database from the image data.

[0671] Furthermore, the server uses automated generation methods to create inventory reports from periodic inventory data and submit them to the administrator. These reports include detailed information on insufficient stock and unused items. As a result, administrators can make appropriate inventory decisions.

[0672] The server also optimizes based on accumulated data and suggests efficient ways to use resources to users. For example, it can suggest reallocating resources that have been unused for a long period for meetings. This suggestion promotes the effective use of resources throughout the organization.

[0673] An example of a prompt generated by the AI ​​model might be, "Please enter the information of the item you wish to register in natural language. Then, take a picture of the item and send it." This allows users to intuitively proceed through the item management process.

[0674] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0675] Step 1:

[0676] The user uses a terminal to input instructions in natural language regarding the registration, use, or return of an item. Specifically, the user inputs text information such as "I want to register a new projector for meeting room 2." This serves as initial input, informing the server of which item the user wants to use and how.

[0677] Step 2:

[0678] The terminal sends the information entered by the user to the server. Specifically, the entered natural language text data is passed directly to the server via the communication channel. The server receives this data and prepares for the next processing step.

[0679] Step 3:

[0680] The server analyzes the received natural language input using a generative AI model. The AI ​​model understands the user's intent from the text data and detects errors or omissions as needed. Based on this analysis, the server generates feedback if corrections are necessary and returns it to the user via the terminal. This process involves syntactic analysis of natural language and extraction of information about the type and use of items.

[0681] Step 4:

[0682] The user takes pictures of an object with their device. For example, they might take pictures of the front and back of a projector. The captured images are then sent back to the server by the device. This image data is stored on the server for further processing in the next step.

[0683] Step 5:

[0684] The server analyzes the received image data using visual information processing tools. It extracts characteristic information about the items from the images (for example, shape, manufacturer name, model number, etc.). This role is performed by an image processing algorithm, which automatically recognizes specific features and compares them with an existing database.

[0685] Step 6:

[0686] The server updates or registers new information in the inventory management database based on the analyzed text information and visually extracted information. The updated results are stored as accurate inventory information, which can then be used for future inventory management and usage monitoring.

[0687] Step 7:

[0688] The server uses an automated generation system to create an inventory report based on the latest inventory information and provides it to the administrator. This report can be output as a PDF or spreadsheet, allowing the administrator to verify the inventory. The report includes information such as inventory shortages and items that have been unused for a long period.

[0689] Step 8:

[0690] The server analyzes inventory data using optimization techniques and generates inventory management policies. For example, it might notify users of suggestions for efficient use of projectors. Such suggestions are presented to users via terminals and can be used to help them decide on effective resource utilization policies.

[0691] (Application Example 1)

[0692] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0693] Managing goods in a logistics center is extremely complex, and errors and data delays are particularly serious problems in inventory management. This can prevent the development of appropriate supply plans, potentially resulting in excess or shortages of inventory. Furthermore, managing lost items and long-term unused goods requires considerable effort, so there is a need for efficient solutions to these challenges.

[0694] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0695] In this invention, the server includes means for analyzing user input information using natural language processing technology and evaluating its accuracy, means for analyzing image data of items using visual information processing technology and extracting item information, and means for creating a report based on inventory information using automatic generation technology and transmitting it to the manager. As a result, inventory checks at the logistics center can be performed accurately in real time, and supply plans can be optimized by predicting excess or insufficient inventory in advance.

[0696] "Natural language processing technology" is a technology that analyzes human language, understands its meaning, and processes it.

[0697] "Visual information processing technology" is a technology that analyzes images and videos and extracts useful information from them.

[0698] "Automated generation technology" refers to technology that automatically creates reports and documents based on data.

[0699] "Optimization techniques" are techniques that involve calculations and adjustments to obtain the best possible results under given conditions.

[0700] A "resource management policy" is a guideline that outlines the most efficient methods for managing goods and inventory.

[0701] "Excess inventory" refers to a situation where more inventory is held than is needed.

[0702] "Inventory shortage" refers to a situation where there is insufficient inventory to meet demand.

[0703] A "supply plan" is the plan for how to supply goods and inventory.

[0704] This invention aims to realize an inventory management system in a logistics center. The server uses natural language processing technology to analyze voice instructions and text input from the user, and extracts and evaluates the necessary information. Images of items taken with a smartphone are then sent to the server, and item information is extracted via visual information processing technology. Specifically, image analysis is performed using software such as OpenCV and TensorFlow.

[0705] The generated data is documented using automated generation technology as management reports and inventory forecast reports, and communicated to administrators. This primarily involves storing and managing data using a database management system located on a cloud server, such as MySQL.

[0706] The server uses AI models to analyze inventory data, predicting excess and shortages, and creating optimal supply plans. By leveraging optimization technology, efficient resource management becomes possible. In this way, precise inventory management is achieved in the logistics center, promoting the effective use of resources.

[0707] For example, if a user enters the prompt "Please run the process to have the AI ​​calculate the appropriate purchase quantity of product A for the next month" into the terminal, the server will analyze the inventory data for the relevant product and provide a recommended purchase quantity. This process reduces the risk of excess or insufficient inventory and optimizes supply.

[0708] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0709] Step 1:

[0710] The user inputs information about an item via voice or text from their smartphone. Using natural language processing technology, the server analyzes this input and interprets its intent. The input consists of user instructions, such as reserving a projector or checking its availability. The analysis results in the output of specific operation commands and item IDs.

[0711] Step 2:

[0712] The server retrieves existing data about an item from the database according to the user's instructions. The input consists of the item ID and instructions, and the server uses a database system on the cloud server to retrieve the corresponding inventory information and usage history. The output is data indicating the item's current status and availability.

[0713] Step 3:

[0714] The user takes a picture of an item with their smartphone and sends it to the server. Using visual information processing technology, the server extracts identification information of the item from the image. The input is the captured image, which is then analyzed using software such as OpenCV. The output includes the type of item and its individual identifier.

[0715] Step 4:

[0716] The server integrates the analyzed linguistic and visual information to perform item registration and update processes. This includes adding new information to the database and updating existing information. The input to this process is the analyzed linguistic and visual information, and the output is the updated database entry.

[0717] Step 5:

[0718] The server analyzes collected inventory data and uses an AI model to predict future inventory trends. Inputs include data such as inventory levels and usage history, and the AI ​​model is used to forecast demand. Outputs include future demand trends and suggestions for appropriate procurement quantities.

[0719] Step 6:

[0720] The server uses automated generation technology to create an optimized supply plan based on the analysis results from the AI ​​model, and communicates it to the administrator as a report. The input is demand forecast data, and the output is a supply plan document that the administrator can use. This supports appropriate decision-making and action.

[0721] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0722] This invention combines an emotion engine with an item management system to enable flexible responses tailored to the user's emotional state. This system includes natural language processing, visual information processing, automatic generation, and optimization means, in addition to the emotion engine. The specific operation of this system is described below.

[0723] The core of the system is the server. The server works in conjunction with a database that holds various types of data. Users use terminals to perform operations such as registering, using, and returning items. Instructions entered by the user are received by the terminal and sent to the server.

[0724] In particular, the emotion engine analyzes the emotions a user displays during operation in real time. For example, if a user becomes frustrated when their input is not correctly received while registering an item, the emotion engine detects their stress level. This allows the server to present the user with optimized support messages and help resolve the problem.

[0725] For example, if a user expresses dissatisfaction upon receiving a notification that "the wireless mouse is out of stock," the emotion engine recognizes that emotion and presents appropriate alternatives, such as "suggesting the use of a similar product" or "setting up a notification when it is back in stock."

[0726] Furthermore, the emotion engine is involved in supplementing periodic inventory reports. Based on managers' reactions to past reports, the process is automated to identify areas for improvement and adjust new reports to better meet the managers' needs.

[0727] Thus, the present invention not only improves the accuracy and efficiency of inventory management, but also enhances the user experience and realizes a more flexible and adaptive inventory management environment.

[0728] The following describes the processing flow.

[0729] Step 1:

[0730] Users input requests for the use of items in natural language through their terminals. For example, "I would like to use the laptop for next week's meeting."

[0731] Step 2:

[0732] The terminal sends the input information received from the user to the server.

[0733] Step 3:

[0734] The server uses natural language processing to analyze the user's request information and accurately understand their intent.

[0735] Step 4:

[0736] The server checks the database to verify the inventory status and availability of the requested item.

[0737] Step 5:

[0738] Based on the verification results, the server activates an emotion engine to recognize the user's emotions and predict the user's response.

[0739] Step 6:

[0740] When a user receives a request, the system uses the device's camera and microphone to collect emotional data from the user's facial expressions and voice, and sends it to the server.

[0741] Step 7:

[0742] The server uses an emotion engine to analyze the received emotion data and evaluate the user's emotional state. For example, it identifies feelings such as joy, frustration, and anxiety.

[0743] Step 8:

[0744] Based on the evaluation results of the emotion engine, the server determines the optimal course of action to improve the user experience. For example, it may provide comforting messages or alternatives if an option is unavailable.

[0745] Step 9:

[0746] The server sends the sentiment engine's analysis results and a corresponding message to the user's device, which then displays them.

[0747] Step 10:

[0748] The inventory management process proceeds smoothly by allowing the user to select the next action based on the response provided and to re-enter any necessary requests or instructions on the terminal.

[0749] (Example 2)

[0750] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0751] In asset management systems, flexible support that takes into account the emotional state of users is not provided. Therefore, there is a need to achieve efficient asset management while reducing user stress. Furthermore, it is necessary to effectively utilize user feedback and continuously improve the system.

[0752] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0753] In this invention, the server includes means for analyzing user input information using natural language processing and evaluating its accuracy, means for detecting the user's emotional state and providing optimized support information based on that state, and means for collecting user feedback data and using it to improve the overall system. This enables more effective and adaptive item management for users.

[0754] "Natural language processing means" refers to technologies that analyze user input information and accurately understand its intent and meaning.

[0755] "Visual information processing means" refers to a technology for analyzing image data of an object and extracting useful information about that object from it.

[0756] An "automatic generation method" is a technology that automatically creates documents such as reports based on predefined information and conditions, and presents them to relevant parties.

[0757] "Optimization methods" refer to techniques that analyze data and propose and implement optimal management methods based on that analysis, in order to efficiently operate policies and processes in inventory management.

[0758] "Means for detecting emotional states" refers to technologies that recognize users' emotions from their actions and inputs, and then quantify or classify them.

[0759] "Means of providing support information" refers to technologies that provide information and suggestions tailored to the user's emotional state in a convenient format.

[0760] "Means of collecting feedback data" refers to technologies that systematically collect data based on user behavior and reactions, and use it for subsequent analysis and system improvement.

[0761] This item management system is server-centric and can flexibly respond to user operations and requests. The server uses natural language processing technology to understand instructions and questions sent by users from their terminals. This technology, for example, utilizes natural language processing libraries to analyze information and perform processing corresponding to the instructions. Furthermore, it uses visual information processing to extract item information from image data, enabling accurate item registration and management. This includes image recognition technology and integration with databases.

[0762] An emotion engine is built in, and the server detects the user's emotional state in real time based on their input. Based on this data, a generative AI model can be used to provide support information optimized for the user's emotions. In this process, for example, a language model is used as the AI ​​model to generate appropriate support messages and suggestions that match the user's emotions.

[0763] The terminal visually displays information provided by the server, allowing users to easily register, use, and return items. Feedback data is collected via the terminal and analyzed on the server. This contributes to improving the overall system performance and user experience.

[0764] For example, if a user expresses dissatisfaction upon receiving a notification on their device that "the wireless mouse is out of stock," the emotion engine recognizes this emotion, and the server presents appropriate alternatives, such as "suggesting the use of a similar product" or "recommending a restock notification." Such suggestions are made by inputting a prompt into the generative AI model that instructs it to "analyze the emotional response when the user checks the stock and generate relevant support messages."

[0765] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0766] Step 1:

[0767] The user uses a terminal to input information regarding the registration, use, and return of items. The entered data is received by the terminal and prepared to be sent to the server. Here, the user is instructed to check the availability of a wireless mouse. The terminal formats this instruction as an HTTP request and sends it to the server.

[0768] Step 2:

[0769] The server receives HTTP requests sent from terminals and parses the instructions using natural language processing. It processes the user's instructions received as input using text analysis techniques and converts the instructions into a format suitable for database searches. This analysis allows the server to quickly identify the necessary information.

[0770] Step 3:

[0771] Based on the analysis results, the server retrieves inventory information for the target item from the database. In this case, the number of wireless mice in stock is searched from the database. The inventory information is obtained as output and used for the next process. A database query is used to retrieve the current status of the item and prepare to inform the user.

[0772] Step 4:

[0773] The server sends input information to the emotion engine to detect the user's emotional state. The emotion engine analyzes the user's emotional response to the input (e.g., frustration or confusion) and returns the results to the server. This emotional data is used as a basis for determining when providing support information.

[0774] Step 5:

[0775] The server generates support messages using a generative AI model based on the output of the emotion engine. The input is the result of the emotion analysis, and the output is the generated support message. For example, if "wireless mouse is out of stock," it will generate suggestions for similar products and suggested settings for restock notifications. The prompt used for this generation is "Analyze the emotional response when the user checks the stock and generate relevant support messages."

[0776] Step 6:

[0777] The server sends the generated support message to the terminal. The terminal receives this message and displays it in an easy-to-understand format for the user. Suggestions and information are presented visually and clearly through the user interface.

[0778] Step 7:

[0779] The user reviews the support message displayed on the device and selects the next action. For example, they might decide to consider using a similar product or set up a restock notification. The device then sends the user's selection back to the server as feedback, and the server uses this data to improve the system.

[0780] (Application Example 2)

[0781] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0782] In inventory management systems, simply managing inventory and items without considering user emotions can lead to user dissatisfaction and stress. This can result in decreased customer satisfaction and inefficient management operations. Furthermore, to provide appropriate responses, a system is needed that can analyze users' emotional states in real time and automatically propose appropriate response strategies.

[0783] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0784] In this invention, the server includes means for analyzing user input information using natural language processing means and evaluating its accuracy, means for analyzing image data of objects using visual information processing means and extracting object information, means for generating reports based on inventory information using automatic generation means and transmitting them to the administrator, and means for analyzing the user's emotional state in real time using emotion analysis means and automatically proposing a response plan. This enables flexible and efficient responses that take into account the user's emotional state.

[0785] "Natural language processing means" refers to technologies that analyze input information such as speech and text from users and understand its accuracy and content.

[0786] "Visual information processing means" refers to a technology that analyzes image data of objects acquired from cameras or image sensors and extracts useful information from it.

[0787] "Automated generation means" refers to technology that automatically generates reports and proposals based on inventory information and user requests, and then communicates them to relevant parties.

[0788] An "optimization method" is a technology that generates an object management policy based on acquired information and makes adjustments to achieve efficient object management.

[0789] "Emotion analysis tools" are technologies that analyze the emotions expressed by users in real time and automatically suggest response strategies and methods of addressing them.

[0790] The system that realizes this invention consists of a server, a terminal used by the user, and a suite of software that supports them. The server is equipped with natural language processing means, visual information processing means, automatic generation means, optimization means, and sentiment analysis means, and uses these to respond to various requests from the user.

[0791] The servers run on cloud services such as Amazon Web Services and Google Cloud Platform. For natural language processing, the Google Cloud Natural Language API is used to analyze user voice and text data. For visual information processing, Microsoft Azure's Computer Vision API is used to extract necessary information from image data input by the user.

[0792] The automated generation method utilizes a TensorFlow-based AI model to automatically create reports and proposals based on inventory information and user preferences. Furthermore, the optimization method generates efficient system operation policies based on the acquired data.

[0793] The emotion analysis system analyzes the user's facial expressions and input in real time to detect the stress and dissatisfaction they are experiencing. This analysis allows the server to quickly provide an ideal response tailored to the user's emotional state. This enables store employees and managers using the terminals to achieve smoother communication with customers.

[0794] A concrete example is its use in a busy store. For instance, when a store employee uses Applia to process a customer's question, "Is this product out of stock?", the system immediately suggests similar products and recommends setting up restock notifications. Furthermore, by providing prompts to the generating AI model such as, "The user is dissatisfied with this product. Please suggest similar products and explain how to set up restock notifications," the AI ​​can design an appropriate response.

[0795] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0796] Step 1:

[0797] Users use their devices to make inquiries about products via voice or text. This input is sent from the device to the server. The server uses the Google Cloud Natural Language API to process this input data in natural language and parse the inquiry. It receives voice or text data as input and obtains the product inquiry as output.

[0798] Step 2:

[0799] The server queries the product inventory database to check the stock status of the queried product. If it is found that the product is out of stock during this process, the server uses a generative AI model powered by TensorFlow to automatically generate suggestions for similar alternative products. It receives information about the queried product as input and generates suggestions for alternative products as output.

[0800] Step 3:

[0801] Using emotion analysis tools, the server analyzes the user's emotional state at the time of their inquiry. It uses Microsoft Azure's Computer Vision API to determine the user's emotional state from their facial expressions and voice. It receives user image and audio data as input and outputs information about the user's emotional state.

[0802] Step 4:

[0803] The server automatically generates the optimal response based on the user's emotional state and inventory status. This response includes suggestions for similar products and instructions on how to set up restock notifications. The generated suggestion text is adjusted according to the user's emotional state. It takes emotional state and inventory information as input and generates a customized suggestion text as output.

[0804] Step 5:

[0805] The generated response is sent from the server to the terminal and presented to the user. Based on this, the user can choose the next action. Here, the suggestion sent from the server is displayed on the terminal. The generated suggestion is received as input and visually presented to the user as output.

[0806] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0807] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0808] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0809] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0810] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0811] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0812] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0813] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0814] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0815] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0816] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0817] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0818] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0819] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0820] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0821] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0822] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0823] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0824] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0825] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0826] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0827] The following is further disclosed regarding the embodiments described above.

[0828] (Claim 1)

[0829] A means for analyzing user input information using natural language processing and evaluating its accuracy,

[0830] A means for analyzing image data of an object using visual information processing means and extracting information about the object,

[0831] A means of generating reports based on inventory information using an automated generation method and communicating them to the manager,

[0832] A means for generating an inventory management policy using optimization means and realizing efficient inventory management,

[0833] A system that includes this.

[0834] (Claim 2)

[0835] The system according to claim 1, which automatically provides processing suggestions for items that have been unused for a long period of time.

[0836] (Claim 3)

[0837] The system according to claim 1, which automatically generates procedural documents regarding lost items and notifies the relevant parties.

[0838] "Example 1"

[0839] (Claim 1)

[0840] A means of analyzing user input information using an AI model, detecting errors, and providing feedback for correction,

[0841] A means for analyzing image data of an object using visual information processing and extracting its features,

[0842] A means of creating an inventory report based on inventory information using an automated generation method and providing it to the administrator,

[0843] A means of creating inventory management policies through optimization and promoting the efficient use of resources,

[0844] A means by which users input instructions for managing items from a terminal and reflect those instructions in a database in real time,

[0845] A system that includes this.

[0846] (Claim 2)

[0847] The system according to claim 1, which automatically proposes methods for effectively utilizing resources that have been unused for a long period of time.

[0848] (Claim 3)

[0849] The system according to claim 1, which automatically generates procedures for lost resources and distributes them to relevant parties.

[0850] "Application Example 1"

[0851] (Claim 1)

[0852] A means of analyzing user input information using natural language processing technology and evaluating its accuracy,

[0853] A means for analyzing image data of an object using visual information processing technology and extracting information about that object,

[0854] A means of creating reports based on inventory information using automated generation technology and communicating them to managers,

[0855] A means of generating resource management policies using optimization technology and achieving efficient inventory management,

[0856] A means of checking inventory in real time based on visual information and providing accurate inventory information,

[0857] A means of predicting excess or shortage inventory based on data analysis and proposing an optimal supply plan,

[0858] A system that includes this.

[0859] (Claim 2)

[0860] The system according to claim 1, which automatically proposes processing for items that have been unused for a long period of time, and also includes proposing an efficient supply plan.

[0861] (Claim 3)

[0862] The system according to claim 1, which includes a procedure for automatically generating procedural documents regarding lost items and notifying relevant parties.

[0863] "Example 2 of combining an emotion engine"

[0864] (Claim 1)

[0865] A means for analyzing user input information using natural language processing and evaluating its accuracy,

[0866] A means for analyzing image data of an object using visual information processing means and extracting information about the object,

[0867] A means of generating reports based on inventory information using an automated generation method and communicating them to the manager,

[0868] A means for generating an inventory management policy using optimization means and realizing efficient inventory management,

[0869] A means for detecting the emotional state of a user and providing optimized support information based on that state,

[0870] A means of collecting feedback data from users and using it to improve the entire system,

[0871] A system that includes this.

[0872] (Claim 2)

[0873] The system according to claim 1, which automatically provides processing suggestions for items that have been unused for a long period of time.

[0874] (Claim 3)

[0875] The system according to claim 1, which automatically generates procedural documents regarding lost items and notifies the relevant parties.

[0876] "Application example 2 when combining with an emotional engine"

[0877] (Claim 1)

[0878] A means for analyzing user input information using natural language processing and evaluating its accuracy,

[0879] A means for analyzing image data of an object using visual information processing means and extracting object information,

[0880] A means of generating reports based on inventory information using an automated generation method and communicating them to the manager,

[0881] A means for generating an object management policy using optimization means and realizing efficient object management,

[0882] A means for analyzing a user's emotional state in real time using emotion analysis tools and automatically proposing a response strategy,

[0883] A system that includes this.

[0884] (Claim 2)

[0885] The system according to claim 1, which automatically provides processing suggestions for objects that have not been used for a long period of time.

[0886] (Claim 3)

[0887] The system according to claim 1, which automatically generates procedural documents regarding a lost object and notifies the relevant parties. [Explanation of symbols]

[0888] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for analyzing user input information using natural language processing and evaluating its accuracy, A means for analyzing image data of an object using visual information processing means and extracting information about the object, A means of generating reports based on inventory information using an automated generation method and communicating them to the manager, A means for generating an inventory management policy using optimization means and realizing efficient inventory management, A system that includes this.

2. The system according to claim 1, which automatically provides processing suggestions for items that have been unused for a long period of time.

3. The system according to claim 1, which automatically generates procedural documents regarding lost items and notifies the relevant parties.

Images