system

Abstract

We provide the system. [Solution] Means of acquiring data, A means for performing revenue forecasting based on the aforementioned data, A means for optimizing the work plan and the machine operation plan based on the aforementioned revenue forecast, Means for visualizing and modifying the optimized work plan and machine operation plan, A method for improving predictive models using feedback data, Means for viewing and modifying the aforementioned work plan in real time via a mobile device, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the food and beverage industry, although the introduction of mechanical devices is expected to reduce labor costs, the increase in fixed costs and the inefficient operation of mechanical devices become problems. Furthermore, there are insufficient means to optimize the work shifts of employees and the operation of mechanical devices, which may lead to a decline in service quality for customers. Therefore, it is required to integrally optimize the operation of mechanical devices and work shifts, improve the profit and efficiency of stores, and provide a comfortable working environment.

Means for Solving the Problems

[0005] The present invention includes means for collecting data, means for making sales forecasts based on the data, means for optimizing work shifts and machinery operation plans based on the sales forecasts, and means for outputting and adjusting the optimized work shifts and machinery operation plans. Furthermore, it includes means for updating the forecast model using feedback data, and acquires feedback data based on actual sales information and the operating status of machinery. This system aims to achieve efficient store operations and improve profits.

[0006] "Data collection means" refers to a device or system that has the function of acquiring various data necessary for operational optimization, such as sales information, weather information, event information, and work preference information.

[0007] A "sales forecasting method" refers to a process and its execution device for predicting future sales based on collected data, using machine learning algorithms, etc.

[0008] "Work shift optimization means" refers to a process and implementation device for formulating optimal work shifts and staffing arrangements based on sales forecast results.

[0009] A "machinery operation plan" refers to the guidelines and plans for formulating the operation schedule of the machinery and equipment used in a store, and for ensuring optimal placement and operation.

[0010] "Feedback data" refers to actual sales information and data on the operating status of machinery and equipment, and is used as performance data to improve predictive models.

[0011] A "predictive model updating means" is a process or apparatus for improving the accuracy of a sales forecast model by utilizing acquired feedback data. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a tagged processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, a tagged RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, a tagged storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, a tagged communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention provides a system for optimizing staff work shifts and machinery operation plans in restaurants. The system consists of server, terminal, and user elements, each playing a specific role.

[0034] The server first uses data collection tools to acquire store sales information, weather information, event information, and employee work preference information. This data is processed by sales forecasting tools, which in particular use machine learning algorithms to predict future sales. Based on this forecast information, the server uses work shift optimization tools to calculate the optimal staffing arrangement and further develops a machine operation plan.

[0035] Store managers, as users, can review the optimized shifts and operational plans provided by the server and manually adjust them as needed. This allows users to implement operations that reflect the needs of the store staff.

[0036] Terminals, or the machines operating within the store, prepare themselves based on the operational schedule received from the server and then perform food serving tasks. As a concrete example, multiple terminals can be operated during peak daytime hours to compensate for labor shortages, while only the minimum necessary terminals can be operated during off-peak nighttime hours, thereby achieving efficient operation.

[0037] Furthermore, the server collects feedback data after business hours and analyzes predictions and actual results. This allows for the improvement of the sales forecast model using forecast model update mechanisms, resulting in more accurate plans for future shifts and machine operation.

[0038] As described above, the present invention provides an integrated solution for improving the operational efficiency of stores, simultaneously achieving reductions in labor costs and improvements in service quality.

[0039] The following describes the processing flow.

[0040] Step 1:

[0041] The server uses data collection methods to gather historical sales information, weather forecast data, store event information, and employee work preference data. This includes using various APIs and accessing databases.

[0042] Step 2:

[0043] The server activates the sales forecasting system and applies machine learning algorithms based on the collected data to predict future sales. The sales forecasting model is created based on historical data and updated as needed.

[0044] Step 3:

[0045] The server uses a work shift optimization method based on predicted sales data to calculate the optimal staffing. At the same time, it formulates a machine operation plan and determines the number of machines and their operating hours to be used in each time slot.

[0046] Step 4:

[0047] Store managers, who are the users, are notified of the optimized work shifts and machine operation plans calculated by the server. Users can review this information and adjust shifts and plans as needed.

[0048] Step 5:

[0049] The terminal begins preparations based on the operational schedule received from the server. Following the specified schedule, it starts serving meals and efficiently delivers them according to a pre-configured workflow.

[0050] Step 6:

[0051] After business hours, the server aggregates feedback data. This includes sales data for the day, machine operation status, and employee work performance.

[0052] Step 7:

[0053] The server updates its sales forecasting model and shift optimization algorithm based on feedback data. This improves the model so that future predictions and optimizations are more accurate.

[0054] (Example 1)

[0055] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0056] In store operations such as restaurants and retail, optimal staffing and efficient use of machinery are required. However, current methods make accurate sales forecasting difficult, leading to excessive labor costs and a decline in service quality. Furthermore, planning that takes into account external factors such as fluctuating weather and events is insufficient, thus requiring practical and flexible operational methods.

[0057] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0058] In this invention, the server includes means for collecting commercial transaction information, weather information, event information, and employee request information via a data processing device; means for predicting future sales using a machine learning algorithm based on the collected information; and means for optimizing labor allocation and machinery operation plans based on the predictions. This enables flexible and efficient store operations and staffing in response to fluctuating demand.

[0059] A "data processing device" is an information processing system used to collect and analyze commercial transaction information, weather information, event information, and employee request information.

[0060] "Commercial transaction information" refers to data on sales and transactions at stores, and is used for business analysis and sales forecasting.

[0061] "Weather information" refers to data on meteorological conditions and is used for analysis as a factor that influences sales and customer numbers.

[0062] "Event information" refers to information about events and activities held in the local area, which contributes to predicting customer traffic and managing resources for stores.

[0063] "Employee request information" refers to information regarding staff work preferences and shifts, and serves as basic data for optimizing labor allocation.

[0064] A "machine learning algorithm" refers to a computational method used to find patterns in input data and make predictions or classifications about the future.

[0065] "Labor allocation" refers to the planning of employee work shifts and assigned tasks in store operations.

[0066] "Machinery and equipment" is a general term for automated devices and systems used in stores, and specifically includes serving robots, vending machines, etc.

[0067] An "operational plan" refers to a plan for efficiently carrying out operations through the appropriate allocation and management of resources.

[0068] This invention is a system for optimizing labor allocation and machinery operation planning in store operations. The system consists of server, terminal, and user elements.

[0069] The server collects information via data processing devices. Specifically, commercial transaction information is obtained from the store's sales information management system, and weather information is obtained using a weather data provision API. Event information is obtained through an event data service, and employee request information is obtained from the labor management system via an API.

[0070] The server uses this data to perform sales forecasts using software for executing machine learning algorithms, such as TENSORFLOW® or PyTorch. Based on the predicted sales data, the server uses a linear programming algorithm to formulate optimal work shifts and machine operation plans.

[0071] Store managers, as users, can view these plans provided by the server on their management terminals and make adjustments as needed through the interface. These adjustments are intuitive and can be performed using a GUI, making it easy for users without special technical knowledge to operate.

[0072] Terminals, or in-store devices, automatically prepare and begin operations according to the operational schedule received from the server. A specific example is that multiple serving robots operate during peak hours, while only the minimum necessary number operate during off-peak hours. This enables efficient resource management.

[0073] Furthermore, after the end of the workday, the server accumulates feedback data on the execution results and updates the predictive model based on this data. Through this process, the accuracy of the predictions improves over time, which is useful for formulating future plans.

[0074] As a concrete example, it is possible to optimize the staffing and machine operation plans for a specific store on days when sunny weather is forecast for the weekend. An example of a prompt message could be, "Please provide an optimal staffing and serving robot deployment plan based on the predicted number of customers on a sunny Sunday next week." Based on such input, the system will propose the optimal plan.

[0075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0076] Step 1:

[0077] The server collects commercial transaction information, weather information, event information, and employee request information via a data processing device. Inputs include data from POS systems, weather APIs, event information services, and labor management systems. The output is an integrated dataset, ready for subsequent processing such as analysis and prediction. Specifically, it uses an authentication key to access data from APIs and stores the obtained information in a database.

[0078] Step 2:

[0079] The server executes a machine learning algorithm using the collected data. The input here is the integrated dataset obtained in Step 1. Data processing involves encoding categorical data and imputing missing values ​​to format the data into an analyzable format. The output is future sales forecast data. In this process, TensorFlow is used to apply a trained model and generate predicted values. Specifically, the server performs CSV file validation as data preprocessing and handles outliers as needed.

[0080] Step 3:

[0081] The server optimizes labor allocation and machinery operation plans based on predicted sales data. The input is the sales forecast data obtained in step 2. The data calculations include an optimization algorithm using linear programming. The output is a specific and optimized shift allocation and operation schedule. In this process, shifts are adjusted to comply with employee labor laws and meet their desired working hours. Specifically, it performs resource allocation simulations based on a numerical model, distinguishing between peak and off-peak periods.

[0082] Step 4:

[0083] The store manager, as the user, reviews the provided work shifts and operational plans and makes adjustments as needed. The input is the plan data from the server, and the output is the adjusted final plan. Specifically, the shift schedule can be viewed on the management screen, and working hours can be edited using drag-and-drop operations via the GUI. These adjustments are made to reflect on-site situations, such as leave requests from specific staff members.

[0084] Step 5:

[0085] The terminal starts operating the machinery according to the operational schedule received from the server. The input is the final operational plan from step 4, and the output is the operational data of the machinery. Specifically, the terminal automatically turns on the power to the serving robot and starts the initialization process. This enables efficient work execution within the set time.

[0086] Step 6:

[0087] The server collects actual sales data and machine operation data as feedback after the end of operations. The input is actual sales and operation performance data obtained from stores, and the output is real-time data for the improved prediction algorithm and the next operation plan. The specific operation includes a process of detecting outliers and deviations from the past through a database update at night and correcting the prediction model for the next operation.

[0088] (Application Example 1)

[0089] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0090] In physical stores, there is a need to efficiently optimize staff work schedules and machine operation schedules, and to enable real-time adjustments. However, conventional systems have problems such as insufficient accuracy in revenue forecasting and difficulty in easily modifying plans. This has led to decreased operational efficiency in physical stores and the resulting increase in extra effort and costs.

[0091] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0092] In this invention, the server includes means for acquiring data, means for performing revenue forecasts based on the data, means for optimizing work plans and machine operation plans based on the revenue forecasts, means for visualizing and making modifiable the optimized work plans and machine operation plans, means for improving the forecast model using feedback data, and means for viewing and modifying the work plans in real time via a mobile terminal. This enables efficient optimization and flexible modification of plans in physical stores.

[0093] "Means of acquiring data" refers to elements that have the function of collecting various types of information necessary for use in revenue forecasting.

[0094] "Means for performing revenue forecasting" refers to elements that perform calculations to predict future revenue based on collected data.

[0095] "Means for optimizing work plans and machine operation plans" refer to elements that efficiently set personnel allocations and machine operation based on projected revenue.

[0096] "Means of visualization and modification" refer to elements that visually display the optimized plan and allow users to easily make changes.

[0097] "Methods for improving predictive models using feedback data" refer to elements that involve aggregating actual operational results and updating predictive algorithms to improve the accuracy of revenue forecasts.

[0098] "Means of viewing and modifying in real time via mobile devices" refers to elements that enable users to check plans in real time using smartphones or tablets and make adjustments as needed.

[0099] The server provides an integrated system to improve the operational efficiency of the stores. First, it collects store sales information, weather information, event information, and part-time staff work preference information through data acquisition methods. This data is stored in a cloud-hosted database and processed by a sales forecasting system. This processing utilizes machine learning algorithms and the TensorFlow library using Python to predict future sales with high accuracy.

[0100] The server has the means to optimize work plans and machine operation plans based on the acquired predictive information. The Django framework is used for communication between the server and the frontend, supporting data movement within the system. Once the optimization process is complete, the plan is visualized through a user interface developed with React Native, allowing store managers to review it via their mobile devices and make real-time modifications as needed.

[0101] The terminal manages the operation of in-store equipment according to an optimized plan. In particular, it increases efficiency by operating more machinery and equipment during peak hours and reducing operation during off-peak hours. In addition, after closing time, the server improves its predictive model based on feedback data and makes even more precise predictions for the next business day.

[0102] For example, if weather changes or local events are predicted, the server reflects this in real time and immediately updates possible staffing and machinery operation plans. This dynamic adaptation maximizes the efficiency of store operations.

[0103] Examples of prompts include: "Update next week's sales forecast and provide optimal work shifts and equipment operation plans based on expected weather and local events."

[0104] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0105] Step 1:

[0106] The server collects sales information, weather information, event information, and employee work preference information using various data acquisition methods. POS systems, weather APIs, event databases, and employee management software are used as inputs. This data is stored in a cloud database and used for subsequent processing.

[0107] Step 2:

[0108] The server performs revenue forecasting based on the collected data. Machine learning algorithms using Python and TensorFlow are applied, performing numerical analysis on the input data and outputting future sales forecasts. This calculation predicts increases or decreases in sales.

[0109] Step 3:

[0110] The server optimizes work plans and machine operation plans based on revenue forecasts. An optimization algorithm using the Django framework takes forecast data as input and calculates staffing and machine operation schedules. An optimized shift schedule is then generated as output.

[0111] Step 4:

[0112] The server visualizes the optimized schedule through a user interface, allowing store managers to check it on their mobile devices. Using React Native, the output is displayed on smartphones and tablets, creating an environment where managers can easily review and modify operations using a user-friendly interface.

[0113] Step 5:

[0114] The terminal operates the machinery and equipment based on an optimized plan received from the server. Specifically, during peak hours, the terminal operates multiple devices to compensate for personnel shortages. During off-peak hours, operation is minimized to maximize energy efficiency.

[0115] Step 6:

[0116] After closing time, the server collects feedback data. This includes actual sales information and machine operation data, which are used as input. Next, the predictive model is updated based on this data, and TensorFlow is used to improve the model's accuracy. Output is then generated to further refine the next shift and machine operation plan.

[0117] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0118] This invention combines a system that optimizes staffing and machine operation in restaurants with an emotion engine that recognizes user emotions. This system consists of a server, terminals, and users, and functions in an integrated manner.

[0119] The server first uses multiple data collection methods to gather sales information, weather information, event information, and employee work preference information. Simultaneously, it obtains emotional information from users' facial expressions, voices, and behaviors through an emotion engine. This emotional information is essential data for sales forecasting and work shift optimization.

[0120] Sales forecasting methods use this data as input to predict future sales, particularly using machine learning algorithms. In this process, supplementary data on customer purchasing intent and satisfaction, including sentiment information, is considered, enabling more accurate predictions.

[0121] Next, the server optimizes work shifts and machine operation plans based on sales forecast data and sentiment-based information. For example, during times when customers are emotionally positive, the system prioritizes their feedback and either increases staffing or deploys additional machinery. As a result, efficient store operations are achieved while maintaining customer satisfaction.

[0122] The user, i.e., the store manager, is notified of the optimized shift and machine operation plan calculated from the server. Based on this, the user can make adjustments to suit the actual situation on site and determine the final work shift.

[0123] The serving robots, acting as terminals, prepare according to the operational schedule received from the server and perform efficient serving tasks at the designated time. Furthermore, they can utilize data provided by the emotion engine to grasp customer emotions in real time from facial expressions and voice, and use this information to adjust the operational schedule.

[0124] Finally, the server collects actual sales data, work performance data, and sentiment data as feedback after business hours. This allows for improvements to the sales forecasting model and operational optimization algorithms to enhance future prediction accuracy.

[0125] This system allows stores to provide efficient service while minimizing labor costs and operating costs for machinery, all while considering customer satisfaction.

[0126] The following describes the processing flow.

[0127] Step 1:

[0128] The server uses data collection tools to gather store sales information, weather information, event information, and employee work preference information. Simultaneously, an emotion engine analyzes the user's facial expressions, voice, and behavior to obtain customer emotion information.

[0129] Step 2:

[0130] The server activates a sales forecasting system, using collected data and sentiment information as input to predict future sales. In particular, it uses machine learning algorithms to generate a predictive model that reflects the influence of sentiment information on customer purchasing behavior.

[0131] Step 3:

[0132] Based on this sales forecast data, the server calculates staffing levels using work shift optimization methods. It also develops a machine operation plan to allocate additional personnel and equipment during peak customer sentiment periods.

[0133] Step 4:

[0134] The store manager, as the user, reviews the optimized work shifts and machine operation plans notified by the server. They then make adjustments as needed, based on on-site conditions and their own experience.

[0135] Step 5:

[0136] The serving robot, acting as the terminal, begins operations based on the received schedule. It monitors customer emotion data in real time, instantly adjusts its actions as needed, and performs efficient food delivery.

[0137] Step 6:

[0138] After closing time, the server collects actual sales data for the day, machine operating status, employee work performance, and sentiment information as feedback data.

[0139] Step 7:

[0140] The server analyzes feedback data and updates the sales forecasting model and work shift optimization algorithm. This improves the accuracy of future predictions and optimizations, enabling further service improvements.

[0141] (Example 2)

[0142] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0143] Restaurants need to efficiently optimize staffing and equipment usage to improve customer satisfaction while keeping operating costs down. However, conventional systems are insufficient as they rely solely on sales data and work preference information for forecasting and planning, and they cannot flexibly adjust operations based on customer sentiment. Therefore, there is a need for a system that can optimize operations by reflecting customer sentiment in real time.

[0144] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0145] In this invention, the server includes means for collecting data, means for making sales forecasts, and means for optimizing work shifts and machine operation plans. This enables more accurate optimization of operation plans not only based on sales forecasts but also by utilizing user sentiment information. Furthermore, continuous accuracy improvement is achieved by updating the prediction model using feedback data. This makes it possible to operate stores efficiently while minimizing operating costs and taking customer satisfaction into consideration.

[0146] "Data collection means" refers to devices or methods for acquiring sales information, weather information, event information, work preference information, and sentiment data.

[0147] A "sales forecasting method" refers to an algorithm or calculation technique used to predict future sales based on collected data.

[0148] A "work shift optimization method" is a system that optimizes employee working hours based on sales forecasts and sentiment information to most efficiently allocate employee working hours.

[0149] "Machinery and equipment operation plan optimization means" refers to a method or apparatus for optimizing the operation schedule of machinery and equipment based on sales forecasts and sentiment information.

[0150] "Feedback data" refers to data including actual sales information, machine operation status, and acquired sentiment information, which is used to update predictive models.

[0151] "Means for acquiring emotional information" refers to a device or method for analyzing a user's facial expressions and voice to identify their emotions.

[0152] An "emotion-based adjustment mechanism" is a system that dynamically adjusts machinery and personnel allocation based on emotional information acquired in real time.

[0153] Modes for carrying out the invention

[0154] This invention is a system for achieving efficient staffing and operation of machinery in restaurants. Specifically, it optimizes work shifts and machinery based on sales forecasts and sentiment data to improve the operational efficiency of the restaurant.

[0155] The server is responsible for collecting data. The hardware used includes a database server, and the software is used to retrieve data using SQL or APIs. Sales information, weather information, event information, work preference information, and sentiment data are collected.

[0156] The server uses the collected data to perform sales forecasts. Here, machine learning models are utilized, employing libraries such as TensorFlow and PyTorch. By also incorporating sentiment information, highly accurate sales forecasts are achieved.

[0157] Subsequently, the server generates optimal work shifts and machinery operation plans based on the prediction results. This uses optimization methods based on linear programming and genetic algorithms to achieve efficient operation.

[0158] The store manager, as the user, receives optimized information provided by the server and makes adjustments according to the actual situation on site. This enables flexible store operations.

[0159] The food delivery robot, which is part of the terminal, receives operational schedules from the server and performs specific tasks. Furthermore, it analyzes customers' facial expressions and voices in real time using an emotion engine and adjusts operations as needed. Another example is the improvement of food delivery services, which directly leads to increased customer satisfaction.

[0160] As a concrete example, the server predicts an increase in customer traffic on specific event days or under certain weather conditions, and plans to increase the operation of serving robots accordingly. Based on this, the user can review staffing during peak hours and operate more efficiently.

[0161] Example of a prompt:

[0162] "Please provide a plan to optimize staffing and machinery operations based on next day's sales forecast and sentiment data."

[0163] This system allows stores to maintain high customer satisfaction while keeping operating costs down.

[0164] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0165] Step 1:

[0166] The server collects sales information, weather information, event information, work preference information, and user sentiment data from the database. Input data is retrieved in digital format, and queries are executed against the database using SQL. This process prepares all the data necessary for prediction and optimization as initial input data. The output includes this prepared data set.

[0167] Step 2:

[0168] The server uses a machine learning model to predict sales based on the collected data. Inputs include sales data, weather conditions, event information, and sentiment data. The algorithm utilizes a prediction model based on TensorFlow. The output generates future sales forecasts. This process involves data analysis and numerical prediction.

[0169] Step 3:

[0170] The server considers sales forecasts and sentiment data to create optimal work shifts and machine operation plans. Inputs include sales forecasts and sentiment information. A linear programming-based optimization algorithm is used to output shifts and operation schedules that are most efficient and cost-effective. Specifically, it creates staffing schedules and robot operation schedules.

[0171] Step 4:

[0172] The server notifies the store manager, who is the user, of the optimized operational plan. The input is the optimized plan, and the output is notification data to the manager. This process involves sending information through email applications and management dashboards. Based on this notification, the manager develops a feasible, final store operational plan.

[0173] Step 5:

[0174] The serving robot, acting as the terminal, prepares according to the operational plan from the server. Its input includes an optimized operational schedule, and its output provides a specific serving operation plan for each time slot. During this process, the robot acquires customer emotion data in real time and dynamically adjusts the serving schedule as needed.

[0175] Step 6:

[0176] The server collects actual sales data, work performance data, and sentiment data obtained after business hours, and uses this as feedback for future model improvements. The input is daily business results data, and the output is feedback information that helps improve the predictive model. This process ensures continuous improvement of the model's accuracy.

[0177] (Application Example 2)

[0178] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0179] In the food and beverage industry, there is a demand for services that respond immediately to individual needs in order to increase customer satisfaction. At the same time, it is necessary to achieve efficient operations while minimizing labor costs and the operating costs of machinery and equipment. However, with conventional methods, it has been difficult to recognize customer emotions in real time and dynamically adjust services based on them. As a result, there has been a challenge in being able to plan appropriate staffing and machinery operations that take emotions into consideration.

[0180] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0181] In this invention, the server includes means for collecting data, means for making sales forecasts based on the data, and means for optimizing work schedules and machine operation plans based on the sales forecasts. This makes it possible to recognize and analyze customer sentiment information in real time and optimize personnel allocation and machine operation accordingly.

[0182] "Means of data collection" refers to a system for acquiring and integrating store sales information, weather information, event information, work preference information, and customer facial and voice information.

[0183] A "means of sales forecasting" refers to a device that uses machine learning algorithms to predict future sales based on collected data.

[0184] "Means for optimizing work schedules and machinery operation plans" refers to a system for planning and adjusting efficient staffing and machinery operation, taking into account predicted sales information and customer sentiment data.

[0185] "Methods for updating predictive models using feedback data" refers to the process of automatically improving a model to enhance the accuracy of sales forecasts by using actual sales data and customer sentiment data.

[0186] "Means for recognizing and analyzing customer emotions through an emotion engine" refers to a device that includes an algorithm for detecting and classifying a customer's emotional state by analyzing image and audio data.

[0187] "A means of adjusting service delivery in real time" refers to a system that dynamically instructs employees to optimize service content based on emotional data acquired from customers.

[0188] "A means of improving service by presenting emotion analysis results to employees using smart devices" refers to a technology that provides information to employees through devices such as smart glasses, enabling them to check the emotional state of customers in real time and respond appropriately.

[0189] This invention is a system for optimizing staffing and the operation of machinery and equipment in restaurants.

[0190] This system consists of a server, smart devices (e.g., smart glasses), and terminals placed within the store. The server is the central hub responsible for collecting data, forecasting sales, optimizing work schedules, and planning the operation of machinery and equipment.

[0191] The server uses multiple data collection methods to gather sales information, weather information, event information, and employee work preference information. It also uses an emotion engine to obtain emotional information from customer facial expressions and voice. Python and the OpenCV library are used for image recognition, and Google's Cloud Speech-to-Text API is used for voice analysis. Based on this data, a machine learning algorithm is used to predict sales.

[0192] The server then uses predicted sales data and sentiment information to create an optimal work schedule and equipment operation plan. It can adjust employee deployment and machine operation in real time based on the customer's emotional state. For example, employees can monitor customer emotions in real time through smart glasses and modify service as needed.

[0193] Furthermore, after business hours, the server collects actual sales data, the results of the tasks performed, and customer sentiment data, and provides feedback to build increasingly accurate sales forecasting models and operational optimization algorithms.

[0194] As a concrete example, consider a scenario where smart glasses are used to monitor the emotional state of customers visiting a restaurant with their family. If a child shows signs of boredom, a special menu item is suggested to the employee. This allows for concrete implementation of how smart devices are used and how information is processed.

[0195] Examples of prompt statements for a generative AI model are as follows:

[0196] "Please tell me about a system that uses video data captured by smart glasses to determine customer emotions in real time through image recognition and voice analysis, and then notifies employees of the appropriate service."

[0197] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0198] Step 1:

[0199] The server collects sales information, weather information, event information, work preference information, and customer facial expressions and voice data as input. This data is gathered from in-store and external databases using sensors, cameras, microphones, etc. This allows for the integrated accumulation of environmental and customer information.

[0200] Step 2:

[0201] The server uses the collected information as input to perform image recognition and speech analysis. For image recognition, it analyzes customer facial expressions using Python and the OpenCV library, and for speech analysis, it uses the Google Cloud Speech-to-Text API. The data is processed by a machine learning model to identify the customer's emotions. As a result, the customer's emotional state is output.

[0202] Step 3:

[0203] The server makes sales forecasts based on customer emotional states and other environmental data. A machine learning algorithm drives the process of predicting future sales. This predicted data is then used as the result for the next step in the process.

[0204] Step 4:

[0205] The server uses sales forecast results and sentiment data as input to optimize work schedules and machine operation plans. An optimization algorithm is used to output staffing plans that are adapted to each customer's needs and sentiments.

[0206] Step 5:

[0207] The smart device (smart glasses) provides employees with real-time sentiment analysis results and optimized service guidance. Through the device, employees can receive immediate feedback, which they can then use to improve their service delivery.

[0208] Step 6:

[0209] Based on feedback from the server, users will ultimately adjust their work schedules and service content. This includes implementing specific service improvements tailored to anticipated situations.

[0210] Step 7:

[0211] The server collects feedback data obtained after business hours and updates its predictive models and optimization algorithms. This improves the overall accuracy and efficiency of the system.

[0212] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0213] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0214] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0215] [Second Embodiment]

[0216] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0217] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0218] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0219] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0220] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0221] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0222] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0223] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0224] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0225] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0226] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0227] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0228] This invention provides a system for optimizing staff work shifts and machinery operation plans in restaurants. The system consists of server, terminal, and user elements, each playing a specific role.

[0229] The server first uses data collection tools to acquire store sales information, weather information, event information, and employee work preference information. This data is processed by sales forecasting tools, which in particular use machine learning algorithms to predict future sales. Based on this forecast information, the server uses work shift optimization tools to calculate the optimal staffing arrangement and further develops a machine operation plan.

[0230] Store managers, as users, can review the optimized shifts and operational plans provided by the server and manually adjust them as needed. This allows users to implement operations that reflect the needs of the store staff.

[0231] Terminals, or the machines operating within the store, prepare themselves based on the operational schedule received from the server and then perform food serving tasks. As a concrete example, multiple terminals can be operated during peak daytime hours to compensate for labor shortages, while only the minimum necessary terminals can be operated during off-peak nighttime hours, thereby achieving efficient operation.

[0232] Furthermore, the server collects feedback data after business hours and analyzes predictions and actual results. This allows for the improvement of the sales forecast model using forecast model update mechanisms, resulting in more accurate plans for future shifts and machine operation.

[0233] As described above, the present invention provides an integrated solution for improving the operational efficiency of stores, simultaneously achieving reductions in labor costs and improvements in service quality.

[0234] The following describes the processing flow.

[0235] Step 1:

[0236] The server uses data collection methods to gather historical sales information, weather forecast data, store event information, and employee work preference data. This includes using various APIs and accessing databases.

[0237] Step 2:

[0238] The server activates the sales forecasting system and applies machine learning algorithms based on the collected data to predict future sales. The sales forecasting model is created based on historical data and updated as needed.

[0239] Step 3:

[0240] The server uses a work shift optimization method based on predicted sales data to calculate the optimal staffing. At the same time, it formulates a machine operation plan and determines the number of machines and their operating hours to be used in each time slot.

[0241] Step 4:

[0242] The store manager, as the user, is notified of the optimized work shifts and machine operation plans calculated by the server. The user can review this information and adjust the shifts and plans as needed.

[0243] Step 5:

[0244] The terminal begins preparations based on the operational schedule received from the server. Following the specified schedule, it starts serving meals and efficiently delivers them according to a pre-configured workflow.

[0245] Step 6:

[0246] After business hours, the server aggregates feedback data. This includes sales data for the day, machine operation status, and employee work performance.

[0247] Step 7:

[0248] The server updates its sales forecasting model and shift optimization algorithm based on feedback data. This improves the model so that future predictions and optimizations are more accurate.

[0249] (Example 1)

[0250] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0251] In store operations such as restaurants and retail, optimal staffing and efficient use of machinery are required. However, current methods make accurate sales forecasting difficult, leading to excessive labor costs and a decline in service quality. Furthermore, planning that takes into account external factors such as fluctuating weather and events is insufficient, thus requiring practical and flexible operational methods.

[0252] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0253] In this invention, the server includes means for collecting commercial transaction information, weather information, event information, and employee request information via a data processing device; means for predicting future sales using a machine learning algorithm based on the collected information; and means for optimizing labor allocation and machinery operation plans based on the predictions. This enables flexible and efficient store operations and staffing in response to fluctuating demand.

[0254] A "data processing device" is an information processing system used to collect and analyze commercial transaction information, weather information, event information, and employee request information.

[0255] "Commercial transaction information" refers to data on sales and transactions at stores, and is used for business analysis and sales forecasting.

[0256] "Weather information" refers to data on meteorological conditions and is used for analysis as a factor that influences sales and customer numbers.

[0257] "Event information" refers to information about events and activities held in the local area, which contributes to predicting customer traffic and managing resources for stores.

[0258] "Employee request information" refers to information regarding staff work preferences and shifts, and serves as basic data for optimizing labor allocation.

[0259] A "machine learning algorithm" refers to a computational method used to find patterns in input data and make predictions or classifications about the future.

[0260] "Labor allocation" refers to the planning of employee work shifts and assigned tasks in store operations.

[0261] "Machinery and equipment" is a general term for automated devices and systems used in stores, and specifically includes serving robots, vending machines, etc.

[0262] An "operational plan" refers to a plan for efficiently carrying out operations through the appropriate allocation and management of resources.

[0263] This invention is a system for optimizing labor allocation and machinery operation planning in store operations. The system consists of server, terminal, and user elements.

[0264] The server collects information via data processing devices. Specifically, commercial transaction information is obtained from the store's sales information management system, and weather information is obtained using a weather data provision API. Event information is obtained through an event data service, and employee request information is obtained from the labor management system via an API.

[0265] The server uses this data to perform sales forecasts using software such as TensorFlow or PyTorch to execute machine learning algorithms. Based on the predicted sales data, the server uses a linear programming algorithm to create an optimal work shift and machinery operation plan.

[0266] Store managers, as users, can view these plans provided by the server on their management terminals and make adjustments as needed through the interface. These adjustments are intuitive and can be performed using a GUI, making it easy for users without special technical knowledge to operate.

[0267] Terminals, or in-store devices, automatically prepare and begin operations according to the operational schedule received from the server. A specific example is that multiple serving robots operate during peak hours, while only the minimum necessary number operate during off-peak hours. This enables efficient resource management.

[0268] Furthermore, after the end of the workday, the server accumulates feedback data on the execution results and updates the predictive model based on this data. Through this process, the accuracy of the predictions improves over time, which is useful for formulating future plans.

[0269] As a concrete example, it is possible to optimize the staffing and machine operation plans for a specific store on days when sunny weather is forecast for the weekend. An example of a prompt message could be, "Please provide an optimal staffing and serving robot deployment plan based on the predicted number of customers on a sunny Sunday next week." Based on such input, the system will propose the optimal plan.

[0270] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0271] Step 1:

[0272] The server collects commercial transaction information, weather information, event information, and employee request information via a data processing device. Inputs include data from POS systems, weather APIs, event information services, and labor management systems. The output is an integrated dataset, ready for subsequent processing such as analysis and prediction. Specifically, it uses an authentication key to access data from APIs and stores the obtained information in a database.

[0273] Step 2:

[0274] The server executes a machine learning algorithm using the collected data. The input here is the integrated dataset obtained in Step 1. Data processing involves encoding categorical data and imputing missing values ​​to format the data into an analyzable format. The output is future sales forecast data. In this process, TensorFlow is used to apply a trained model and generate predicted values. Specifically, the server performs CSV file validation as data preprocessing and handles outliers as needed.

[0275] Step 3:

[0276] The server optimizes labor allocation and machinery operation plans based on predicted sales data. The input is the sales forecast data obtained in step 2. The data calculations include an optimization algorithm using linear programming. The output is a specific and optimized shift allocation and operation schedule. In this process, shifts are adjusted to comply with employee labor laws and meet their desired working hours. Specifically, it performs resource allocation simulations based on a numerical model, distinguishing between peak and off-peak periods.

[0277] Step 4:

[0278] The store manager, as the user, reviews the provided work shifts and operational plans and makes adjustments as needed. The input is the plan data from the server, and the output is the adjusted final plan. Specifically, the shift schedule can be viewed on the management screen, and working hours can be edited using drag-and-drop operations via the GUI. These adjustments are made to reflect on-site situations, such as leave requests from specific staff members.

[0279] Step 5:

[0280] The terminal starts the operation of the mechanical device according to the operation schedule received from the server. The input is the final operation plan in Step 4, and the output is the operation performance data of the mechanical device. As a specific operation, the terminal automatically turns on the power of the food delivery robot and starts the initialization process. This enables efficient task execution within the set time.

[0281] Step 6:

[0282] After the operation ends, the server collects the actual sales data and the operation data of the mechanical device as feedback. The input is the actual sales and operation performance data obtained at the store, and the output is the improved prediction algorithm and real-time data for the next operation plan. The specific operation includes a process of detecting outliers and deviations from the past through late-night database updates and making corrections to the next prediction model.

[0283] (Application Example 1)

[0284] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0285] In a physical store, it is required to efficiently optimize the staff's work schedule and the operation plan of the mechanical device and enable real-time adjustment. However, the conventional system had problems such as insufficient accuracy in revenue prediction and difficulty in modifying the plan. This led to a decrease in the operation efficiency of the physical store and caused extra labor and costs.

[0286] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0287] In this invention, the server includes means for acquiring data, means for performing revenue forecasts based on the data, means for optimizing work plans and machine operation plans based on the revenue forecasts, means for visualizing and making modifiable the optimized work plans and machine operation plans, means for improving the forecast model using feedback data, and means for viewing and modifying the work plans in real time via a mobile terminal. This enables efficient optimization and flexible modification of plans in physical stores.

[0288] "Means of acquiring data" refers to elements that have the function of collecting various types of information necessary for use in revenue forecasting.

[0289] "Means for performing revenue forecasting" refers to elements that perform calculations to predict future revenue based on collected data.

[0290] "Means for optimizing work plans and machine operation plans" refer to elements that efficiently set personnel allocations and machine operation based on projected revenue.

[0291] "Means of visualization and modification" refer to elements that visually display the optimized plan and allow users to easily make changes.

[0292] "Methods for improving predictive models using feedback data" refer to elements that involve aggregating actual operational results and updating predictive algorithms to improve the accuracy of revenue forecasts.

[0293] "Means of viewing and modifying in real time via mobile devices" refers to elements that enable users to check plans in real time using smartphones or tablets and make adjustments as needed.

[0294] The server provides an integrated system to improve the operational efficiency of the stores. First, it collects store sales information, weather information, event information, and part-time staff work preference information through data acquisition methods. This data is stored in a cloud-hosted database and processed by a sales forecasting system. This processing utilizes machine learning algorithms and the TensorFlow library using Python to predict future sales with high accuracy.

[0295] The server has the means to optimize work plans and machine operation plans based on the acquired predictive information. The Django framework is used for communication between the server and the frontend, supporting data movement within the system. Once the optimization process is complete, the plan is visualized through a user interface developed with React Native, allowing store managers to review it via their mobile devices and make real-time modifications as needed.

[0296] The terminal manages the operation of in-store equipment according to an optimized plan. In particular, it increases efficiency by operating more machinery and equipment during peak hours and reducing operation during off-peak hours. In addition, after closing time, the server improves its predictive model based on feedback data and makes even more precise predictions for the next business day.

[0297] For example, if weather changes or local events are predicted, the server reflects this in real time and immediately updates possible staffing and machinery operation plans. This dynamic adaptation maximizes the efficiency of store operations.

[0298] Examples of prompts include: "Update next week's sales forecast and provide optimal work shifts and equipment operation plans based on expected weather and local events."

[0299] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0300] Step 1:

[0301] The server collects sales information, weather information, event information, and employees' work preference information by using means for obtaining data. The POS system, weather API, event database, and employee management software are used as inputs. These data are stored in a cloud database and connected for subsequent processing.

[0302] Step 2:

[0303] The server performs a revenue prediction based on the collected data. A machine learning algorithm is applied using Python and TensorFlow, numerical analysis is performed on the input data, and a future sales prediction is output. By this calculation, the increase or decrease in sales is predicted.

[0304] Step 3:

[0305] The server optimizes the work plan and the machine device operation plan based on the results of the revenue prediction. The optimization algorithm operates using the Django framework, receives prediction data as input, calculates the personnel allocation and the operation schedule of the machine devices. Then, an optimized shift schedule is generated as output.

[0306] Step 4:

[0307] The server visualizes the optimized schedule through the user interface so that the store manager, who is the user, can check it on the mobile terminal. Using React Native, it outputs on smartphones and tablets, and prepares an environment where the manager can check and modify operations with an interface that is easy to work with.

[0308] Step 5:

[0309] The terminal operates the machinery and equipment based on an optimized plan received from the server. Specifically, during peak hours, the terminal operates multiple devices to compensate for personnel shortages. During off-peak hours, operation is minimized to maximize energy efficiency.

[0310] Step 6:

[0311] After closing time, the server collects feedback data. This includes actual sales information and machine operation data, which are used as input. Next, the predictive model is updated based on this data, and TensorFlow is used to improve the model's accuracy. Output is then generated to further refine the next shift and machine operation plan.

[0312] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0313] This invention combines a system that optimizes staffing and machine operation in restaurants with an emotion engine that recognizes user emotions. This system consists of a server, terminals, and users, and functions in an integrated manner.

[0314] The server first uses multiple data collection methods to gather sales information, weather information, event information, and employee work preference information. Simultaneously, it obtains emotional information from users' facial expressions, voices, and behaviors through an emotion engine. This emotional information is essential data for sales forecasting and work shift optimization.

[0315] Sales forecasting methods use this data as input to predict future sales, particularly using machine learning algorithms. In this process, supplementary data on customer purchasing intent and satisfaction, including sentiment information, is considered, enabling more accurate predictions.

[0316] Next, the server optimizes work shifts and machine operation plans based on sales forecast data and sentiment-based information. For example, during times when customers are emotionally positive, the system prioritizes their feedback and either increases staffing or deploys additional machinery. As a result, efficient store operations are achieved while maintaining customer satisfaction.

[0317] The user, i.e., the store manager, is notified of the optimized shift and machine operation plan calculated from the server. Based on this, the user can make adjustments to suit the actual situation on site and determine the final work shift.

[0318] The serving robots, acting as terminals, prepare according to the operational schedule received from the server and perform efficient serving tasks at the designated time. Furthermore, they can utilize data provided by the emotion engine to grasp customer emotions in real time from facial expressions and voice, and use this information to adjust the operational schedule.

[0319] Finally, the server collects actual sales data, work performance data, and sentiment data as feedback after business hours. This allows for improvements to the sales forecasting model and operational optimization algorithms to enhance future prediction accuracy.

[0320] This system allows stores to provide efficient service while minimizing labor costs and operating costs for machinery, all while considering customer satisfaction.

[0321] The following describes the processing flow.

[0322] Step 1:

[0323] The server uses data collection tools to gather store sales information, weather information, event information, and employee work preference information. Simultaneously, an emotion engine analyzes the user's facial expressions, voice, and behavior to obtain customer emotion information.

[0324] Step 2:

[0325] The server activates a sales forecasting system, using collected data and sentiment information as input to predict future sales. In particular, it uses machine learning algorithms to generate a predictive model that reflects the influence of sentiment information on customer purchasing behavior.

[0326] Step 3:

[0327] Based on this sales forecast data, the server calculates staffing levels using work shift optimization methods. It also develops a machine operation plan to allocate additional personnel and equipment during peak customer sentiment periods.

[0328] Step 4:

[0329] The store manager, as the user, reviews the optimized work shifts and machine operation plans notified by the server. They then make adjustments as needed, based on on-site conditions and their own experience.

[0330] Step 5:

[0331] The serving robot, acting as the terminal, begins operations based on the received schedule. It monitors customer emotion data in real time, instantly adjusts its actions as needed, and performs efficient food delivery.

[0332] Step 6:

[0333] After closing time, the server collects actual sales data for the day, machine operating status, employee work performance, and sentiment information as feedback data.

[0334] Step 7:

[0335] The server analyzes feedback data and updates the sales forecasting model and work shift optimization algorithm. This improves the accuracy of future predictions and optimizations, enabling further service improvements.

[0336] (Example 2)

[0337] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0338] Restaurants need to efficiently optimize staffing and equipment usage to improve customer satisfaction while keeping operating costs down. However, conventional systems are insufficient as they rely solely on sales data and work preference information for forecasting and planning, and they cannot flexibly adjust operations based on customer sentiment. Therefore, there is a need for a system that can optimize operations by reflecting customer sentiment in real time.

[0339] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0340] In this invention, the server includes means for collecting data, means for making sales forecasts, and means for optimizing work shifts and machine operation plans. This enables more accurate optimization of operation plans not only based on sales forecasts but also by utilizing user sentiment information. Furthermore, continuous accuracy improvement is achieved by updating the prediction model using feedback data. This makes it possible to operate stores efficiently while minimizing operating costs and taking customer satisfaction into consideration.

[0341] "Data collection means" refers to devices or methods for acquiring sales information, weather information, event information, work preference information, and sentiment data.

[0342] A "sales forecasting method" refers to an algorithm or calculation technique used to predict future sales based on collected data.

[0343] A "work shift optimization method" is a system that optimizes employee working hours based on sales forecasts and sentiment information to most efficiently allocate employee working hours.

[0344] "Machinery and equipment operation plan optimization means" refers to a method or apparatus for optimizing the operation schedule of machinery and equipment based on sales forecasts and sentiment information.

[0345] "Feedback data" refers to data including actual sales information, machine operation status, and acquired sentiment information, which is used to update predictive models.

[0346] "Means for acquiring emotional information" refers to a device or method for analyzing a user's facial expressions and voice to identify their emotions.

[0347] An "emotion-based adjustment mechanism" is a system that dynamically adjusts machinery and personnel allocation based on emotional information acquired in real time.

[0348] Modes for carrying out the invention

[0349] This invention is a system for achieving efficient staffing and operation of machinery in restaurants. Specifically, it optimizes work shifts and machinery based on sales forecasts and sentiment data to improve the operational efficiency of the restaurant.

[0350] The server is responsible for collecting data. The hardware used includes a database server, and the software is used to retrieve data using SQL or APIs. Sales information, weather information, event information, work preference information, and sentiment data are collected.

[0351] The server uses the collected data to perform sales forecasts. Here, machine learning models are utilized, employing libraries such as TensorFlow and PyTorch. By also incorporating sentiment information, highly accurate sales forecasts are achieved.

[0352] Subsequently, the server generates optimal work shifts and machinery operation plans based on the prediction results. This uses optimization methods based on linear programming and genetic algorithms to achieve efficient operation.

[0353] The store manager, as the user, receives optimized information provided by the server and makes adjustments according to the actual situation on site. This enables flexible store operations.

[0354] The food delivery robot, which is part of the terminal, receives operational schedules from the server and performs specific tasks. Furthermore, it analyzes customers' facial expressions and voices in real time using an emotion engine and adjusts operations as needed. Another example is the improvement of food delivery services, which directly leads to increased customer satisfaction.

[0355] As a concrete example, the server predicts an increase in customer traffic on specific event days or under certain weather conditions, and plans to increase the operation of serving robots accordingly. Based on this, the user can review staffing during peak hours and operate more efficiently.

[0356] Example of a prompt:

[0357] "Please provide a plan to optimize staffing and machinery operations based on next day's sales forecast and sentiment data."

[0358] This system allows stores to maintain high customer satisfaction while keeping operating costs down.

[0359] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0360] Step 1:

[0361] The server collects sales information, weather information, event information, work preference information, and user sentiment data from the database. Input data is retrieved in digital format, and queries are executed against the database using SQL. This process prepares all the data necessary for prediction and optimization as initial input data. The output includes this prepared data set.

[0362] Step 2:

[0363] The server uses a machine learning model to predict sales based on the collected data. Inputs include sales data, weather conditions, event information, and sentiment data. The algorithm utilizes a prediction model based on TensorFlow. The output generates future sales forecasts. This process involves data analysis and numerical prediction.

[0364] Step 3:

[0365] The server considers sales forecasts and sentiment data to create optimal work shifts and machine operation plans. Inputs include sales forecasts and sentiment information. A linear programming-based optimization algorithm is used to output shifts and operation schedules that are most efficient and cost-effective. Specifically, it creates staffing schedules and robot operation schedules.

[0366] Step 4:

[0367] The server notifies the store manager, who is the user, of the optimized operational plan. The input is the optimized plan, and the output is notification data to the manager. This process involves sending information through email applications and management dashboards. Based on this notification, the manager develops a feasible, final store operational plan.

[0368] Step 5:

[0369] The serving robot, acting as the terminal, prepares according to the operational plan from the server. Its input includes an optimized operational schedule, and its output provides a specific serving operation plan for each time slot. During this process, the robot acquires customer emotion data in real time and dynamically adjusts the serving schedule as needed.

[0370] Step 6:

[0371] The server collects actual sales data, work performance data, and sentiment data obtained after business hours, and uses this as feedback for future model improvements. The input is daily business results data, and the output is feedback information that helps improve the predictive model. This process ensures continuous improvement of the model's accuracy.

[0372] (Application Example 2)

[0373] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0374] In the food and beverage industry, there is a demand for services that respond immediately to individual needs in order to increase customer satisfaction. At the same time, it is necessary to achieve efficient operations while minimizing labor costs and the operating costs of machinery and equipment. However, with conventional methods, it has been difficult to recognize customer emotions in real time and dynamically adjust services based on them. As a result, there has been a challenge in being able to plan appropriate staffing and machinery operations that take emotions into consideration.

[0375] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0376] In this invention, the server includes means for collecting data, means for making sales forecasts based on the data, and means for optimizing work schedules and machine operation plans based on the sales forecasts. This makes it possible to recognize and analyze customer sentiment information in real time and optimize personnel allocation and machine operation accordingly.

[0377] "Means of data collection" refers to a system for acquiring and integrating store sales information, weather information, event information, work preference information, and customer facial and voice information.

[0378] A "means of sales forecasting" refers to a device that uses machine learning algorithms to predict future sales based on collected data.

[0379] "Means for optimizing work schedules and machinery operation plans" refers to a system for planning and adjusting efficient staffing and machinery operation, taking into account predicted sales information and customer sentiment data.

[0380] "Methods for updating predictive models using feedback data" refers to the process of automatically improving a model to enhance the accuracy of sales forecasts by using actual sales data and customer sentiment data.

[0381] "Means for recognizing and analyzing customer emotions through an emotion engine" refers to a device that includes an algorithm for detecting and classifying a customer's emotional state by analyzing image and audio data.

[0382] "A means of adjusting service delivery in real time" refers to a system that dynamically instructs employees to optimize service content based on emotional data acquired from customers.

[0383] "A means of improving service by presenting emotion analysis results to employees using smart devices" refers to a technology that provides information to employees through devices such as smart glasses, enabling them to check the emotional state of customers in real time and respond appropriately.

[0384] This invention is a system for optimizing staffing and the operation of machinery and equipment in restaurants.

[0385] This system consists of a server, smart devices (e.g., smart glasses), and terminals placed within the store. The server is the central hub responsible for collecting data, forecasting sales, optimizing work schedules, and planning the operation of machinery and equipment.

[0386] The server uses multiple data collection methods to gather sales information, weather information, event information, and employee work preference information. It also uses an emotion engine to obtain emotional information from customer facial expressions and voice. Python and OpenCV libraries are used for image recognition, and the Google Cloud Speech-to-Text API is used for voice analysis. Based on this data, a machine learning algorithm is used to predict sales.

[0387] The server then uses predicted sales data and sentiment information to create an optimal work schedule and equipment operation plan. It can adjust employee deployment and machine operation in real time based on the customer's emotional state. For example, employees can monitor customer emotions in real time through smart glasses and modify service as needed.

[0388] Furthermore, after business hours, the server collects actual sales data, the results of the tasks performed, and customer sentiment data, and provides feedback to build increasingly accurate sales forecasting models and operational optimization algorithms.

[0389] As a concrete example, consider a scenario where smart glasses are used to monitor the emotional state of customers visiting a restaurant with their family. If a child shows signs of boredom, a special menu item is suggested to the employee. This allows for concrete implementation of how smart devices are used and how information is processed.

[0390] Examples of prompt statements for a generative AI model are as follows:

[0391] "Please tell me about a system that uses video data captured by smart glasses to determine customer emotions in real time through image recognition and voice analysis, and then notifies employees of the appropriate service."

[0392] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0393] Step 1:

[0394] The server collects sales information, weather information, event information, work preference information, and customer facial expressions and voice data as input. This data is gathered from in-store and external databases using sensors, cameras, microphones, etc. This allows for the integrated accumulation of environmental and customer information.

[0395] Step 2:

[0396] The server uses the collected information as input to perform image recognition and speech analysis. For image recognition, it analyzes customer facial expressions using Python and the OpenCV library, and for speech analysis, it uses the Google Cloud Speech-to-Text API. The data is processed by a machine learning model to identify the customer's emotions. As a result, the customer's emotional state is output.

[0397] Step 3:

[0398] The server makes sales forecasts based on customer emotional states and other environmental data. A machine learning algorithm drives the process of predicting future sales. This predicted data is then used as the result for the next step in the process.

[0399] Step 4:

[0400] The server uses sales forecast results and sentiment data as input to optimize work schedules and machine operation plans. An optimization algorithm is used to output staffing plans that are adapted to each customer's needs and sentiments.

[0401] Step 5:

[0402] The smart device (smart glasses) provides employees with real-time sentiment analysis results and optimized service guidance. Through the device, employees can receive immediate feedback, which they can then use to improve their service delivery.

[0403] Step 6:

[0404] Based on feedback from the server, users will ultimately adjust their work schedules and service content. This includes implementing specific service improvements tailored to anticipated situations.

[0405] Step 7:

[0406] The server collects feedback data obtained after business hours and updates its predictive models and optimization algorithms. This improves the overall accuracy and efficiency of the system.

[0407] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0408] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0409] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0410] [Third Embodiment]

[0411] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0412] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0413] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0414] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0415] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0416] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0417] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0418] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0419] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0420] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0421] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0422] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0423] This invention provides a system for optimizing staff work shifts and machinery operation plans in restaurants. The system consists of server, terminal, and user elements, each playing a specific role.

[0424] The server first uses data collection tools to acquire store sales information, weather information, event information, and employee work preference information. This data is processed by sales forecasting tools, which in particular use machine learning algorithms to predict future sales. Based on this forecast information, the server uses work shift optimization tools to calculate the optimal staffing arrangement and further develops a machine operation plan.

[0425] Store managers, as users, can review the optimized shifts and operational plans provided by the server and manually adjust them as needed. This allows users to implement operations that reflect the needs of the store staff.

[0426] Terminals, or the machines operating within the store, prepare themselves based on the operational schedule received from the server and then perform food serving tasks. As a concrete example, multiple terminals can be operated during peak daytime hours to compensate for labor shortages, while only the minimum necessary terminals can be operated during off-peak nighttime hours, thereby achieving efficient operation.

[0427] Furthermore, the server collects feedback data after business hours and analyzes predictions and actual results. This allows for the improvement of the sales forecast model using forecast model update mechanisms, resulting in more accurate plans for future shifts and machine operation.

[0428] As described above, the present invention provides an integrated solution for improving the operational efficiency of stores, simultaneously achieving reductions in labor costs and improvements in service quality.

[0429] The following describes the processing flow.

[0430] Step 1:

[0431] The server uses data collection methods to gather historical sales information, weather forecast data, store event information, and employee work preference data. This includes using various APIs and accessing databases.

[0432] Step 2:

[0433] The server activates the sales forecasting system and applies machine learning algorithms based on the collected data to predict future sales. The sales forecasting model is created based on historical data and updated as needed.

[0434] Step 3:

[0435] The server uses a work shift optimization method based on predicted sales data to calculate the optimal staffing. At the same time, it formulates a machine operation plan and determines the number of machines and their operating hours to be used in each time slot.

[0436] Step 4:

[0437] The store manager, as the user, is notified of the optimized work shifts and machine operation plans calculated by the server. The user can review this information and adjust the shifts and plans as needed.

[0438] Step 5:

[0439] The terminal begins preparations based on the operational schedule received from the server. Following the specified schedule, it starts serving meals and efficiently delivers them according to a pre-configured workflow.

[0440] Step 6:

[0441] After business hours, the server aggregates feedback data. This includes sales data for the day, machine operation status, and employee work performance.

[0442] Step 7:

[0443] The server updates its sales forecasting model and shift optimization algorithm based on feedback data. This improves the model so that future predictions and optimizations are more accurate.

[0444] (Example 1)

[0445] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0446] In store operations such as restaurants and retail, optimal staffing and efficient use of machinery are required. However, current methods make accurate sales forecasting difficult, leading to excessive labor costs and a decline in service quality. Furthermore, planning that takes into account external factors such as fluctuating weather and events is insufficient, thus requiring practical and flexible operational methods.

[0447] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0448] In this invention, the server includes means for collecting commercial transaction information, weather information, event information, and employee request information via a data processing device; means for predicting future sales using a machine learning algorithm based on the collected information; and means for optimizing labor allocation and machinery operation plans based on the predictions. This enables flexible and efficient store operations and staffing in response to fluctuating demand.

[0449] A "data processing device" is an information processing system used to collect and analyze commercial transaction information, weather information, event information, and employee request information.

[0450] "Commercial transaction information" refers to data on sales and transactions at stores, and is used for business analysis and sales forecasting.

[0451] "Weather information" refers to data on meteorological conditions and is used for analysis as a factor that influences sales and customer numbers.

[0452] "Event information" refers to information about events and activities held in the local area, which contributes to predicting customer traffic and managing resources for stores.

[0453] "Employee request information" refers to information regarding staff work preferences and shifts, and serves as basic data for optimizing labor allocation.

[0454] A "machine learning algorithm" refers to a computational method used to find patterns in input data and make predictions or classifications about the future.

[0455] "Labor allocation" refers to the planning of employee work shifts and assigned tasks in store operations.

[0456] "Machinery and equipment" is a general term for automated devices and systems used in stores, and specifically includes serving robots, vending machines, etc.

[0457] An "operational plan" refers to a plan for efficiently carrying out operations through the appropriate allocation and management of resources.

[0458] This invention is a system for optimizing labor allocation and machinery operation planning in store operations. The system consists of server, terminal, and user elements.

[0459] The server collects information via data processing devices. Specifically, commercial transaction information is obtained from the store's sales information management system, and weather information is obtained using a weather data provision API. Event information is obtained through an event data service, and employee request information is obtained from the labor management system via an API.

[0460] The server uses this data to perform sales forecasts using software such as TensorFlow or PyTorch to execute machine learning algorithms. Based on the predicted sales data, the server uses a linear programming algorithm to create an optimal work shift and machinery operation plan.

[0461] Store managers, as users, can view these plans provided by the server on their management terminals and make adjustments as needed through the interface. These adjustments are intuitive and can be performed using a GUI, making it easy for users without special technical knowledge to operate.

[0462] Terminals, or in-store devices, automatically prepare and begin operations according to the operational schedule received from the server. A specific example is that multiple serving robots operate during peak hours, while only the minimum necessary number operate during off-peak hours. This enables efficient resource management.

[0463] Furthermore, after the end of the workday, the server accumulates feedback data on the execution results and updates the predictive model based on this data. Through this process, the accuracy of the predictions improves over time, which is useful for formulating future plans.

[0464] As a concrete example, it is possible to optimize the staffing and machine operation plans for a specific store on days when sunny weather is forecast for the weekend. An example of a prompt message could be, "Please provide an optimal staffing and serving robot deployment plan based on the predicted number of customers on a sunny Sunday next week." Based on such input, the system will propose the optimal plan.

[0465] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0466] Step 1:

[0467] The server collects commercial transaction information, weather information, event information, and employee request information via a data processing device. Inputs include data from POS systems, weather APIs, event information services, and labor management systems. The output is an integrated dataset, ready for subsequent processing such as analysis and prediction. Specifically, it uses an authentication key to access data from APIs and stores the obtained information in a database.

[0468] Step 2:

[0469] The server executes a machine learning algorithm using the collected data. The input here is the integrated dataset obtained in Step 1. Data processing involves encoding categorical data and imputing missing values ​​to format the data into an analyzable format. The output is future sales forecast data. In this process, TensorFlow is used to apply a trained model and generate predicted values. Specifically, the server performs CSV file validation as data preprocessing and handles outliers as needed.

[0470] Step 3:

[0471] The server optimizes labor allocation and machinery operation plans based on predicted sales data. The input is the sales forecast data obtained in step 2. The data calculations include an optimization algorithm using linear programming. The output is a specific and optimized shift allocation and operation schedule. In this process, shifts are adjusted to comply with employee labor laws and meet their desired working hours. Specifically, it performs resource allocation simulations based on a numerical model, distinguishing between peak and off-peak periods.

[0472] Step 4:

[0473] The store manager, as the user, reviews the provided work shifts and operational plans and makes adjustments as needed. The input is the plan data from the server, and the output is the adjusted final plan. Specifically, the shift schedule can be viewed on the management screen, and working hours can be edited using drag-and-drop operations via the GUI. These adjustments are made to reflect on-site situations, such as leave requests from specific staff members.

[0474] Step 5:

[0475] The terminal starts operating the machinery according to the operational schedule received from the server. The input is the final operational plan from step 4, and the output is the operational data of the machinery. Specifically, the terminal automatically turns on the power to the serving robot and starts the initialization process. This enables efficient work execution within the set time.

[0476] Step 6:

[0477] The server collects actual sales data and machine operation data as feedback after the end of operations. The input is actual sales and operation performance data obtained from stores, and the output is real-time data for the improved prediction algorithm and the next operation plan. The specific operation includes a process of detecting outliers and deviations from the past through a database update at night and correcting the prediction model for the next operation.

[0478] (Application Example 1)

[0479] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0480] In physical stores, there is a need to efficiently optimize staff work schedules and machine operation schedules, and to enable real-time adjustments. However, conventional systems have problems such as insufficient accuracy in revenue forecasting and difficulty in easily modifying plans. This has led to decreased operational efficiency in physical stores and the resulting increase in extra effort and costs.

[0481] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0482] In this invention, the server includes means for acquiring data, means for performing revenue forecasts based on the data, means for optimizing work plans and machine operation plans based on the revenue forecasts, means for visualizing and making modifiable the optimized work plans and machine operation plans, means for improving the forecast model using feedback data, and means for viewing and modifying the work plans in real time via a mobile terminal. This enables efficient optimization and flexible modification of plans in physical stores.

[0483] "Means of acquiring data" refers to elements that have the function of collecting various types of information necessary for use in revenue forecasting.

[0484] "Means for performing revenue forecasting" refers to elements that perform calculations to predict future revenue based on collected data.

[0485] "Means for optimizing work plans and machine operation plans" refer to elements that efficiently set personnel allocations and machine operation based on projected revenue.

[0486] "Means of visualization and modification" refer to elements that visually display the optimized plan and allow users to easily make changes.

[0487] "Methods for improving predictive models using feedback data" refer to elements that involve aggregating actual operational results and updating predictive algorithms to improve the accuracy of revenue forecasts.

[0488] "Means of viewing and modifying in real time via mobile devices" refers to elements that enable users to check plans in real time using smartphones or tablets and make adjustments as needed.

[0489] The server provides an integrated system to improve the operational efficiency of the stores. First, it collects store sales information, weather information, event information, and part-time staff work preference information through data acquisition methods. This data is stored in a cloud-hosted database and processed by a sales forecasting system. This processing utilizes machine learning algorithms and the TensorFlow library using Python to predict future sales with high accuracy.

[0490] The server has the means to optimize work plans and machine operation plans based on the acquired predictive information. The Django framework is used for communication between the server and the frontend, supporting data movement within the system. Once the optimization process is complete, the plan is visualized through a user interface developed with React Native, allowing store managers to review it via their mobile devices and make real-time modifications as needed.

[0491] The terminal manages the operation of in-store equipment according to an optimized plan. In particular, it increases efficiency by operating more machinery and equipment during peak hours and reducing operation during off-peak hours. In addition, after closing time, the server improves its predictive model based on feedback data and makes even more precise predictions for the next business day.

[0492] For example, if weather changes or local events are predicted, the server reflects this in real time and immediately updates possible staffing and machinery operation plans. This dynamic adaptation maximizes the efficiency of store operations.

[0493] Examples of prompts include: "Update next week's sales forecast and provide optimal work shifts and equipment operation plans based on expected weather and local events."

[0494] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0495] Step 1:

[0496] The server collects sales information, weather information, event information, and employee work preference information using various data acquisition methods. POS systems, weather APIs, event databases, and employee management software are used as inputs. This data is stored in a cloud database and used for subsequent processing.

[0497] Step 2:

[0498] The server performs revenue forecasting based on the collected data. Machine learning algorithms using Python and TensorFlow are applied, performing numerical analysis on the input data and outputting future sales forecasts. This calculation predicts increases or decreases in sales.

[0499] Step 3:

[0500] The server optimizes work plans and machine operation plans based on revenue forecasts. An optimization algorithm using the Django framework takes forecast data as input and calculates staffing and machine operation schedules. An optimized shift schedule is then generated as output.

[0501] Step 4:

[0502] The server visualizes the optimized schedule through a user interface, allowing store managers to check it on their mobile devices. Using React Native, the output is displayed on smartphones and tablets, creating an environment where managers can easily review and modify operations using a user-friendly interface.

[0503] Step 5:

[0504] The terminal operates the machinery and equipment based on an optimized plan received from the server. Specifically, during peak hours, the terminal operates multiple devices to compensate for personnel shortages. During off-peak hours, operation is minimized to maximize energy efficiency.

[0505] Step 6:

[0506] After closing time, the server collects feedback data. This includes actual sales information and machine operation data, which are used as input. Next, the predictive model is updated based on this data, and TensorFlow is used to improve the model's accuracy. Output is then generated to further refine the next shift and machine operation plan.

[0507] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0508] This invention combines a system that optimizes staffing and machine operation in restaurants with an emotion engine that recognizes user emotions. This system consists of a server, terminals, and users, and functions in an integrated manner.

[0509] The server first uses multiple data collection methods to gather sales information, weather information, event information, and employee work preference information. Simultaneously, it obtains emotional information from users' facial expressions, voices, and behaviors through an emotion engine. This emotional information is essential data for sales forecasting and work shift optimization.

[0510] Sales forecasting methods use this data as input to predict future sales, particularly using machine learning algorithms. In this process, supplementary data on customer purchasing intent and satisfaction, including sentiment information, is considered, enabling more accurate predictions.

[0511] Next, the server optimizes work shifts and machine operation plans based on sales forecast data and sentiment-based information. For example, during times when customers are emotionally positive, the system prioritizes their feedback and either increases staffing or deploys additional machinery. As a result, efficient store operations are achieved while maintaining customer satisfaction.

[0512] The user, i.e., the store manager, is notified of the optimized shift and machine operation plan calculated from the server. Based on this, the user can make adjustments to suit the actual situation on site and determine the final work shift.

[0513] The serving robots, acting as terminals, prepare according to the operational schedule received from the server and perform efficient serving tasks at the designated time. Furthermore, they can utilize data provided by the emotion engine to grasp customer emotions in real time from facial expressions and voice, and use this information to adjust the operational schedule.

[0514] Finally, the server collects actual sales data, work performance data, and sentiment data as feedback after business hours. This allows for improvements to the sales forecasting model and operational optimization algorithms to enhance future prediction accuracy.

[0515] This system allows stores to provide efficient service while minimizing labor costs and operating costs for machinery, all while considering customer satisfaction.

[0516] The following describes the processing flow.

[0517] Step 1:

[0518] The server uses data collection tools to gather store sales information, weather information, event information, and employee work preference information. Simultaneously, an emotion engine analyzes the user's facial expressions, voice, and behavior to obtain customer emotion information.

[0519] Step 2:

[0520] The server activates a sales forecasting system, using collected data and sentiment information as input to predict future sales. In particular, it uses machine learning algorithms to generate a predictive model that reflects the influence of sentiment information on customer purchasing behavior.

[0521] Step 3:

[0522] Based on this sales forecast data, the server calculates staffing levels using work shift optimization methods. It also develops a machine operation plan to allocate additional personnel and equipment during peak customer sentiment periods.

[0523] Step 4:

[0524] The store manager, as the user, reviews the optimized work shifts and machine operation plans notified by the server. They then make adjustments as needed, based on on-site conditions and their own experience.

[0525] Step 5:

[0526] The serving robot, acting as the terminal, begins operations based on the received schedule. It monitors customer emotion data in real time, instantly adjusts its actions as needed, and performs efficient food delivery.

[0527] Step 6:

[0528] After closing time, the server collects actual sales data for the day, machine operating status, employee work performance, and sentiment information as feedback data.

[0529] Step 7:

[0530] The server analyzes feedback data and updates the sales forecasting model and work shift optimization algorithm. This improves the accuracy of future predictions and optimizations, enabling further service improvements.

[0531] (Example 2)

[0532] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0533] Restaurants need to efficiently optimize staffing and equipment usage to improve customer satisfaction while keeping operating costs down. However, conventional systems are insufficient as they rely solely on sales data and work preference information for forecasting and planning, and they cannot flexibly adjust operations based on customer sentiment. Therefore, there is a need for a system that can optimize operations by reflecting customer sentiment in real time.

[0534] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0535] In this invention, the server includes means for collecting data, means for making sales forecasts, and means for optimizing work shifts and machine operation plans. This enables more accurate optimization of operation plans not only based on sales forecasts but also by utilizing user sentiment information. Furthermore, continuous accuracy improvement is achieved by updating the prediction model using feedback data. This makes it possible to operate stores efficiently while minimizing operating costs and taking customer satisfaction into consideration.

[0536] "Data collection means" refers to devices or methods for acquiring sales information, weather information, event information, work preference information, and sentiment data.

[0537] A "sales forecasting method" refers to an algorithm or calculation technique used to predict future sales based on collected data.

[0538] A "work shift optimization method" is a system that optimizes employee working hours based on sales forecasts and sentiment information to most efficiently allocate employee working hours.

[0539] "Machinery and equipment operation plan optimization means" refers to a method or apparatus for optimizing the operation schedule of machinery and equipment based on sales forecasts and sentiment information.

[0540] "Feedback data" refers to data including actual sales information, machine operation status, and acquired sentiment information, which is used to update predictive models.

[0541] "Means for acquiring emotional information" refers to a device or method for analyzing a user's facial expressions and voice to identify their emotions.

[0542] An "emotion-based adjustment mechanism" is a system that dynamically adjusts machinery and personnel allocation based on emotional information acquired in real time.

[0543] Modes for carrying out the invention

[0544] This invention is a system for achieving efficient staffing and operation of machinery in restaurants. Specifically, it optimizes work shifts and machinery based on sales forecasts and sentiment data to improve the operational efficiency of the restaurant.

[0545] The server is responsible for collecting data. The hardware used includes a database server, and the software is used to retrieve data using SQL or APIs. Sales information, weather information, event information, work preference information, and sentiment data are collected.

[0546] The server uses the collected data to perform sales forecasts. Here, machine learning models are utilized, employing libraries such as TensorFlow and PyTorch. By also incorporating sentiment information, highly accurate sales forecasts are achieved.

[0547] Subsequently, the server generates optimal work shifts and machinery operation plans based on the prediction results. This uses optimization methods based on linear programming and genetic algorithms to achieve efficient operation.

[0548] The store manager, as the user, receives optimized information provided by the server and makes adjustments according to the actual situation on site. This enables flexible store operations.

[0549] The food delivery robot, which is part of the terminal, receives operational schedules from the server and performs specific tasks. Furthermore, it analyzes customers' facial expressions and voices in real time using an emotion engine and adjusts operations as needed. Another example is the improvement of food delivery services, which directly leads to increased customer satisfaction.

[0550] As a concrete example, the server predicts an increase in customer traffic on specific event days or under certain weather conditions, and plans to increase the operation of serving robots accordingly. Based on this, the user can review staffing during peak hours and operate more efficiently.

[0551] Example of a prompt:

[0552] "Please provide a plan to optimize staffing and machinery operations based on next day's sales forecast and sentiment data."

[0553] This system allows stores to maintain high customer satisfaction while keeping operating costs down.

[0554] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0555] Step 1:

[0556] The server collects sales information, weather information, event information, work preference information, and user sentiment data from the database. Input data is retrieved in digital format, and queries are executed against the database using SQL. This process prepares all the data necessary for prediction and optimization as initial input data. The output includes this prepared data set.

[0557] Step 2:

[0558] The server uses a machine learning model to predict sales based on the collected data. Inputs include sales data, weather conditions, event information, and sentiment data. The algorithm utilizes a prediction model based on TensorFlow. The output generates future sales forecasts. This process involves data analysis and numerical prediction.

[0559] Step 3:

[0560] The server considers sales forecasts and sentiment data to create optimal work shifts and machine operation plans. Inputs include sales forecasts and sentiment information. A linear programming-based optimization algorithm is used to output shifts and operation schedules that are most efficient and cost-effective. Specifically, it creates staffing schedules and robot operation schedules.

[0561] Step 4:

[0562] The server notifies the store manager, who is the user, of the optimized operational plan. The input is the optimized plan, and the output is notification data to the manager. This process involves sending information through email applications and management dashboards. Based on this notification, the manager develops a feasible, final store operational plan.

[0563] Step 5:

[0564] The serving robot, acting as the terminal, prepares according to the operational plan from the server. Its input includes an optimized operational schedule, and its output provides a specific serving operation plan for each time slot. During this process, the robot acquires customer emotion data in real time and dynamically adjusts the serving schedule as needed.

[0565] Step 6:

[0566] The server collects actual sales data, work performance data, and sentiment data obtained after business hours, and uses this as feedback for future model improvements. The input is daily business results data, and the output is feedback information that helps improve the predictive model. This process ensures continuous improvement of the model's accuracy.

[0567] (Application Example 2)

[0568] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0569] In the food and beverage industry, there is a demand for services that respond immediately to individual needs in order to increase customer satisfaction. At the same time, it is necessary to achieve efficient operations while minimizing labor costs and the operating costs of machinery and equipment. However, with conventional methods, it has been difficult to recognize customer emotions in real time and dynamically adjust services based on them. As a result, there has been a challenge in being able to plan appropriate staffing and machinery operations that take emotions into consideration.

[0570] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0571] In this invention, the server includes means for collecting data, means for making sales forecasts based on the data, and means for optimizing work schedules and machine operation plans based on the sales forecasts. This makes it possible to recognize and analyze customer sentiment information in real time and optimize personnel allocation and machine operation accordingly.

[0572] "Means of data collection" refers to a system for acquiring and integrating store sales information, weather information, event information, work preference information, and customer facial and voice information.

[0573] A "means of sales forecasting" refers to a device that uses machine learning algorithms to predict future sales based on collected data.

[0574] "Means for optimizing work schedules and machinery operation plans" refers to a system for planning and adjusting efficient staffing and machinery operation, taking into account predicted sales information and customer sentiment data.

[0575] "Methods for updating predictive models using feedback data" refers to the process of automatically improving a model to enhance the accuracy of sales forecasts by using actual sales data and customer sentiment data.

[0576] "Means for recognizing and analyzing customer emotions through an emotion engine" refers to a device that includes an algorithm for detecting and classifying a customer's emotional state by analyzing image and audio data.

[0577] "A means of adjusting service delivery in real time" refers to a system that dynamically instructs employees to optimize service content based on emotional data acquired from customers.

[0578] "A means of improving service by presenting emotion analysis results to employees using smart devices" refers to a technology that provides information to employees through devices such as smart glasses, enabling them to check the emotional state of customers in real time and respond appropriately.

[0579] This invention is a system for optimizing staffing and the operation of machinery and equipment in restaurants.

[0580] This system consists of a server, smart devices (e.g., smart glasses), and terminals placed within the store. The server is the central hub responsible for collecting data, forecasting sales, optimizing work schedules, and planning the operation of machinery and equipment.

[0581] The server uses multiple data collection methods to gather sales information, weather information, event information, and employee work preference information. It also uses an emotion engine to obtain emotional information from customer facial expressions and voice. Python and OpenCV libraries are used for image recognition, and the Google Cloud Speech-to-Text API is used for voice analysis. Based on this data, a machine learning algorithm is used to predict sales.

[0582] The server then uses predicted sales data and sentiment information to create an optimal work schedule and equipment operation plan. It can adjust employee deployment and machine operation in real time based on the customer's emotional state. For example, employees can monitor customer emotions in real time through smart glasses and modify service as needed.

[0583] Furthermore, after business hours, the server collects actual sales data, the results of the tasks performed, and customer sentiment data, and provides feedback to build increasingly accurate sales forecasting models and operational optimization algorithms.

[0584] As a concrete example, consider a scenario where smart glasses are used to monitor the emotional state of customers visiting a restaurant with their family. If a child shows signs of boredom, a special menu item is suggested to the employee. This allows for concrete implementation of how smart devices are used and how information is processed.

[0585] Examples of prompt statements for a generative AI model are as follows:

[0586] "Please tell me about a system that uses video data captured by smart glasses to determine customer emotions in real time through image recognition and voice analysis, and then notifies employees of the appropriate service."

[0587] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0588] Step 1:

[0589] The server collects sales information, weather information, event information, work preference information, and customer facial expressions and voice data as input. This data is gathered from in-store and external databases using sensors, cameras, microphones, etc. This allows for the integrated accumulation of environmental and customer information.

[0590] Step 2:

[0591] The server uses the collected information as input to perform image recognition and speech analysis. For image recognition, it analyzes customer facial expressions using Python and the OpenCV library, and for speech analysis, it uses the Google Cloud Speech-to-Text API. The data is processed by a machine learning model to identify the customer's emotions. As a result, the customer's emotional state is output.

[0592] Step 3:

[0593] The server makes sales forecasts based on customer emotional states and other environmental data. A machine learning algorithm drives the process of predicting future sales. This predicted data is then used as the result for the next step in the process.

[0594] Step 4:

[0595] The server uses sales forecast results and sentiment data as input to optimize work schedules and machine operation plans. An optimization algorithm is used to output staffing plans that are adapted to each customer's needs and sentiments.

[0596] Step 5:

[0597] The smart device (smart glasses) provides employees with real-time sentiment analysis results and optimized service guidance. Through the device, employees can receive immediate feedback, which they can then use to improve their service delivery.

[0598] Step 6:

[0599] Based on feedback from the server, users will ultimately adjust their work schedules and service content. This includes implementing specific service improvements tailored to anticipated situations.

[0600] Step 7:

[0601] The server collects feedback data obtained after business hours and updates its predictive models and optimization algorithms. This improves the overall accuracy and efficiency of the system.

[0602] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0603] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0604] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0605] [Fourth Embodiment]

[0606] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0607] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0608] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0609] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0610] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0611] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0612] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0613] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0614] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0615] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0616] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0617] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0618] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0619] This invention provides a system for optimizing staff work shifts and machinery operation plans in restaurants. The system consists of server, terminal, and user elements, each playing a specific role.

[0620] The server first uses data collection tools to acquire store sales information, weather information, event information, and employee work preference information. This data is processed by sales forecasting tools, which in particular use machine learning algorithms to predict future sales. Based on this forecast information, the server uses work shift optimization tools to calculate the optimal staffing arrangement and further develops a machine operation plan.

[0621] Store managers, as users, can review the optimized shifts and operational plans provided by the server and manually adjust them as needed. This allows users to implement operations that reflect the needs of the store staff.

[0622] Terminals, or the machines operating within the store, prepare themselves based on the operational schedule received from the server and then perform food serving tasks. As a concrete example, multiple terminals can be operated during peak daytime hours to compensate for labor shortages, while only the minimum necessary terminals can be operated during off-peak nighttime hours, thereby achieving efficient operation.

[0623] Furthermore, the server collects feedback data after business hours and analyzes predictions and actual results. This allows for the improvement of the sales forecast model using forecast model update mechanisms, resulting in more accurate plans for future shifts and machine operation.

[0624] As described above, the present invention provides an integrated solution for improving the operational efficiency of stores, simultaneously achieving reductions in labor costs and improvements in service quality.

[0625] The following describes the processing flow.

[0626] Step 1:

[0627] The server uses data collection methods to gather historical sales information, weather forecast data, store event information, and employee work preference data. This includes using various APIs and accessing databases.

[0628] Step 2:

[0629] The server activates the sales forecasting system and applies machine learning algorithms based on the collected data to predict future sales. The sales forecasting model is created based on historical data and updated as needed.

[0630] Step 3:

[0631] The server uses a work shift optimization method based on predicted sales data to calculate the optimal staffing. At the same time, it formulates a machine operation plan and determines the number of machines and their operating hours to be used in each time slot.

[0632] Step 4:

[0633] The store manager, as the user, is notified of the optimized work shifts and machine operation plans calculated by the server. The user can review this information and adjust the shifts and plans as needed.

[0634] Step 5:

[0635] The terminal begins preparations based on the operational schedule received from the server. Following the specified schedule, it starts serving meals and efficiently delivers them according to a pre-configured workflow.

[0636] Step 6:

[0637] After business hours, the server aggregates feedback data. This includes sales data for the day, machine operation status, and employee work performance.

[0638] Step 7:

[0639] The server updates its sales forecasting model and shift optimization algorithm based on feedback data. This improves the model so that future predictions and optimizations are more accurate.

[0640] (Example 1)

[0641] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0642] In store operations such as restaurants and retail, optimal staffing and efficient use of machinery are required. However, current methods make accurate sales forecasting difficult, leading to excessive labor costs and a decline in service quality. Furthermore, planning that takes into account external factors such as fluctuating weather and events is insufficient, thus requiring practical and flexible operational methods.

[0643] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0644] In this invention, the server includes means for collecting commercial transaction information, weather information, event information, and employee request information via a data processing device; means for predicting future sales using a machine learning algorithm based on the collected information; and means for optimizing labor allocation and machinery operation plans based on the predictions. This enables flexible and efficient store operations and staffing in response to fluctuating demand.

[0645] A "data processing device" is an information processing system used to collect and analyze commercial transaction information, weather information, event information, and employee request information.

[0646] "Commercial transaction information" refers to data on sales and transactions at stores, and is used for business analysis and sales forecasting.

[0647] "Weather information" refers to data on meteorological conditions and is used for analysis as a factor that influences sales and customer numbers.

[0648] "Event information" refers to information about events and activities held in the local area, which contributes to predicting customer traffic and managing resources for stores.

[0649] "Employee request information" refers to information regarding staff work preferences and shifts, and serves as basic data for optimizing labor allocation.

[0650] A "machine learning algorithm" refers to a computational method used to find patterns in input data and make predictions or classifications about the future.

[0651] "Labor allocation" refers to the planning of employee work shifts and assigned tasks in store operations.

[0652] "Machinery and equipment" is a general term for automated devices and systems used in stores, and specifically includes serving robots, vending machines, etc.

[0653] An "operational plan" refers to a plan for efficiently carrying out operations through the appropriate allocation and management of resources.

[0654] This invention is a system for optimizing labor allocation and machinery operation planning in store operations. The system consists of server, terminal, and user elements.

[0655] The server collects information via data processing devices. Specifically, commercial transaction information is obtained from the store's sales information management system, and weather information is obtained using a weather data provision API. Event information is obtained through an event data service, and employee request information is obtained from the labor management system via an API.

[0656] The server uses this data to perform sales forecasts using software such as TensorFlow or PyTorch to execute machine learning algorithms. Based on the predicted sales data, the server uses a linear programming algorithm to create an optimal work shift and machinery operation plan.

[0657] Store managers, as users, can view these plans provided by the server on their management terminals and make adjustments as needed through the interface. These adjustments are intuitive and can be performed using a GUI, making it easy for users without special technical knowledge to operate.

[0658] Terminals, or in-store devices, automatically prepare and begin operations according to the operational schedule received from the server. A specific example is that multiple serving robots operate during peak hours, while only the minimum necessary number operate during off-peak hours. This enables efficient resource management.

[0659] Furthermore, after the end of the workday, the server accumulates feedback data on the execution results and updates the predictive model based on this data. Through this process, the accuracy of the predictions improves over time, which is useful for formulating future plans.

[0660] As a concrete example, it is possible to optimize the staffing and machine operation plans for a specific store on days when sunny weather is forecast for the weekend. An example of a prompt message could be, "Please provide an optimal staffing and serving robot deployment plan based on the predicted number of customers on a sunny Sunday next week." Based on such input, the system will propose the optimal plan.

[0661] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0662] Step 1:

[0663] The server collects commercial transaction information, weather information, event information, and employee request information via a data processing device. Inputs include data from POS systems, weather APIs, event information services, and labor management systems. The output is an integrated dataset, ready for subsequent processing such as analysis and prediction. Specifically, it uses an authentication key to access data from APIs and stores the obtained information in a database.

[0664] Step 2:

[0665] The server executes a machine learning algorithm using the collected data. The input here is the integrated dataset obtained in Step 1. Data processing involves encoding categorical data and imputing missing values ​​to format the data into an analyzable format. The output is future sales forecast data. In this process, TensorFlow is used to apply a trained model and generate predicted values. Specifically, the server performs CSV file validation as data preprocessing and handles outliers as needed.

[0666] Step 3:

[0667] The server optimizes labor allocation and machinery operation plans based on predicted sales data. The input is the sales forecast data obtained in step 2. The data calculations include an optimization algorithm using linear programming. The output is a specific and optimized shift allocation and operation schedule. In this process, shifts are adjusted to comply with employee labor laws and meet their desired working hours. Specifically, it performs resource allocation simulations based on a numerical model, distinguishing between peak and off-peak periods.

[0668] Step 4:

[0669] The store manager, as the user, reviews the provided work shifts and operational plans and makes adjustments as needed. The input is the plan data from the server, and the output is the adjusted final plan. Specifically, the shift schedule can be viewed on the management screen, and working hours can be edited using drag-and-drop operations via the GUI. These adjustments are made to reflect on-site situations, such as leave requests from specific staff members.

[0670] Step 5:

[0671] The terminal starts operating the machinery according to the operational schedule received from the server. The input is the final operational plan from step 4, and the output is the operational data of the machinery. Specifically, the terminal automatically turns on the power to the serving robot and starts the initialization process. This enables efficient work execution within the set time.

[0672] Step 6:

[0673] The server collects actual sales data and machine operation data as feedback after the end of operations. The input is actual sales and operation performance data obtained from stores, and the output is real-time data for the improved prediction algorithm and the next operation plan. The specific operation includes a process of detecting outliers and deviations from the past through a database update at night and correcting the prediction model for the next operation.

[0674] (Application Example 1)

[0675] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0676] In physical stores, there is a need to efficiently optimize staff work schedules and machine operation schedules, and to enable real-time adjustments. However, conventional systems have problems such as insufficient accuracy in revenue forecasting and difficulty in easily modifying plans. This has led to decreased operational efficiency in physical stores and the resulting increase in extra effort and costs.

[0677] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0678] In this invention, the server includes means for acquiring data, means for performing revenue forecasts based on the data, means for optimizing work plans and machine operation plans based on the revenue forecasts, means for visualizing and making modifiable the optimized work plans and machine operation plans, means for improving the forecast model using feedback data, and means for viewing and modifying the work plans in real time via a mobile terminal. This enables efficient optimization and flexible modification of plans in physical stores.

[0679] "Means of acquiring data" refers to elements that have the function of collecting various types of information necessary for use in revenue forecasting.

[0680] "Means for performing revenue forecasting" refers to elements that perform calculations to predict future revenue based on collected data.

[0681] "Means for optimizing work plans and machine operation plans" refer to elements that efficiently set personnel allocations and machine operation based on projected revenue.

[0682] "Means of visualization and modification" refer to elements that visually display the optimized plan and allow users to easily make changes.

[0683] "Methods for improving predictive models using feedback data" refer to elements that involve aggregating actual operational results and updating predictive algorithms to improve the accuracy of revenue forecasts.

[0684] "Means of viewing and modifying in real time via mobile devices" refers to elements that enable users to check plans in real time using smartphones or tablets and make adjustments as needed.

[0685] The server provides an integrated system to improve the operational efficiency of the stores. First, it collects store sales information, weather information, event information, and part-time staff work preference information through data acquisition methods. This data is stored in a cloud-hosted database and processed by a sales forecasting system. This processing utilizes machine learning algorithms and the TensorFlow library using Python to predict future sales with high accuracy.

[0686] The server has the means to optimize work plans and machine operation plans based on the acquired predictive information. The Django framework is used for communication between the server and the frontend, supporting data movement within the system. Once the optimization process is complete, the plan is visualized through a user interface developed with React Native, allowing store managers to review it via their mobile devices and make real-time modifications as needed.

[0687] The terminal manages the operation of in-store equipment according to an optimized plan. In particular, it increases efficiency by operating more machinery and equipment during peak hours and reducing operation during off-peak hours. In addition, after closing time, the server improves its predictive model based on feedback data and makes even more precise predictions for the next business day.

[0688] For example, if weather changes or local events are predicted, the server reflects this in real time and immediately updates possible staffing and machinery operation plans. This dynamic adaptation maximizes the efficiency of store operations.

[0689] Examples of prompts include: "Update next week's sales forecast and provide optimal work shifts and equipment operation plans based on expected weather and local events."

[0690] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0691] Step 1:

[0692] The server collects sales information, weather information, event information, and employee work preference information using various data acquisition methods. POS systems, weather APIs, event databases, and employee management software are used as inputs. This data is stored in a cloud database and used for subsequent processing.

[0693] Step 2:

[0694] The server performs revenue forecasting based on the collected data. Machine learning algorithms using Python and TensorFlow are applied, performing numerical analysis on the input data and outputting future sales forecasts. This calculation predicts increases or decreases in sales.

[0695] Step 3:

[0696] The server optimizes work plans and machine operation plans based on revenue forecasts. An optimization algorithm using the Django framework takes forecast data as input and calculates staffing and machine operation schedules. An optimized shift schedule is then generated as output.

[0697] Step 4:

[0698] The server visualizes the optimized schedule through a user interface, allowing store managers to check it on their mobile devices. Using React Native, the output is displayed on smartphones and tablets, creating an environment where managers can easily review and modify operations using a user-friendly interface.

[0699] Step 5:

[0700] The terminal operates the machinery and equipment based on an optimized plan received from the server. Specifically, during peak hours, the terminal operates multiple devices to compensate for personnel shortages. During off-peak hours, operation is minimized to maximize energy efficiency.

[0701] Step 6:

[0702] After closing time, the server collects feedback data. This includes actual sales information and machine operation data, which are used as input. Next, the predictive model is updated based on this data, and TensorFlow is used to improve the model's accuracy. Output is then generated to further refine the next shift and machine operation plan.

[0703] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0704] This invention combines a system that optimizes staffing and machine operation in restaurants with an emotion engine that recognizes user emotions. This system consists of a server, terminals, and users, and functions in an integrated manner.

[0705] The server first uses multiple data collection methods to gather sales information, weather information, event information, and employee work preference information. Simultaneously, it obtains emotional information from users' facial expressions, voices, and behaviors through an emotion engine. This emotional information is essential data for sales forecasting and work shift optimization.

[0706] Sales forecasting methods use this data as input to predict future sales, particularly using machine learning algorithms. In this process, supplementary data on customer purchasing intent and satisfaction, including sentiment information, is considered, enabling more accurate predictions.

[0707] Next, the server optimizes work shifts and machine operation plans based on sales forecast data and sentiment-based information. For example, during times when customers are emotionally positive, the system prioritizes their feedback and either increases staffing or deploys additional machinery. As a result, efficient store operations are achieved while maintaining customer satisfaction.

[0708] The user, i.e., the store manager, is notified of the optimized shift and machine operation plan calculated from the server. Based on this, the user can make adjustments to suit the actual situation on site and determine the final work shift.

[0709] The serving robots, acting as terminals, prepare according to the operational schedule received from the server and perform efficient serving tasks at the designated time. Furthermore, they can utilize data provided by the emotion engine to grasp customer emotions in real time from facial expressions and voice, and use this information to adjust the operational schedule.

[0710] Finally, the server collects actual sales data, work performance data, and sentiment data as feedback after business hours. This allows for improvements to the sales forecasting model and operational optimization algorithms to enhance future prediction accuracy.

[0711] This system allows stores to provide efficient service while minimizing labor costs and operating costs for machinery, all while considering customer satisfaction.

[0712] The following describes the processing flow.

[0713] Step 1:

[0714] The server uses data collection tools to gather store sales information, weather information, event information, and employee work preference information. Simultaneously, an emotion engine analyzes the user's facial expressions, voice, and behavior to obtain customer emotion information.

[0715] Step 2:

[0716] The server activates a sales forecasting system, using collected data and sentiment information as input to predict future sales. In particular, it uses machine learning algorithms to generate a predictive model that reflects the influence of sentiment information on customer purchasing behavior.

[0717] Step 3:

[0718] Based on this sales forecast data, the server calculates staffing levels using work shift optimization methods. It also develops a machine operation plan to allocate additional personnel and equipment during peak customer sentiment periods.

[0719] Step 4:

[0720] The store manager, as the user, reviews the optimized work shifts and machine operation plans notified by the server. They then make adjustments as needed, based on on-site conditions and their own experience.

[0721] Step 5:

[0722] The serving robot, acting as the terminal, begins operations based on the received schedule. It monitors customer emotion data in real time, instantly adjusts its actions as needed, and performs efficient food delivery.

[0723] Step 6:

[0724] After closing time, the server collects actual sales data for the day, machine operating status, employee work performance, and sentiment information as feedback data.

[0725] Step 7:

[0726] The server analyzes feedback data and updates the sales forecasting model and work shift optimization algorithm. This improves the accuracy of future predictions and optimizations, enabling further service improvements.

[0727] (Example 2)

[0728] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0729] Restaurants need to efficiently optimize staffing and equipment usage to improve customer satisfaction while keeping operating costs down. However, conventional systems are insufficient as they rely solely on sales data and work preference information for forecasting and planning, and they cannot flexibly adjust operations based on customer sentiment. Therefore, there is a need for a system that can optimize operations by reflecting customer sentiment in real time.

[0730] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0731] In this invention, the server includes means for collecting data, means for making sales forecasts, and means for optimizing work shifts and machine operation plans. This enables more accurate optimization of operation plans not only based on sales forecasts but also by utilizing user sentiment information. Furthermore, continuous accuracy improvement is achieved by updating the prediction model using feedback data. This makes it possible to operate stores efficiently while minimizing operating costs and taking customer satisfaction into consideration.

[0732] "Data collection means" refers to devices or methods for acquiring sales information, weather information, event information, work preference information, and sentiment data.

[0733] A "sales forecasting method" refers to an algorithm or calculation technique used to predict future sales based on collected data.

[0734] A "work shift optimization method" is a system that optimizes employee working hours based on sales forecasts and sentiment information to most efficiently allocate employee working hours.

[0735] "Machinery and equipment operation plan optimization means" refers to a method or apparatus for optimizing the operation schedule of machinery and equipment based on sales forecasts and sentiment information.

[0736] "Feedback data" refers to data including actual sales information, machine operation status, and acquired sentiment information, which is used to update predictive models.

[0737] "Means for acquiring emotional information" refers to a device or method for analyzing a user's facial expressions and voice to identify their emotions.

[0738] An "emotion-based adjustment mechanism" is a system that dynamically adjusts machinery and personnel allocation based on emotional information acquired in real time.

[0739] Modes for carrying out the invention

[0740] This invention is a system for achieving efficient staffing and operation of machinery in restaurants. Specifically, it optimizes work shifts and machinery based on sales forecasts and sentiment data to improve the operational efficiency of the restaurant.

[0741] The server is responsible for collecting data. The hardware used includes a database server, and the software is used to retrieve data using SQL or APIs. Sales information, weather information, event information, work preference information, and sentiment data are collected.

[0742] The server uses the collected data to perform sales forecasts. Here, machine learning models are utilized, employing libraries such as TensorFlow and PyTorch. By also incorporating sentiment information, highly accurate sales forecasts are achieved.

[0743] Subsequently, the server generates optimal work shifts and machinery operation plans based on the prediction results. This uses optimization methods based on linear programming and genetic algorithms to achieve efficient operation.

[0744] The store manager, as the user, receives optimized information provided by the server and makes adjustments according to the actual situation on site. This enables flexible store operations.

[0745] The food delivery robot, which is part of the terminal, receives operational schedules from the server and performs specific tasks. Furthermore, it analyzes customers' facial expressions and voices in real time using an emotion engine and adjusts operations as needed. Another example is the improvement of food delivery services, which directly leads to increased customer satisfaction.

[0746] As a concrete example, the server predicts an increase in customer traffic on specific event days or under certain weather conditions, and plans to increase the operation of serving robots accordingly. Based on this, the user can review staffing during peak hours and operate more efficiently.

[0747] Example of a prompt:

[0748] "Please provide a plan to optimize staffing and machinery operations based on next day's sales forecast and sentiment data."

[0749] This system allows stores to maintain high customer satisfaction while keeping operating costs down.

[0750] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0751] Step 1:

[0752] The server collects sales information, weather information, event information, work preference information, and user sentiment data from the database. Input data is retrieved in digital format, and queries are executed against the database using SQL. This process prepares all the data necessary for prediction and optimization as initial input data. The output includes this prepared data set.

[0753] Step 2:

[0754] The server uses a machine learning model to predict sales based on the collected data. Inputs include sales data, weather conditions, event information, and sentiment data. The algorithm utilizes a prediction model based on TensorFlow. The output generates future sales forecasts. This process involves data analysis and numerical prediction.

[0755] Step 3:

[0756] The server considers sales forecasts and sentiment data to create optimal work shifts and machine operation plans. Inputs include sales forecasts and sentiment information. A linear programming-based optimization algorithm is used to output shifts and operation schedules that are most efficient and cost-effective. Specifically, it creates staffing schedules and robot operation schedules.

[0757] Step 4:

[0758] The server notifies the store manager, who is the user, of the optimized operational plan. The input is the optimized plan, and the output is notification data to the manager. This process involves sending information through email applications and management dashboards. Based on this notification, the manager develops a feasible, final store operational plan.

[0759] Step 5:

[0760] The serving robot, acting as the terminal, prepares according to the operational plan from the server. Its input includes an optimized operational schedule, and its output provides a specific serving operation plan for each time slot. During this process, the robot acquires customer emotion data in real time and dynamically adjusts the serving schedule as needed.

[0761] Step 6:

[0762] The server collects actual sales data, work performance data, and sentiment data obtained after business hours, and uses this as feedback for future model improvements. The input is daily business results data, and the output is feedback information that helps improve the predictive model. This process ensures continuous improvement of the model's accuracy.

[0763] (Application Example 2)

[0764] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0765] In the food and beverage industry, there is a demand for services that respond immediately to individual needs in order to increase customer satisfaction. At the same time, it is necessary to achieve efficient operations while minimizing labor costs and the operating costs of machinery and equipment. However, with conventional methods, it has been difficult to recognize customer emotions in real time and dynamically adjust services based on them. As a result, there has been a challenge in being able to plan appropriate staffing and machinery operations that take emotions into consideration.

[0766] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0767] In this invention, the server includes means for collecting data, means for making sales forecasts based on the data, and means for optimizing work schedules and machine operation plans based on the sales forecasts. This makes it possible to recognize and analyze customer sentiment information in real time and optimize personnel allocation and machine operation accordingly.

[0768] "Means of data collection" refers to a system for acquiring and integrating store sales information, weather information, event information, work preference information, and customer facial and voice information.

[0769] A "means of sales forecasting" refers to a device that uses machine learning algorithms to predict future sales based on collected data.

[0770] "Means for optimizing work schedules and machinery operation plans" refers to a system for planning and adjusting efficient staffing and machinery operation, taking into account predicted sales information and customer sentiment data.

[0771] "Methods for updating predictive models using feedback data" refers to the process of automatically improving a model to enhance the accuracy of sales forecasts by using actual sales data and customer sentiment data.

[0772] "Means for recognizing and analyzing customer emotions through an emotion engine" refers to a device that includes an algorithm for detecting and classifying a customer's emotional state by analyzing image and audio data.

[0773] "A means of adjusting service delivery in real time" refers to a system that dynamically instructs employees to optimize service content based on emotional data acquired from customers.

[0774] "A means of improving service by presenting emotion analysis results to employees using smart devices" refers to a technology that provides information to employees through devices such as smart glasses, enabling them to check the emotional state of customers in real time and respond appropriately.

[0775] This invention is a system for optimizing staffing and the operation of machinery and equipment in restaurants.

[0776] This system consists of a server, smart devices (e.g., smart glasses), and terminals placed within the store. The server is the central hub responsible for collecting data, forecasting sales, optimizing work schedules, and planning the operation of machinery and equipment.

[0777] The server uses multiple data collection methods to gather sales information, weather information, event information, and employee work preference information. It also uses an emotion engine to obtain emotional information from customer facial expressions and voice. Python and OpenCV libraries are used for image recognition, and the Google Cloud Speech-to-Text API is used for voice analysis. Based on this data, a machine learning algorithm is used to predict sales.

[0778] The server then uses predicted sales data and sentiment information to create an optimal work schedule and equipment operation plan. It can adjust employee deployment and machine operation in real time based on the customer's emotional state. For example, employees can monitor customer emotions in real time through smart glasses and modify service as needed.

[0779] Furthermore, after business hours, the server collects actual sales data, the results of the tasks performed, and customer sentiment data, and provides feedback to build increasingly accurate sales forecasting models and operational optimization algorithms.

[0780] As a concrete example, consider a scenario where smart glasses are used to monitor the emotional state of customers visiting a restaurant with their family. If a child shows signs of boredom, a special menu item is suggested to the employee. This allows for concrete implementation of how smart devices are used and how information is processed.

[0781] Examples of prompt statements for a generative AI model are as follows:

[0782] "Please tell me about a system that uses video data captured by smart glasses to determine customer emotions in real time through image recognition and voice analysis, and then notifies employees of the appropriate service."

[0783] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0784] Step 1:

[0785] The server collects sales information, weather information, event information, work preference information, and customer facial expressions and voice data as input. This data is gathered from in-store and external databases using sensors, cameras, microphones, etc. This allows for the integrated accumulation of environmental and customer information.

[0786] Step 2:

[0787] The server uses the collected information as input to perform image recognition and speech analysis. For image recognition, it analyzes customer facial expressions using Python and the OpenCV library, and for speech analysis, it uses the Google Cloud Speech-to-Text API. The data is processed by a machine learning model to identify the customer's emotions. As a result, the customer's emotional state is output.

[0788] Step 3:

[0789] The server makes sales forecasts based on customer emotional states and other environmental data. A machine learning algorithm drives the process of predicting future sales. This predicted data is then used as the result for the next step in the process.

[0790] Step 4:

[0791] The server uses sales forecast results and sentiment data as input to optimize work schedules and machine operation plans. An optimization algorithm is used to output staffing plans that are adapted to each customer's needs and sentiments.

[0792] Step 5:

[0793] The smart device (smart glasses) provides employees with real-time sentiment analysis results and optimized service guidance. Through the device, employees can receive immediate feedback, which they can then use to improve their service delivery.

[0794] Step 6:

[0795] Based on feedback from the server, users will ultimately adjust their work schedules and service content. This includes implementing specific service improvements tailored to anticipated situations.

[0796] Step 7:

[0797] The server collects feedback data obtained after business hours and updates its predictive models and optimization algorithms. This improves the overall accuracy and efficiency of the system.

[0798] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0799] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0800] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0801] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0802] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. In the upper and lower directions of the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Also, the upper side of the concentric circles is where "pleasant" emotions are located, and the lower side is where "unpleasant" emotions are located. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0803] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0804] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0805] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0806] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0807] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0808] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0809] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0810] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0811] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0812] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0813] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0814] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0815] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0816] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0817] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0818] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0819] The following is further disclosed regarding the embodiments described above.

[0820] (Claim 1)

[0821] Means of collecting data,

[0822] A means for making sales forecasts based on the aforementioned data,

[0823] A means for optimizing work shifts and machinery operation plans based on the aforementioned sales forecast,

[0824] A means for outputting the optimized work shift and machine operation plan and enabling adjustments,

[0825] A means of updating the predictive model using feedback data,

[0826] A system that includes this.

[0827] (Claim 2)

[0828] The system according to claim 1, wherein the feedback data is based on actual sales information and the operating status of machinery and equipment.

[0829] (Claim 3)

[0830] The system according to claim 1, wherein the data collection means acquires data including weather information, event information, and work preference information.

[0831] "Example 1"

[0832] (Claim 1)

[0833] A means of collecting commercial transaction information, weather information, event information, and employee request information via a data processing device,

[0834] A means of predicting future sales using machine learning algorithms based on collected information,

[0835] Means for optimizing labor allocation and machinery operation plans based on the aforementioned predictions,

[0836] A means to output an optimized plan and allow administrators to edit it,

[0837] A means of accumulating operational results and learning to improve the accuracy of the prediction algorithm,

[0838] A system that includes this.

[0839] (Claim 2)

[0840] The system according to claim 1, comprising a feedback means that uses actual transaction results and activity data of machinery and equipment.

[0841] (Claim 3)

[0842] The system according to claim 1, characterized in that it acquires weather information, event information, and worker request information in an integrated manner during the data acquisition stage.

[0843] "Application Example 1"

[0844] (Claim 1)

[0845] Means of acquiring data,

[0846] A means for performing revenue forecasting based on the aforementioned data,

[0847] A means for optimizing the work plan and the machine operation plan based on the aforementioned revenue forecast,

[0848] Means for visualizing and modifying the optimized work plan and machine operation plan,

[0849] A method for improving predictive models using feedback data,

[0850] Means for viewing and modifying the aforementioned work plan in real time via a mobile device,

[0851] A system that includes this.

[0852] (Claim 2)

[0853] The system according to claim 1, wherein the feedback data is based on actual revenue information and the operating status of the machinery and equipment.

[0854] (Claim 3)

[0855] The system according to claim 1, wherein the data acquisition means acquires data including natural information, event information, and work preference information.

[0856] "Example 2 of combining an emotion engine"

[0857] (Claim 1)

[0858] Means of collecting data,

[0859] A means for making sales forecasts based on the aforementioned data,

[0860] A means for optimizing work shifts and machinery operation plans based on the aforementioned sales forecast,

[0861] A means for outputting the optimized work shift and machine operation plan and enabling adjustments,

[0862] A means of updating the predictive model using feedback data,

[0863] A means for acquiring user sentiment information and utilizing it to optimize the aforementioned sales forecast and operational plan,

[0864] A means for adjusting the operation of a machine in real time based on the aforementioned emotional information,

[0865] A system that includes this.

[0866] (Claim 2)

[0867] The system according to claim 1, wherein the feedback data is based on actual sales information, the operating status of machinery and equipment, and acquired sentiment information.

[0868] (Claim 3)

[0869] The system according to claim 1, wherein the data collection means acquires data including weather information, event information, work preference information, and sentiment data.

[0870] "Application example 2 when combining with an emotional engine"

[0871] (Claim 1)

[0872] Means of collecting data,

[0873] A means for making sales forecasts based on the aforementioned data,

[0874] A means for optimizing work schedules and machine operation plans based on the aforementioned sales forecast,

[0875] A means for outputting the optimized work schedule and machine operation plan and enabling adjustments,

[0876] A means of updating the predictive model using feedback data,

[0877] A means of recognizing and analyzing customer emotions through an emotion engine,

[0878] A means of adjusting service delivery in real time based on customer sentiment information,

[0879] A means of improving service by presenting employees with emotion analysis results using smart devices,

[0880] A system that includes this.

[0881] (Claim 2)

[0882] The system according to claim 1, wherein the feedback data is based on actual sales information, the operating status of machinery and equipment, and customer sentiment data.

[0883] (Claim 3)

[0884] The system according to claim 1, wherein the data collection means acquires weather information, event information, work preference information, and customer facial expressions and voice information. [Explanation of Symbols]

[0885] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means of acquiring data, A means for performing revenue forecasting based on the aforementioned data, A means for optimizing the work plan and the machine operation plan based on the aforementioned revenue forecast, Means for visualizing and modifying the optimized work plan and machine operation plan, A method for improving predictive models using feedback data, Means for viewing and modifying the aforementioned work plan in real time via a mobile device, A system that includes this.

2. The system according to claim 1, wherein the feedback data is based on actual revenue information and the operating status of the machinery and equipment.

3. The system according to claim 1, wherein the data acquisition means acquires data including natural information, event information, and work preference information.

Images