Information processing device and information processing system

Abstract

To provide an information processing apparatus and system for efficiently collecting evaluations of a commodity to recommend the commodity based on results of the evaluation.SOLUTION: In a recommendation system configured by connecting an order terminal, an external data acquisition apparatus, a server device and a store clerk terminal so as to communicate with each other, the server device includes: an attribute determination unit which determines an attribute of a customer from image data obtained by imaging the customer who orders a commodity and associates the attribute with identification information of the commodity ordered by the customer; an evaluation score calculation unit which detects a speech portion related to an evaluation of the ordered commodity from voice data generated by recording the content of speech of the customer, and calculates an evaluation score by quantifying the evaluation of the commodity on the basis of the content of the detected speech portion; a menu evaluation list generation unit which generates a list in which the evaluation score is associated with a set of the identification and the attribute associated with the attribute determination unit; and a menu update unit which generates, for each attribute, menu information for displaying a menu screen for supporting ordering a commodity, on the basis of a relationship between the attribute and the evaluation score in the list.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus and an information processing system.

Background Art

[0002] Conventionally, in stores such as restaurants, the intention to purchase is improved by presenting seasonal menus and recommended menus. Conventionally, a technique has also been proposed for estimating the age group and gender of a customer from a face image of the customer captured and presenting a recommended menu according to the estimation result.

[0003] By the way, in stores such as restaurants, recommended menus are selected based on customers' evaluations of products (food and beverages). Conventionally, evaluations of products have been collected via questionnaire sheets provided in the store. In recent years, evaluations of products have also been collected using SNS (Social Networking Service) or the like.

[0004] With the questionnaire method, it is possible to obtain accurate evaluations, but there is a problem that the number of customers who fill out the questionnaire is limited, and there are many customers who do not use it unless added value is provided. Also, with the method using SNS or the like, there is a problem that the implementation hurdle is high because it is necessary to use the SNS platform. In addition, since there is a possibility that users who do not actually use the store may input evaluations, there is also a problem in terms of reliability.

Summary of the Invention

Problems to be Solved by the Invention

[0005] The problem to be solved by the present invention is to provide an information processing apparatus and an information processing system capable of efficiently collecting evaluations of products and making recommendations for products based on the evaluation results.

Means for Solving the Problems

[0006] The information processing device of this embodiment includes: a first determination means for determining the attributes of a customer from image data of a customer who is taking an order for a product; an association means for associating identification information of a product ordered by the customer with the attributes determined by the first determination means; a detection means for detecting a portion of speech related to the evaluation of the ordered product from audio data recording the content of the customer's speech; a calculation means for calculating an evaluation score that quantifies the evaluation of the product based on the content of the speech portion detected by the detection means; a first generation means for generating a list that associates the set of identification information and attributes associated by the association means with the evaluation score calculated by the calculation means for the product corresponding to the identification information; and a second generation means for generating menu information for each attribute that can display a menu screen to support ordering the product based on the relationship between the attributes included in the list and the evaluation score for each piece of identification information. Furthermore, the detection means detects the utterance position in the speech data where a keyword representing the ordered product appears, and the calculation means calculates an evaluation score for the product based on the meaning of evaluation words that appear before and after the utterance position of the keyword. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a schematic diagram of a recommendation system according to an embodiment of this system. [Figure 2] Figure 2 shows an example of the hardware configuration of an order terminal according to the embodiment. [Figure 3] Figure 3 shows an example of the hardware configuration of a server device according to this embodiment. [Figure 4] Figure 4 shows an example of the data structure of a product master according to an embodiment. [Figure 5] Figure 5 shows an example of the data structure of the first configuration file according to the embodiment. [Figure 6] Figure 6 shows an example of the data structure of the second configuration file according to the embodiment. [Figure 7] Figure 7 shows an example of the data structure of an order management file according to the embodiment. [Figure 8] Figure 8 shows an example of the data structure of a menu evaluation list file according to the embodiment. [Figure 9]Figure 9 shows an example of the functional configuration of an order terminal according to the embodiment. [Figure 10] Figure 10 shows an example of the functional configuration of a server device according to this embodiment. [Figure 11] Figure 11 shows an example of a menu evaluation list screen according to the embodiment. [Figure 12] Figure 12 is a flowchart showing an example of order processing performed during the learning phase in the order terminal and server device of the embodiment. [Figure 13] Figure 13 is a flowchart showing an example of the evaluation point registration process performed during the learning phase in the order terminal and server device of the embodiment. [Figure 14] Figure 14 is a flowchart showing an example of the menu evaluation list generation process performed during the learning phase in the order terminal and server device of the embodiment. [Figure 15] Figure 15 is a flowchart showing an example of the process related to the display of a menu screen performed at the product recommendation stage in the order terminal and server device of the embodiment. [Figure 16] Figure 16 is a flowchart showing an example of the reference reception process performed by the server device of the embodiment. [Modes for carrying out the invention]

[0008] The following describes the information processing apparatus and information processing system of the embodiments with reference to the drawings. The embodiments described below describe an example in which the information processing apparatus and information processing system are applied to a restaurant. However, the present invention is not limited to the embodiments described below.

[0009] Figure 1 shows an overview of a recommendation system. Recommendation system 1 is an example of an information processing system. Recommendation system 1 is installed, for example, in a restaurant and collects customer evaluations of products (food and beverages) and recommends products based on those evaluations.

[0010] Recommendation system 1 comprises an order terminal 10, an external data acquisition device 20, a server device 30, and a staff terminal 40. The order terminal 10, the external data acquisition device 20, the server device 30, and the staff terminal 40 are connected to each other via a network N such as a LAN (Local Area Network).

[0011] The order terminal 10 is a self-ordering terminal device operated by customers. The order terminal 10 is installed, for example, at each table T in the store and is implemented using a terminal device such as a tablet.

[0012] The order terminal 10 displays a screen (hereinafter also referred to as the order screen) on the display unit 114 that can assist in ordering products based on menu information provided by the server device 30. When a product to be ordered is selected based on the order screen, the order terminal 10 places an order for the product by transmitting a product ID that can identify that product, along with a table ID that can identify the table T on which the terminal is located, to the server device 30. The order terminal 10 is also equipped with an imaging unit 116 and a sound collection unit 117, and transmits data acquired by these devices to the server device 30.

[0013] The external data acquisition device 20 includes imaging devices 21 and sound collection devices 22 installed within the store. The imaging device 21 is, for example, a surveillance camera, and transmits image data of the store's interior to the server device 30. The imaging device 21 is, for example, provided for each table T, and is positioned to capture images of customers using the table T. The imaging device 21 transmits image data of customers, etc., to the server device 30 along with the table ID of the table T it is responsible for.

[0014] The sound collection device 22 is, for example, a microphone, which collects the sound in the store and outputs the collected sound data to the server device 30. The sound collection device 22 is provided for each table T, for example, in pairs with the imaging device 21, and collects the sound of the customers using the table T. The sound collection device 22 transmits the sound data obtained by recording the customers' voices to the server device 30 together with the table ID of the table T for which the device is responsible.

[0015] It is preferable to attach a time stamp indicating the acquisition date and time to the data acquired by the external data acquisition device 20 (imaging device 21, sound collection device 22). Also, it is preferable to attach a time stamp indicating the acquisition date and time to the data acquired by the imaging unit 116 and the sound collection unit 117 of the order terminal 10. Thereby, it becomes possible to synchronize various data acquired by the imaging unit 116, the sound collection unit 117, and the external data acquisition device 20, for example, the actions and voices (utterance contents) of the customers performed at the same timing.

[0016] In the present embodiment, the imaging unit 116 and the sound collection unit 117 of the order terminal 10 are used, for example, for the purpose of acquiring the appearance and voice of the customer operating the order terminal 10, whereas the imaging device 21 and the sound collection device 22 are used for the purpose of acquiring the appearance and voice of one or more customers seated at the table T. Note that a configuration not using the external data acquisition device 20 may also be adopted.

[0017] The server device 30 is an example of an information processing device. The server device 30 is, for example, a store server provided in the store. The server device 30 provides each of the order terminals 10 with menu information indicating the menu of the products sold in the store. Also, the server device 30 receives the product ID of the product ordered by the customer from the order terminal 10, and stores and manages it as order information. The server device 30 outputs the product name and the like of the ordered product ID to an output device such as a printing device or a display device provided in the kitchen or the like, thereby notifying the store clerk of the product ordered by the customer.

[0018] Furthermore, the server device 30 collects various data acquired by the order terminal 10 and the external data acquisition device 20. By analyzing the collected data, the server device 30 estimates customer attributes and aggregates customer evaluations of ordered products. Based on the evaluation results for each menu item, the server device 30 generates customized menu information for each customer attribute and provides the generated menu information to the order terminal 10.

[0019] The employee terminal 40 is a terminal device operated by an employee. The employee terminal 40 may be a terminal device similar to the order terminal 10, or it may be a POS terminal, etc. The employee terminal 40 can access the server device 30 and check and display the data managed by the server device 30.

[0020] Next, we will describe the configuration of the main components of the recommendation system 1 mentioned above.

[0021] First, the hardware configuration of the order terminal 10 will be explained with reference to Figure 2. Figure 2 is a diagram showing an example of the hardware configuration of the order terminal 10. As shown in Figure 2, the order terminal 10 is equipped with a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, and a RAM (Random Access Memory) 113.

[0022] The CPU 111 is an example of a processor that comprehensively controls each part of the order terminal 10. The ROM 112 stores various programs. The RAM 113 is a workspace for displaying programs and various data.

[0023] The CPU 111, ROM 112, and RAM 113 are connected via a bus or the like to form a control unit 110 of the computer configuration. In the control unit 110, the CPU 111 operates according to the program stored in the memory unit 119 and loaded into the RAM 113, thereby executing various processes.

[0024] The order terminal 10 also includes a display unit 114, an operation unit 115, an imaging unit 116, a sound collection unit 117, a communication unit 118, and a storage unit 119, etc.

[0025] The display unit 114 is a display device such as an LCD (Liquid Crystal Display). The display unit 114 displays various information under the control of the CPU 111. For example, the display unit 114 displays a menu screen to assist in ordering products.

[0026] The operation unit 115 is an input device such as a keyboard or pointing device. The operation unit 115 outputs the operation content received from the customer to the CPU 111. The operation unit 115 may also be a touch panel provided on the display screen of the display unit 114.

[0027] The imaging unit 116 is a digital camera having an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). The imaging unit 116 is positioned to capture the face of a customer operating their own order terminal 10. The imaging unit 116 outputs the image data obtained by the imaging to the CPU 111. The captured image data may be a video or still images captured at predetermined time intervals.

[0028] The sound pickup unit 117 is a sound pickup device such as a microphone. The sound pickup unit 117 picks up sound from around the order terminal 10 and outputs the sound data obtained from the sound pickup to the CPU 111. For example, under the control of the control unit 110, the sound pickup unit 117 picks up the voice of a customer who has placed an order for a product and is talking about that product.

[0029] The communication unit 118 is a communication interface for communicating with each device connected to the network N. Under the control of the CPU 111, the communication unit 118 sends and receives various data with the server device 30 and other devices connected to the network N. When the communication unit 118 sends various data and information to the server device 30, it will also send the table ID of the table T on which its order terminal 10 is located.

[0030] The memory unit 119 is an auxiliary storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). The memory unit 119 stores various programs and setting information that the CPU 111 can execute. The memory unit 119 also stores menu information M, which shows the menu of products sold in the store.

[0031] Menu information M includes, for example, a display layout related to the display of the menu screen, information about the products to be assigned as menus to the display layout (product ID, product name, product image, etc.), and information that defines the order in which the products are displayed and the size in which they are displayed.

[0032] The control unit 110 generates a menu screen by assigning the product name, product image, etc. of the corresponding product to the display layout according to the display order and size specified in the menu information M, and displays it on the display unit 114. On the menu screen, it is possible to select the displayed product, and the control unit 110 accepts the product ID of the selected product as the item to be ordered. The control unit 110 then sends the item ID of the item to be ordered to the server device 30 to place the order for the product.

[0033] Furthermore, the hardware configuration of the employee terminal 40 may be the same as that of the order terminal 10. For example, the hardware configuration of the employee terminal 40 may be the same as that of the order terminal 10, but with the imaging unit 116 and the sound collection unit 117 removed.

[0034] Figure 3 shows an example of the hardware configuration of the server device 30. As shown in Figure 3, the server device 30 includes a CPU 311, a ROM 312, and a RAM 313.

[0035] The CPU 311 is an example of a processor that comprehensively controls all parts of the server device 30. The ROM 312 stores various programs. The RAM 313 is a workspace for displaying programs and various data.

[0036] The CPU 311, ROM 312, and RAM 313 are connected via a bus or the like to form a control unit 310 of the computer configuration. In the control unit 310, the CPU 311 operates according to the program stored in the memory unit 317 and loaded into the RAM 313, thereby executing various processes.

[0037] Furthermore, the server device 30 includes a display unit 314, an operation unit 315, a communication unit 316, and a storage unit 317, etc.

[0038] The display unit 314 is a display device such as an LCD. The display unit 314 displays various information under the control of the CPU 311. The operation unit 315 is an input device such as a keyboard or a pointing device. The operation unit 315 outputs the operation content received from the operator to the CPU 311. The operation unit 315 may also be a touch panel provided on the display screen of the display unit 314.

[0039] The communication unit 316 is a communication interface for communicating with each device connected to the network N. Under the control of the CPU 311, the communication unit 316 sends and receives various types of data with the order terminal 10, external data acquisition device 20, and employee terminal 40, etc., which are connected to the network N.

[0040] The memory unit 317 is an auxiliary storage device such as an HDD or SSD. The memory unit 317 stores various programs and setting information that the CPU 311 can execute. The memory unit 317 also stores the product master 3171, the first analysis model 3172, the second analysis model 3173, the evaluation score setting file 3174, the order management file 3175, the menu evaluation list file 3176, the menu creation source file 3177, and so on.

[0041] The product master 3171 is a master file that stores information about products sold in stores. For example, the product master 3171 has the data structure shown in Figure 4. Figure 4 is a diagram showing an example of the data structure of the product master 3171. As shown in Figure 4, the product master 3171 stores product information such as the product name, price, and product image of the product corresponding to the product ID, associated with the product ID.

[0042] Furthermore, the product master 3171 stores keywords for the product corresponding to the product ID, in association with that product ID. Here, the keywords store various words used to represent the product, such as the product name, abbreviation, or common name. For example, if the product is "carbonara," the keywords will store words such as "carbonara," "pasta," and "noodles." Also, for example, if the product is "shortcake," the keywords will store words such as "cake," "strawberry," and "cream."

[0043] The first analysis model 3172 is a data analysis model capable of deriving customer attributes from image data of customers. For example, the first analysis model 3172 may be a learning model that utilizes known facial recognition technology, or it may be an algorithm that does not use machine learning. In the former case, for example, the first analysis model 3172 may be a learning model generated by machine learning such as deep learning, using training data consisting of a pair of image data of faces and figures of various people and ground truth data indicating the attributes of those people. The first analysis model 3172 takes image data of customers captured by the order terminal 10 or external data acquisition device 20 as input and outputs customer attributes estimated based on said image data. For example, the first analysis model 3172 outputs the customer's age (age group) and gender as customer attributes.

[0044] The second analysis model 3173 is a data analysis model capable of analyzing the content of speech uttered by a customer from audio data recorded from the customer's voice. For example, the second analysis model 3173 may be a learning model using known speech recognition technology, or it may be an algorithm that does not use machine learning. In the former case, for example, the second analysis model 3173 may be a learning model generated by machine learning such as deep learning, using training data consisting of pairs of audio data recorded from the voices of various people and correct data showing the content of their speech. The second analysis model 3173 takes customer audio data collected by the order terminal 10 or external data acquisition device 20 as input and outputs the content of the customer's speech estimated based on the audio data.

[0045] Furthermore, the second analysis model 3173 may be a model that simultaneously identifies the person who spoke and estimates the content of their speech from image data of multiple people and their voice data, using known facial recognition technology or the like. In this case, the second analysis model 3173 extracts the facial region of a person from the image data and identifies the person who is speaking based on changes in the movement of their mouth. The second analysis model 3173 also estimates the content of the person's speech from the voice data during the period in which the mouth movement was detected. The second analysis model 3173 then outputs the person identification result and the content of their speech. In addition, the second analysis model 3173 may read the emotions of the identified person from changes in their facial expression and output the reading result along with the person identification result and the content of their speech.

[0046] The evaluation score setting file 3174 is a file that contains information for deriving product evaluations from the words included in the utterance. The evaluation score setting file 3174 consists of one or more files and is used when analyzing the utterance.

[0047] For example, the evaluation score setting file 3174 has a first setting file 31741 that associates a basic evaluation score, which quantifies the degree of evaluation based on each word used when evaluating a product (hereinafter also referred to as an evaluation word). The evaluation score setting file 3174 also has a second setting file 31742 that associates modifiers that express degree and are used in conjunction with evaluation words with weight coefficients corresponding to those modifiers.

[0048] Figure 5 shows an example of the data structure of the first configuration file 31741. As shown in Figure 5, the first configuration file 31741 stores evaluation words and base evaluation scores in association. Multiple types of evaluation words are stored to cover a wide range of words, from those that mean positive evaluations such as "delicious," "tasty," "bad," and "not tasty," to those that mean negative evaluations. In addition, base evaluation scores are set that correspond to the meaning of the evaluation words. Figure 5 shows an example where a base evaluation score of "+1" is assigned to positive evaluations and a base evaluation score of "-1" is assigned to negative evaluations. Note that the base evaluation scores are not limited to these. For example, different base evaluation scores may be assigned to each evaluation word.

[0049] Figure 6 shows an example of the data structure of the second configuration file 31742. As shown in Figure 6, the second configuration file 31742 stores the relationship between adverbs of degree and weights (coefficients). Multiple types of adverbs expressing degree are stored in the adverbs of degree section, such as "very," "extremely," "very," and "not at all." Note that the adverbs of degree section may also store other modifiers expressing degree (adjectival nouns, etc.). The weights are set to values ​​of weight coefficients corresponding to the degree expressed by the adverbs of degree. Figure 5 shows an example where the weight increases as the degree expressed by the adverb of degree increases. The values ​​set for the weights are multiplied by the base evaluation score of the first configuration file 31741 when calculating the evaluation score, which will be described later.

[0050] The order management file 3175 is a file for storing and managing order information. For example, the order management file 3175 has the data structure shown in Figure 7. Figure 7 is a diagram showing an example of the data structure of the order management file 3175. As shown in Figure 7, the order management file 3175 stores the product ID of the ordered product, associating each of the tables T with an identifiable table ID. Here, the table ID and product ID are order information notified from the order terminal 10.

[0051] Furthermore, the order management file 3175 stores the attributes of the customer who ordered the product corresponding to the product ID, in association with the product ID. Here, the customer attributes stored are those derived using the first analysis model 3172. Figure 7 shows an example where age (age group) and gender are stored as customer attributes. The order management file 3175 also stores the customer's rating score for the product with the product ID, in association with the product ID. Here, the rating score stored is the rating score derived using the second analysis model 3173 and the rating score setting file 3174.

[0052] The menu evaluation list file 3176 is a file for storing and managing the results of product evaluations aggregated by customer attributes. For example, the menu evaluation list file 3176 has the data structure shown in Figure 8. Figure 8 is a diagram showing an example of the data structure of the menu evaluation list file 3176.

[0053] As shown in Figure 8, the menu evaluation list file 3176 stores the product name of the product corresponding to each product ID. The menu evaluation list file 3176 also stores the overall evaluation score for each product ID for each customer attribute. Here, the overall evaluation score is the sum of the evaluation scores of customers with the same attributes. For example, Figure 8 shows the overall evaluation scores for the products "Carbonara," "Shortcake," "Salad," and "Hamburg Steak" for customers who are in their "30s" and are "female."

[0054] The menu source file 3177 is a group of data files used to generate menu information. For example, the menu source file 3177 includes basic information such as product images and descriptions of products sold in the store, associated with the product IDs of those products. The menu source file 3177 also includes layout information that constitutes the display layout and icon images related to the display of the menu screen. Furthermore, the menu source file 3177 includes additional information that defines the products to be assigned as menu items in the display layout and the display order of those products.

[0055] In the process described later, the server device 30 generates customized menu information based on the overall evaluation score of the menu evaluation list file 3176, and sends it to the order terminal 10.

[0056] Note that the hardware configuration of the server device 30 is not limited to the example in Figure 3. For example, a separate storage device, such as a database server, may be configured to store any or all of the product master 3171 to menu creation source files 3177. In this case, the server device 30 can access the separate storage device to refer to or edit any or all of the product master 3171 to menu creation source files 3177 stored in that storage device.

[0057] Next, the functional configuration of the order terminal 10 and the server device 30 will be described.

[0058] Figure 9 shows an example of the functional configuration of the order terminal 10. As shown in Figure 9, the order terminal 10 is equipped with a menu display unit 151, an order reception unit 152, and a data acquisition unit 153 as its functional configuration. Specifically, the control unit 110 (CPU 111) of the order terminal 10 realizes the above-described functional configuration by executing a program stored in the ROM 112 or the storage unit 119.

[0059] In this embodiment, the above-described functional configuration is a software configuration realized through the cooperation of the processor and program of the order terminal 10, but it is not limited to this, and a hardware configuration in which some or all of the functional configuration is realized by a dedicated circuit or the like is also possible.

[0060] The menu display unit 151 is an example of a display means. The menu display unit 151 acquires menu information transmitted from the server device 30 via the communication unit 118. The menu display unit 151 also stores the acquired menu information in the storage unit 119 and, at a predetermined timing, displays a menu screen based on that menu information on the display unit 114.

[0061] For example, when the menu display unit 151 detects that an operation has started on its own order terminal 10 via the operation unit 115, it displays a menu screen on the display unit 114. Alternatively, for example, when the menu display unit 151 receives notification of customer attributes from the server device 30, it uses menu information corresponding to those attributes to display a menu screen based on that menu information on the display unit 114. In the latter example, the menu display unit 151 may wait for a predetermined time after detecting the start of an operation for notification of customer attributes from the server device 30, and if no notification is received, it may display a menu screen using the default set menu information.

[0062] The order receiving unit 152 accepts the selection of products to be ordered via the operation unit 115. Specifically, the order receiving unit 152 accepts the selection of products based on the menu screen displayed on the display unit 114. Once a product is selected, the order receiving unit 152 transmits order information, including the product ID of that product, to the server device 30 via the communication unit 118. Furthermore, when the order receiving unit 152 receives the order confirmation operation, it transmits information instructing the confirmation of the order to the server device 30 via the communication unit 118.

[0063] The data acquisition unit 153 acquires data from the imaging unit 116 and the sound collection unit 117. Specifically, the data acquisition unit 153 controls the operation of the imaging unit 116 and the sound collection unit 117 and acquires the data obtained by the imaging unit 116 and the sound collection unit 117 during their operation. The data acquisition unit 153 transmits the acquired data to the server device 30 via the communication unit 118.

[0064] For example, when the data acquisition unit 153 detects that an operation of its own order terminal 10 has started via the operation unit 115, it activates the imaging unit 116 and starts acquiring imaging data. For example, when the data acquisition unit 153 receives an order confirmation operation, it stops the imaging unit 116 and starts the sound pickup unit 117 to begin acquiring audio data. Then, when an operation to instruct the payment of goods is performed, the data acquisition unit 153 stops the sound pickup unit 117. The data acquisition unit 153 may also operate the imaging unit 116 together with the sound pickup unit 117 after the order confirmation operation.

[0065] Figure 10 shows an example of the functional configuration of the server device 30. As shown in Figure 10, the server device 30 includes a data acquisition unit 351, an attribute determination unit 352, an order management unit 353, an evaluation score calculation unit 354, a menu evaluation list generation unit 355, a menu update unit 356, and a reference request reception unit 357. Specifically, the control unit 310 (CPU 311) of the server device 30 realizes the above-described functional configuration by executing a program stored in the ROM 312 or the storage unit 317.

[0066] In this embodiment, the above-described functional configuration is a software configuration realized through the cooperation of the processor and program of the server device 30, but it is not limited to this, and a hardware configuration in which some or all of the functional configuration is realized by dedicated circuits, etc., is also possible.

[0067] The data acquisition unit 351 collects imaging data and audio data transmitted from the order terminal 10 and the external data acquisition device 20 via the communication unit 316. The collected imaging data and audio data may be processed in real time by the attribute determination unit 352 and the evaluation score calculation unit 354, or they may be temporarily stored in the storage unit 317, etc. In the latter case, the collected data may be stored in chronological order for each data containing the same table ID.

[0068] The attribute determination unit 352 is an example of the first determination means and association means. The attribute determination unit 352 determines the attributes of the customer operating the order terminal 10 based on the customer's face or appearance included in the image data of the order terminal 10 collected by the data collection unit 351. Specifically, the attribute determination unit 352 inputs the collected image data into the first analysis model 3172, and determines the attributes output by the first analysis model 3172 as the attributes of the customer operating the order terminal 10. Furthermore, in the recommendation stage described later, after the menu evaluation list file 3176 has been generated, the attribute determination unit 352 determines the customer's attributes based on the image data and notifies the order terminal 10 that transmitted the image data.

[0069] The order management unit 353 receives order information transmitted from the order terminal 10. The order management unit 353 also stores and manages the received order information in the order management file 3175.

[0070] Specifically, when the order management unit 353 receives order information from the order terminal 10, it stores the table ID and product ID included in the order information in the order management file 3175. In addition, the order management unit 353 works in cooperation with the attribute determination unit 352 to store the customer attributes derived at the time of product ordering in the order management file 3175, associating them with the product ID of the product in question.

[0071] Furthermore, when the order management unit 353 receives an order confirmation notification from the order terminal 10, it outputs the product name and table ID of the ordered product to the output device and changes the order flag to "ordered". Specifically, the order management unit 353 searches the order management file 3175 for order information where the order flag associated with the table ID is "not ordered", based on the table ID included in the information instructing order confirmation. The order management unit 353 reads the product name corresponding to the product ID included in the retrieved order information from the product master 3171 and outputs it to the output device along with the table ID, etc. Then, the order management unit 353 changes the order flag of the order information that has been output to the output device to "ordered".

[0072] The evaluation score calculation unit 354 is an example of a detection means, a calculation means, a first generation means, and a second determination means. The evaluation score calculation unit 354 detects the utterance portion related to the evaluation of the ordered product from the voice data of the order terminal 10 collected by the data collection unit 351, and calculates an evaluation score based on the content of the detected utterance portion.

[0073] Specifically, the evaluation score calculation unit 354 inputs the audio data to be processed into the second analysis model 3173, and obtains the output of the second analysis model 3173 as the content of the audio data. Next, based on the table ID transmitted along with the audio data to be processed, the evaluation score calculation unit 354 identifies the product IDs included in the records stored in the order management file 3175 associated with the table IDs, for which evaluation scores have not yet been registered. Next, the evaluation score calculation unit 354 reads the keywords corresponding to the identified product IDs from the product master 3171.

[0074] Next, the evaluation score calculation unit 354 detects the portion of the speech in the voice data that relates to the customer's evaluation of the ordered product, based on the combination of the keyword read from the product master 3171 and the evaluation word defined in the first setting file 31741.

[0075] Specifically, when the evaluation score calculation unit 354 detects a keyword from the utterance, it determines whether an evaluation word exists in the utterance positions before and after the keyword, and if so, reads out the standard evaluation score for that evaluation word. Here, "preceding and succeeding utterance positions" refers to the words or phrases located before and after the base word (keyword). The number of words or phrases to be included in the preceding and succeeding range can be set arbitrarily.

[0076] For example, suppose a customer orders "carbonara," and while the payment for that order is not yet completed, audio data recorded at the table where the customer is seated contains the utterance, "This pasta is very delicious." In this example, the evaluation score calculation unit 354 reads the keywords "carbonara," "pasta," and "noodles" corresponding to the product "carbonara" from the product master 3171. Next, the evaluation score calculation unit 354 detects the keywords from the utterance content of the audio data. In the above example, since the utterance contains the keyword "pasta," the evaluation score calculation unit 354 detects the position of the utterance "pasta."

[0077] Next, the evaluation score calculation unit 354 attempts to detect an evaluation word defined in the first configuration file 31741 from before and after the utterance position of the keyword. In the example sentence above, the evaluation word "delicious" is present after the keyword "pasta," so the evaluation score calculation unit 354 reads the base evaluation score "+1" corresponding to that evaluation word from the first configuration file 31741.

[0078] Furthermore, when the evaluation score calculation unit 354 detects an evaluation word, it determines whether or not a degree adverb defined in the second configuration file 31742 exists before or after the utterance position of that evaluation word. If a degree adverb exists, the evaluation score calculation unit 354 calculates an evaluation score by multiplying the reference evaluation score by the weight corresponding to the degree adverb. If no degree adverb exists, the evaluation score calculation unit 354 calculates an evaluation score based on the value of the reference evaluation score.

[0079] For example, in the above example sentence, the adverb of degree "very" is present before the evaluation word "delicious," so the evaluation score calculation unit 354 determines that an adverb of degree is present and reads the weight "2" corresponding to that adverb of degree from the second configuration file 31742. Then, the evaluation score calculation unit 354 calculates a value "+2" by multiplying the base evaluation score "+1" by the weight "2" as the evaluation score for the product "carbonara."

[0080] The evaluation score calculation unit 354 then registers the calculated evaluation score value to the evaluation score of the corresponding product ID identified from the order management file 3175. In this way, the evaluation score calculation unit 354 associates the product ID and customer attribute pair with the evaluation score related to that product ID. Here, the record that associates the product ID and customer attribute pair with the evaluation score is an example of a list.

[0081] It should be noted that while the above process allows for registering ratings in association with product IDs, these ratings may not reflect the ratings of the customer who actually ordered the product, but rather the ratings of other customers who were seated at the same table as that customer.

[0082] Therefore, the evaluation score calculation unit 354 may use other data collected by the data collection unit 351 to determine whether the utterance portion related to the evaluation of a product detected from the voice data is the utterance portion of a customer who actually ordered the product, and if it is determined to be the utterance portion of a customer who actually ordered the product, it may calculate an evaluation value or assign an evaluation value.

[0083] For example, if multiple customers are seated at the same table, the evaluation score based on the statements of other customers, not the one who ordered the product, may be included in the product's evaluation score. In such cases, the evaluation score calculation unit 354 uses various data related to the table ID of the table to identify which customer's statements are being made and assigns the evaluation score of the customer who actually ordered the product to that product.

[0084] For example, the evaluation score calculation unit 354 identifies a person speaking based on the movements of a person in the image data (e.g., mouth movements) from the data (voice data and image data) obtained from the order terminal 10 or external data acquisition device 20, based on data obtained from devices with the same table ID. The evaluation score calculation unit 354 also estimates the content of the person's speech from the voice data for the period in which the mouth movements were detected. Furthermore, the evaluation score calculation unit 354 works in cooperation with the attribute determination unit 352 to determine the attribute of the person whose mouth movements were detected, and identifies a product ID associated with the identified attribute from among the product IDs stored in association with the table ID and in records for which evaluation scores have not yet been registered. The evaluation score calculation unit 354 then registers the calculated evaluation score value as the evaluation score for the identified product ID. Furthermore, if there is no product ID associated with the identified attribute, the evaluation score calculation unit 354 suppresses the calculation of an evaluation score or suppresses the assignment of an evaluation score. In this case, the evaluation score calculation unit 354 may perform the above processing using the second analysis model 3173, which simultaneously identifies the person who made the utterance and estimates the content of the utterance.

[0085] As another example, the evaluation score calculation unit 354 may determine the type of product served (product ID), the location of the product, and the location of the customer eating the product from the image data obtained from the order terminal 10 or the external data acquisition device 20, and link the product ID to the customer based on the determination result. Specifically, when the evaluation score calculation unit 354 calculates an evaluation score from the customer's speech content, it registers the evaluation score in association with the product ID of the product corresponding to the location where the customer is located. The evaluation score calculation unit 354 may also determine the ID of the served product by comparing the product image of each product stored in the menu creation source file 3177 with the product image shown in the image data. Alternatively, the evaluation score calculation unit 354 may determine the ID of the served product by using a product recognition model that applies known image recognition technology. Furthermore, when identifying the location of each customer, the attributes determined by the attribute determination unit 352 may be used.

[0086] This method allows for the identification of the relationship between a customer and the product they ordered, even when multiple customers with similar attributes are seated at the same table. Furthermore, it allows for the identification of the relationship between a customer and the product they ordered, even when multiple customers are seated at the same table and served similar products (e.g., different types of pasta).

[0087] Furthermore, the method for identifying the relationship between a customer and the product they ordered is not limited to the method described in the example above. For example, if a customer's facial image is registered in advance, the evaluation score calculation unit 354 may use known facial recognition technology to authenticate each customer and identify the relationship between each authenticated individual and the product they ordered.

[0088] Furthermore, the evaluation score calculation unit 354 may selectively execute any one of the methods for registering evaluation scores described above, or it may execute them in combination. For example, if the evaluation score calculation unit 354 can detect only one customer from the image data of the same table, it may register the evaluation score using only the audio data. Alternatively, if the evaluation score calculation unit 354 detects multiple customers from the image data of the same table, it may register the evaluation score using a combination of the image data and the audio data.

[0089] The menu evaluation list generation unit 355, as an example of the first generation means, generates a menu evaluation list calculated based on the information registered in the order management file 3175, with each customer's overall evaluation score for each product calculated according to the customer's attributes. The menu evaluation list generation unit 355 then registers the generated menu evaluation list in the menu evaluation list file 3176.

[0090] Specifically, for each attribute type registered in the order management file 3175, i.e., the combination of age and gender, the evaluation score for the products ordered by customers of that type is added up for each product ID to calculate the overall evaluation score for each menu item. Then, based on the calculation results, the order management file 3175 generates a menu evaluation list associating product ID, product name, customer attributes, and overall evaluation score, and registers it in the menu evaluation list file 3176 (see Figure 8) or updates an existing menu evaluation list. Here, the menu evaluation list is just an example.

[0091] Furthermore, the timing at which the menu evaluation list generation unit 355 generates the menu evaluation list can be arbitrarily set, for example, once a week or once a month.

[0092] The menu update unit 356 is an example of a second generation means. The menu update unit 356 generates menu information to be provided to the order terminal 10 using the product master 3171 and the menu source file 3177. Specifically, the menu update unit 356 reads the product name, price, product image, description, etc., related to the product ID of the menu items from the basic information of the product master 3171 and the menu source file 3177, and generates menu information that forms the basis of the menu screen by assigning this information for each product to the display layout.

[0093] Furthermore, in the recommendation stage described later, the menu update unit 356 generates customized menu information for each customer attribute, based on the overall product evaluation score for each customer attribute registered in the menu evaluation list file 3176, which includes products to be assigned to the display layout and the order in which they are displayed.

[0094] For example, the menu update unit 356 prioritizes the display of items with higher overall ratings by customizing their display order and allocating more space to them. The menu update unit 356 then provides menu information generated for each customer attribute to each order terminal 10. As a result, the menu information reflects the preferences of each customer attribute. Therefore, by displaying the menu screen using this menu information, popular items (menus) can be recommended for each attribute.

[0095] As another example, the menu update unit 356 may prioritize the display order of products with an overall rating of zero or unrated products for the target attribute, or allocate a larger display space to them. This allows the menu information to proactively promote products that were not previously noticed by customers with specific attributes. Therefore, by displaying the menu screen using this menu information, it is possible to recommend hidden products (menus) that were not previously noticed by each attribute.

[0096] The method of providing menu information to the order terminal 10 is not particularly limited. For example, the menu update unit 356 may distribute the information to each of the order terminals 10 simultaneously when the store opens, so that the menu information M is stored in the memory unit 119 of the order terminal 10. Alternatively, the menu update unit 356 may transmit menu information to the order terminal 10 each time a customer starts an operation on that terminal. In the latter case, the menu update unit 356 may transmit menu information corresponding to the customer's attributes to the originating order terminal 10 based on the result of determining the customer's attributes based on the image data transmitted from the order terminal 10.

[0097] The reference request receiving unit 357, in response to a reference request from an external device, provides the requesting external device with a screen displaying the status of the menu evaluation list file 3176. For example, when the reference request receiving unit 357 receives a reference request from the employee terminal 40, it provides the employee terminal 40 with the menu evaluation list screen shown in Figure 11, and displays it on the employee terminal 40's display.

[0098] Here, Figure 11 shows an example of a menu rating list screen. As shown in Figure 11, the menu rating list screen has an operator Ba that can specify gender and an operator Bb that can specify age as customer attributes that serve as search criteria. The menu rating list screen also has a display button Bc that instructs the display of menu rating lists that match the display criteria.

[0099] The operator (store clerk) operating the menu evaluation list screen specifies the desired attributes by operating the operators Ba and Bb, and then operates the display button Bc. When the reference request receiving unit 357 receives the operation of the display button Bc via the menu evaluation list screen, it searches the menu evaluation list file 3176 for the menu evaluation list corresponding to the attributes specified by operators Ba and Bb, and displays the search results in the display area A.

[0100] Figure 11 shows an example of a menu evaluation list when the customer's attributes are specified as gender "female" and age "21-30". This allows the operator, such as a store employee operating the employee terminal 40, to easily check the customer's evaluation of each product for each customer attribute by referring to the menu evaluation list screen.

[0101] Note that the menu evaluation list screen is not limited to the example shown in Figure 11. For example, the menu evaluation list screen may use products as search criteria and display the overall evaluation value for the specified product for each customer attribute.

[0102] The following describes examples of the operation of the order terminal 10 and server device 30 as described above. First, referring to Figures 12 to 14, we will describe examples of the operation of the order terminal 10 and server device 30 during the stage related to the generation of the menu evaluation list (hereinafter also referred to as the learning stage).

[0103] Figure 12 is a flowchart illustrating an example of order processing performed during the learning phase in the order terminal 10 and the server device 30. First, the control unit 110 of the order terminal 10 waits until operation of its own order terminal 10 is initiated (step S11; No). Once operation is initiated (step S11; Yes), the data acquisition unit 153 starts imaging by operating the imaging unit 116 (step S12). Next, the data acquisition unit 153 transmits the imaging data obtained by imaging by the imaging unit 116 to the server device 30 (step S13). Transmission of imaging data to the server device 30 continues while imaging is being performed by the imaging unit 116.

[0104] Meanwhile, in the server device 30, the data acquisition unit 351 collects various data transmitted from the order terminal 10 and the external data acquisition device 20. When the attribute determination unit 352 receives the imaging data transmitted from the order terminal 10 (step S21), it inputs this imaging data into the first analysis model 3172 to determine the attributes of the customer operating the order terminal 10 (step S22). The attribute determination unit 352 continues to perform attribute determination based on the imaging data while the imaging data is being transmitted from the order terminal 10.

[0105] Furthermore, in the order terminal 10, the menu display unit 151 displays a menu screen based on the menu information M on the display unit 114 in response to the start of operation of the order terminal 10 (step S14). The menu information M is assumed to display a default menu screen in which each product is assigned in a predetermined display order that does not depend on the customer's attributes.

[0106] Next, the order receiving unit 152 determines whether or not the product to be ordered has been selected based on the menu screen (step S15). If the product selection is not accepted (step S15; No), the order receiving unit 152 proceeds to step S17. If the product selection is accepted (step S15; Yes), the order receiving unit 152 sends order information including the product ID of the selected product to the server device 30 (step S16) and proceeds to step S17.

[0107] Meanwhile, in the server device 30, the order management unit 353 is waiting to receive order information (step S23; No → step S25; No). When the order management unit 353 receives order information (step S23; Yes), it associates the product ID included in the order information with the attributes determined in step S22 when the order information was received and registers it in the order management file 3175 (step S24). Specifically, based on the table ID received along with the order information, the order management unit 353 associates the attributes determined from the image data of the order terminal 10 corresponding to this table ID with the product ID and registers it in the order management file. This makes it possible to associate the attributes of the customer who ordered the product with the product ID of the product and register it in the order management file 3175.

[0108] Furthermore, in the order terminal 10, the order reception unit 152 determines in step S17 whether or not an order confirmation operation has been performed (step S17). If the confirmation operation is not accepted (step S17; No), the order reception unit 152 returns to step S15, and accepts the selection of the items to be ordered until the confirmation operation is performed.

[0109] For example, if multiple customers are seated at the same table, when customers change and operate the order terminal 10, the faces of the customers captured by the imaging unit 116 will also change. In this process, the image data captured by the imaging unit 116 is transmitted to the server device 30 in real time, and at the same time, the product ID of the selected product is also transmitted to the server device 30. Therefore, even if the customer operating the order terminal 10 changes, the relationship between the customer's attributes and the ordered product can be maintained.

[0110] Furthermore, when the order receiving unit 152 receives the confirmation operation (step S17; Yes), it notifies the server device 30 of the order confirmation (step S18). Then, in response to the confirmation operation in step S17, the data acquisition unit 153 stops the operation of the imaging unit 116 (step S19) and terminates the process. Note that if imaging is to continue after the order confirmation, the data acquisition unit 153 may skip step S19.

[0111] Meanwhile, in the server device 30, the order management unit 353 is waiting for order confirmation notification (step S25; No → step S23; No), and when it receives order confirmation notification (step S25; Yes), it proceeds to step S26. In step S26, the product ID and corresponding product name included in the order information received in step S23 are output to the output device to inform the store staff (step S26). Then, the order management unit 353 changes the order flag of the product ID that has been sent to the output device to "Ordered" (step S27), and terminates the process.

[0112] Through the above process, the product ID of the product ordered from the order terminal 10 will be registered in the order management file 3175, associated with the attributes of the customer who ordered the product.

[0113] Figure 13 is a flowchart illustrating an example of the evaluation score registration process performed during the learning phase in the order terminal 10 and the server device 30. It is assumed that the order terminal 10 has completed the order processing described in Figure 12 (step S31).

[0114] When the order processing is complete, the data acquisition unit 153 of the order terminal 10 activates the sound pickup unit 117 and starts recording (step S32). The data acquisition unit 153 may also configure video recording in addition to audio recording by activating the imaging unit 116.

[0115] While the recording is in progress, the ordered items are served to table T, and the customer begins eating and drinking. During this time, the customer's speech is captured by the sound pickup unit 117 and recorded in the audio data. In addition, the external data acquisition device 20 also collects the customer's appearance and speech while using table T and transmits it to the server device 30.

[0116] Furthermore, while recording is in progress, the data acquisition unit 153 waits for an operation to instruct accounting (step S33; No), and when it receives an operation to instruct accounting (step S33; Yes), it stops the sound pickup unit 117 (step S34). If the imaging unit 116 was operating, the data acquisition unit 153 also stops the imaging unit 116 in step S34.

[0117] Next, the data acquisition unit 153 transmits the audio data obtained by recording to the server device 30 (step S35), and terminates the process. If the imaging unit 116 was operating, the data acquisition unit 153 also transmits the imaging data to the server device 30 in step S34.

[0118] Meanwhile, in the server device 30, the data collection unit 351 collects various data transmitted from the order terminal 10 and the external data acquisition device 20. When the evaluation score calculation unit 354 receives the voice data transmitted from the order terminal 10 (step S41), it identifies the records to be processed from the order management file based on the table ID transmitted along with the voice data (step S42). Specifically, the evaluation score calculation unit 354 processes the records that contain the corresponding table ID and for which no evaluation score has been entered.

[0119] Next, the evaluation score calculation unit 354 inputs the audio data into the second analysis model 3173 to begin analyzing the audio data (step S43). Specifically, the evaluation score calculation unit 354 obtains the utterance content recorded in the audio data from the output of the second analysis model 3173. The evaluation score calculation unit 354 also obtains keywords related to the product with the product ID from the product master 3171 based on the product ID included in the record to be processed. Then, the evaluation score calculation unit 354 attempts to detect keywords from the utterance content.

[0120] Next, the evaluation score calculation unit 354 determines whether or not it has detected a keyword from the utterance (step S44). If it determines that no keyword has been detected (step S44; No), the evaluation score calculation unit 354 registers information indicating an unknown evaluation score (for example, N / A) in the evaluation score column of the record to be processed, and terminates the process.

[0121] Furthermore, if the evaluation score calculation unit 354 detects a keyword from the utterance (step S44; Yes), it determines whether or not an evaluation word exists before or after the utterance position where the keyword was detected (step S45). If it determines that no evaluation word exists (step S45; No), the evaluation score calculation unit 354 registers information indicating an unknown evaluation score in the evaluation score column of the record to be processed, and terminates the process.

[0122] Furthermore, if the data collection unit 351 determines that an evaluation word exists before or after the utterance position of the keyword (step S45; Yes), it reads out the reference evaluation score corresponding to that evaluation word (step S46). Next, the evaluation score calculation unit 354 determines whether or not an adverb of degree exists before or after the utterance position of the evaluation word (step S47). If it determines that no adverb of degree exists (step S47; No), the evaluation score calculation unit 354 uses the reference evaluation score read out in step S46 as the evaluation score and proceeds to step S49.

[0123] Furthermore, if the evaluation score calculation unit 354 determines that an adverb of degree exists before or after the utterance position of the evaluation word (step S47; Yes), it multiplies the reference evaluation score by the weight corresponding to that adverb of degree and uses that value as the evaluation score (step S48), then proceeds to step S49.

[0124] Next, the evaluation score calculation unit 354 identifies the product ID of the product that is the subject of evaluation from among the product IDs included in the record to be processed (step S49). For example, the evaluation score calculation unit 354 identifies the product ID of the product corresponding to the keyword detected from the utterance as the product to be evaluated. Alternatively, for example, in order to associate the evaluation score with the product actually ordered by the customer who made the evaluation utterance, the evaluation score calculation unit 354 may identify the product ID using the results of analysis of other data with the same table ID recorded at the same time, or the results of the attribute determination unit 352.

[0125] Next, the evaluation score calculation unit 354 associates the evaluation score with the identified product ID and registers it (step S50), and then terminates the process.

[0126] Through the above process, the evaluation score for the products ordered from the order terminal 10 will be registered in the order management file 3175.

[0127] Figure 14 is a flowchart showing an example of the menu evaluation list generation process performed during the learning phase in the order terminal 10 and server device 30. It is assumed that this process has already been performed multiple times, as described in Figure 13.

[0128] First, the menu evaluation list generation unit 355 of the server device 30 refers to the order management file 3175 (step S61). Next, the menu evaluation list generation unit 355 calculates an overall evaluation value by adding the evaluation values ​​for each product ID registered in the order management file 3175 for each product ID and for each customer attribute (step S62). Here, the menu evaluation list generation unit 355 calculates the overall evaluation value by excluding records in which the evaluation value field is not registered or where the evaluation score is unknown.

[0129] Next, the menu evaluation list generation unit 355 registers the calculated overall evaluation value, associated with the product ID and customer attributes related to the calculation, into the menu evaluation list file 3176 (step S63). If an existing menu evaluation list is registered in the menu evaluation list file 3176, the menu evaluation list generation unit 355 updates the menu evaluation list by overwriting it with the menu evaluation list or by overwriting it with the difference value.

[0130] Next, the menu update unit 356 generates customized menu information for each customer based on the menu evaluation list file 3176 (step S64). Then, the menu update unit 356 sends the generated menu information to each of the order terminals 10 (step S65) and terminates the process.

[0131] Furthermore, the timing at which the menu update unit 356 generates menu information is not limited to immediately after the generation of the menu evaluation list. For example, the menu update unit 356 may generate menu information on a schedule independent of the menu evaluation list generation process.

[0132] Meanwhile, when the order terminal 10 receives menu information (step S71), the menu display unit 151 stores it as menu information M in the storage unit 119 (step S72), and then terminates processing.

[0133] Through the above process, the server device 30 calculates an overall evaluation value for each customer attribute for each product from the accumulated order and evaluation score history, and generates a menu evaluation list that associates the products and customer attributes related to the calculation of the overall evaluation value. The server device 30 also generates customized menu information for each customer attribute based on the menu evaluation list and provides it to the order terminal 10. As a result, each order terminal 10 can move to the stage of recommending products that correspond to the customer's attributes by displaying a menu screen based on the customized menu information.

[0134] Figure 15 is a flowchart illustrating an example of the process related to the display of a menu screen performed at the product recommendation stage in the order terminal 10 and server device 30 of the embodiment. Steps S81 to S83 of the order terminal 10 and steps S91 and S92 of the server device 30 are the same as steps S11 to S13 of the order terminal 10 and steps S21 and S22 of the server device 30 described in Figure 12, so their explanation is omitted.

[0135] In step S92, the attribute determination unit 352 of the server device 30 determines the customer's attributes and then notifies the order terminal 10 that transmitted the imaging data of the determined attributes (step S93).

[0136] Meanwhile, when the menu display unit 151 of the order terminal 10 receives attribute notification from the server device 30 (step S84), it reads menu information M corresponding to the notified attribute from the order terminal 10 and displays a menu screen based on the menu information M on the display unit 114 (step S85). Subsequent processing is the same as steps S15 and S23 onwards in Figure 12.

[0137] Through the above process, the order terminal 10 can display a menu screen using menu information customized for the attributes of the customer operating the order terminal 10. As a result, the order terminal 10 can display a menu screen that corresponds to the customer's attributes, and can, for example, recommend products that suit the customer's preferences or recommend products that the customer would not normally order.

[0138] Next, with reference to Figure 16, the process of receiving a menu evaluation list request performed by the server device 30 will be described. Figure 16 is a flowchart showing an example of the process of receiving a request request performed by the server device 30.

[0139] In Figure 16, the employee terminal 40 is shown as the device that requests access to the menu evaluation list, but the requesting device is not limited to the employee terminal 40. For example, the server device 30 itself may be the requesting device. In this case, the server device 30 receives the request to access the menu evaluation list via the operation unit 315.

[0140] First, the employee terminal 40 sends a request to the server device 30 to refer to the menu evaluation list in response to the operator's actions (step S101).

[0141] When the reference request receiving unit 357 of the server device 30 receives a reference request (step S111), it generates a menu evaluation list screen to display the menu evaluation list status (step S112). Then, the reference request receiving unit 357 sends the generated menu evaluation list screen to the requesting store employee terminal 40 (step S113).

[0142] When the employee terminal 40 receives the menu evaluation list screen from the server device 30 (step S102), it displays the menu evaluation list screen on a display unit (not shown) (step S103).

[0143] As explained in Figure 11, the menu evaluation list screen has operators for specifying, for example, customer attributes that serve as display conditions. The reference request receiving unit 357 of the server device 30 updates the menu evaluation list to be displayed on the menu evaluation list screen according to the display conditions specified by the employee terminal 40.

[0144] This allows store employees and other operators using the employee terminal 40 to easily check customer ratings for each product by referring to the menu rating list screen, broken down by customer attributes. For example, by referring to the menu rating list screen, the operator of the employee terminal 40 can easily check which customer attributes are popular with the products offered at their store, or which are unpopular with which customer attributes.

[0145] As explained above, the recommendation system 1 determines the attributes of a customer based on image data of the customer who is ordering a product, and associates and stores the product ID of the product ordered by the customer with the determined customer attributes. The recommendation system 1 also detects the portion of the customer's speech related to their evaluation of the ordered product from the audio data that records the customer's speech, and calculates an evaluation score that quantifies the evaluation of the product based on the content of the detected speech portion. The recommendation system 1 then generates a list (each record in the order management file 3175 or the menu evaluation list in the menu evaluation list file 3176) that associates the set of identification information and attributes with the evaluation score calculated for the product corresponding to that identification information.

[0146] This allows recommendation system 1 to collect evaluations of ordered products from customers who have actually used the store, without requiring them to fill out questionnaires, thus enabling efficient collection of product feedback.

[0147] Furthermore, recommendation system 1 generates menu information for each customer attribute, based on the relationship between the customer attributes included in the generated list and the rating score for each product ID, enabling the display of a menu screen to assist in ordering products. In addition, recommendation system 1 displays the menu screen using menu information corresponding to the attributes determined based on image data of the customer who is ordering the product, from among the menu information for each customer attribute.

[0148] As a result, recommendation system 1 can recommend products that match the customer's attributes, based on evaluation results aggregated for each customer's attributes, via a menu screen, thus enabling efficient product recommendations based on evaluation results.

[0149] The embodiments described above can also be modified and implemented as appropriate by changing some of the configurations or functions of each of the devices described above. Therefore, several modifications of the embodiments described above will be described below as other embodiments. In the following, we will mainly describe the differences from the embodiments described above, and will omit detailed explanations of points that are common with what has already been described. Furthermore, the modifications described below may be implemented individually or in combination as appropriate.

[0150] (Variation 1) In the above-described embodiment, the server device 30 determines the attributes of the customer operating the order terminal 10, but the embodiment is not limited to this. For example, each order terminal 10 may function as an example of an information processing device to determine the attributes of the customer operating its own order terminal 10. In this case, the order terminal 10 is equipped with a first analysis model 3172 and an attribute determination unit 352, and the attributes of the customer can be determined from the imaging data captured by the imaging unit 116 by the function of the attribute determination unit 352.

[0151] With this configuration, for example, the order terminal 10 can perform the processing related to displaying the menu screen at the product recommendation stage (see Figure 15) solely on its own. As a result, the order terminal 10 can display the menu screen corresponding to the customer's attributes at a faster speed, thereby improving usability.

[0152] (Modification 2) In the above embodiment, the server device 30 calculates the evaluation score for the product, but the system is not limited to this, and each order terminal 10 may calculate the evaluation score. In this case, the order terminal 10 is equipped with a product master 3171, a second analysis model 3173, an evaluation score setting file 3174, an evaluation score calculation unit 354, etc., and the evaluation score calculation unit 354 can calculate the evaluation score for the product from the audio data collected by the sound collection unit 117. Alternatively, the order terminal 10 may be configured to analyze the correspondence between the customer who ordered the product and the product that was the subject of evaluation using the imaging data captured by the imaging unit 116.

[0153] Furthermore, the order terminal 10 may be configured to analyze the correspondence between a customer who ordered a product and the product being evaluated, by including a first analysis model 3172 and an attribute determination unit 352, or by cooperating with an external data acquisition device 20 that images the table T on which the order terminal 10 is placed.

[0154] With this configuration, for example, each order terminal 10 can perform the collection of evaluation points necessary during the learning phase. This reduces the load on the server device 30.

[0155] (Variation 3) In the embodiment described above, the menu update unit 356 generates menu information based on the list (menu evaluation list) generated by the menu evaluation list generation unit 355, but it is not limited to this. For example, the menu update unit 356 may generate menu information based on the list generated by the evaluation score calculation unit 354, that is, each record in the order management file 3175.

[0156] In this case, the menu update unit 356 only needs to generate menu information for each customer attribute based on the relationship between the customer's attributes and the evaluation score for each product ID, which is included in each record in the order management file 3175 that has a valid evaluation score registered.

[0157] This provides the same effects as in the above embodiment, enabling efficient product recommendations based on evaluation results.

[0158] (Modification 4) In the above embodiment, the evaluation score registration process performed during the learning phase (see Figure 13) is configured to analyze audio data recorded between order completion and the instruction to perform accounting operations on the server device 30. However, the system is not limited to this configuration, and the audio data picked up by the order terminal 10 may be analyzed in real time by the server device 30.

[0159] (Variation 5) In the above embodiment, the server device 30 manages orders, but the system is not limited to this, and other devices (hereinafter also referred to as order management devices) may manage orders. In this case, the server device 30 stores customer attributes and ratings in association with the product IDs of ordered products by referring to or manipulating the order management file managed by the order management device.

[0160] The programs executed in each of the above-described embodiments are provided pre-installed in ROM, storage units, etc. Alternatively, the programs executed in each of the above-described embodiments may be provided as installable or executable files recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk).

[0161] Furthermore, the programs executed by each of the above-described embodiments may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the programs executed by each of the above-described embodiments may be provided or distributed via a network such as the Internet.

[0162] Although embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments and their variations can be implemented in a variety of other forms, and various omissions, substitutions, changes, and combinations can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0163] 1. Recommendation System 10 Order terminals 20 External data acquisition device 30 Server Devices 40. Staff terminal 151 Menu display section 152 Order Reception Department 153 Data Acquisition Unit 351 Data Collection Unit 352 Attribute determination section 353 Order Management Department 354 Evaluation score calculation unit 355 Menu Evaluation List Generation Unit 356 Menu Update Section 357 Reference Request Reception Department [Prior art documents] [Patent Documents]

[0164] [Patent Document 1] Japanese Patent Publication No. 2016-177755

Claims

1. A first determination means that determines the attributes of a customer from image data of a customer who is placing an order for a product, A matching means that associates the identification information of the product ordered by the customer with the attributes determined by the first determination means, A detection means for detecting the portion of a customer's speech related to their evaluation of the ordered product from audio data that records the customer's speech content, A calculation means calculates an evaluation score that quantifies the evaluation of the product based on the content of the utterance detected by the detection means, A first generation means generates a list that associates the set of identification information and attributes associated by the association means with the evaluation score calculated by the calculation means for the product corresponding to the identification information. A second generation means generates menu information for each attribute that can display a menu screen to support ordering the product, based on the relationship between the attribute included in the list and the evaluation score for each of the identification information, Equipped with, The detection means detects the utterance position in the speech content of the voice data where a keyword representing the ordered product appears, The calculation means is an information processing device that calculates an evaluation score for the product based on the meaning of words representing evaluation that appear before and after the utterance position of the keyword.

2. The calculation means calculates an evaluation score for the product based on the magnitude represented by modifiers that appear before and after the utterance position of the word representing the evaluation. The information processing apparatus according to claim 1.

3. The system further includes a second determination means that determines whether the utterance portion relating to the evaluation of the product detected by the detection means is an utterance portion made by the customer who ordered the product, based on the customer's actions shown in the aforementioned imaging data. The calculation means calculates an evaluation score when the second determination means determines that the utterance is made by the customer who ordered the product. The information processing apparatus according to claim 1.

4. The information processing apparatus according to claim 1, wherein the second generation means generates menu information that allows the display of a menu screen that preferentially presents products with high evaluation scores for each attribute.

5. A display means for displaying a menu screen to assist in ordering products, A first determination means that determines the attributes of a customer from image data of a customer who is placing an order for a product, A matching means that associates the identification information of the product ordered by the customer with the attributes determined by the first determination means, A detection means for detecting the portion of a customer's speech related to their evaluation of the ordered product from audio data that records the customer's speech content, A calculation means calculates an evaluation score that quantifies the evaluation of the product based on the content of the utterance detected by the detection means, A first generation means generates a list that associates the set of identification information and attributes associated by the association means with the evaluation score calculated by the calculation means for the product corresponding to the identification information. A second generation means generates menu information for each attribute that can display a menu screen to support ordering the product, based on the relationship between the attribute included in the list and the evaluation score for each of the identification information, It has, The detection means detects the utterance position in the speech content of the voice data where a keyword representing the ordered product appears, The calculation means calculates an evaluation score for the product based on the meaning of words representing evaluation that appear before and after the utterance position of the keyword. The display means displays the menu screen using menu information from the menu information generated by the second generation means that corresponds to the attribute determined by the first determination means. Information processing system.

Images