Information gathering system

Abstract

To gather personal data from information by a plurality of users.SOLUTION: An information gathering system comprises an order terminal and a control part. The order terminal displays, on a screen, a character image to which identification information set for each user is related and an image of an article, and can move and operate the image of the article on the screen onto or to nearby a display region of the character image. The control part generates preference information for an article of each user corresponding to the character image when the image of the article is moved onto or to nearby the display region of each character image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0004] , , , , , , , , , , , ,

[0005] ,

[0003]

[0001] Embodiments of the present invention relate to an information collection system.

Background Art

[0002] Generally, in stores such as restaurants, the mainstream form was to take orders from customers through store clerks. However, in order to overcome the difficulty of securing store clerks and improve business efficiency, many stores have adopted a form in which customers operate order terminals installed on each table to place orders by themselves.

[0003] In addition, the store side may conduct a questionnaire survey on customers who visit the store for the purpose of enhancing customer satisfaction, improving the repeat rate, and acquiring new customers. A general questionnaire survey is conducted in a form where a store clerk directly gives a questionnaire to a customer and asks the customer to fill it out, or in a form where information such as the name, quantity, and order date and time of items ordered from an order slip or order terminal is obtained.

[0004] As for the above-mentioned questionnaire survey, for example, in a restaurant, when one person in a group of multiple people typically answers the survey, the name of the ordered dish, the order quantity, and the evaluation of the quality of that dish can be collected as information. However, it is impossible to collect who ate what and the distributed amount or number within the group as personal data. On the other hand, even if an attempt is made to collect personal data by distributing questionnaires individually, it is difficult to collect them from everyone. Even if the questionnaires could be collected from everyone, it would be time-consuming to organize the information for associating it with the orders placed by the group. Similarly, even for information collected from an order slip or order terminal, it cannot be separately collected as each person's personal data.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] Therefore, the objective is to provide an information collection system that enables surveys conducted among groups of multiple people, allowing for the collection of individual responses while maintaining the group's relationships. [Means for solving the problem]

[0007] An information collection system according to one embodiment displays on the screen a character image associated with identification information set for each user and an image of an item, and the image of the item on the screen Part or all of the area The character image can be moved on or near the display area. Based on the said transfer operation, personal data indicating the amount of the aforementioned items distributed to each user is transmitted. Order terminal and the aforementioned Based on the personal data transmitted from the order terminal The system comprises a control unit that generates preference information for the item for each user corresponding to the character image. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is an explanatory diagram showing an example configuration of an information collection system according to the first embodiment. [Figure 2] Figure 2 is a sequence diagram illustrating the first information gathering process. [Figure 3] Figure 3 is a flowchart explaining how to answer the questionnaire. [Figure 4A] Figure 4A shows an example of a screen layout that indicates how to answer a questionnaire. [Figure 4B] Figure 4B shows an example of a screen layout that indicates how to answer a questionnaire. [Figure 4C] Figure 4C shows an example of a screen layout that indicates how to answer a questionnaire. [Figure 4D] Figure 4D shows an example of a screen layout illustrating how to answer a questionnaire. [Figure 5] Figure 5 is a sequence diagram illustrating the second information gathering process. [Figure 6] Figure 6 is a sequence diagram illustrating the third information gathering process. [Figure 7] Figure 7 is a sequence diagram illustrating the fourth information gathering step. [Figure 8] Figure 8 is a sequence diagram illustrating the fifth information gathering step. [Figure 9] Figure 9 is an explanatory diagram showing an example configuration of an information collection system according to the second embodiment. [Figure 10] Figure 10 is a sequence diagram illustrating the sixth information gathering step. [Figure 11] Figure 11 is a sequence diagram illustrating the seventh information gathering step. [Figure 12] Figure 12 is a sequence diagram illustrating the eighth information gathering step. [Figure 13] Figure 13 is a sequence diagram illustrating the ninth information gathering step. [Figure 14] Figure 14 is a sequence diagram illustrating the tenth information gathering step. [Modes for carrying out the invention]

[0009] <First Embodiment> The following describes an information collection device for constructing an information collection system according to the first embodiment, with reference to the drawings. Figure 1 is an explanatory diagram showing an example of the configuration of the information collection device 1 according to the first embodiment.

[0010] The information collection device 1 of this embodiment displays on the screen a plurality of users (so-called avatars) each set with an ID and characterized, together with the ordered dishes (images). On this screen, by moving a part or all of the image area of the dish onto or near the user characterized by drag and drop, the dish is linked to each individual user. By performing the same operation on a plurality of users, personal data for the dishes eaten and drunk by each user can be created. Using this information collection method, even in a group of multiple people, questionnaire surveys and other information collection to be collected as personal data can be realized. In the following description, the item targeted in the example of the description is referred to as "dish", and the image of the user characterized known as an avatar is referred to as a character image.

[0011] This information collection device 1 is composed of a store server 2, an order terminal 3, a cash register (POS terminal) 4, a kitchen terminal 5, and a camera 6. This store server 2 is connected to the order terminal 3, the cash register 4, the kitchen terminal 5, and the camera 6 using wired or wireless means, and control signals and image signals are communicated. This information collection device 1 can be connected to an external user terminal 7 such as a mobile phone by wireless communication. Here, for example, an example using wireless communication such as WiFi or short-range wireless communication such as Bluetooth (registered trademark) will be described.

[0012] The store server 2 is composed of, for example, a personal computer, and includes a communication unit 12 equipped with a wireless antenna 11, a storage unit 13, a receipt information storage unit 14, an image processing unit 15, an output unit 16, and a control unit (processor) 17. These component parts are connected by a bus 18 serving as a transmission path.

[0013] The communication unit 12 converts a control signal or image data into a wireless signal capable of communication and transmits it from the wireless antenna 11, and converts the wireless signal received by the wireless antenna 11 into a control signal or image data. Further, the communication unit 12 has an interface function for devices connected to the store server 2.

[0014] The storage unit 13 stores at least the processor's control program, menu information related to dishes (such as dish names and prices), a customer ID (which is an identification number or registration number that identifies the customer), image data, and customer preference information. This storage unit 13 includes a non-volatile memory for storing the processor's (control unit 17) control program and other information, and a volatile memory for storing the menu information, customer ID, image data, and customer preference information in a rewritable manner. As the non-volatile memory, for example, ROM (read-only memory) or flash memory can be used. As the volatile memory, an HDD (hard disk drive) or SSD (solid state drive) that can be used as a work area can be used.

[0015] The receipt information storage unit 14 stores the receipt information contained in the receipt or electronic receipt created when payment is made by the payment machine (POS terminal) 4, which will be described later. In this embodiment, the receipt information includes at least the date and time of payment, the store name, the name of the food and beverage (item name), the quantity, the amount (unit price, total amount and tax), the payment method, and the name of the person who made the payment. This information is associated with an ID (registration number).

[0016] The image processing unit 15 converts the video signal captured by the camera 6 into a digital image signal and outputs it as image data to the control unit 17. The camera 6 is positioned above the customer table and is capable of capturing multiple customers surrounding the table in the same frame. The camera 6 captures each customer serving themselves from the food placed on the table. Of course, the camera is not limited to one unit; multiple cameras may be positioned to divide the table surface and capture partial images, which are then combined during image processing to create a single overall image. A sensor may also be placed near the table. The sensor can detect when customers are seated around the table. This sensor detection can automatically start the camera 6 from taking pictures. In this embodiment, the camera 6 stops taking pictures when the store server 2 receives a payment request signal from the order terminal 3.

[0017] The output unit 16 outputs the order information (dish name, quantity, etc.) received from the order terminal 3 to the kitchen terminal 5 installed in the kitchen. Kitchen terminal 5 is placed in the kitchen where the cooks are located and informs the cooks of the names and quantities of the dishes that have been ordered. Kitchen terminal 5 has a display screen and displays the order information (names of dishes and their quantities) received from output unit 16 as an instruction to start cooking.

[0018] The control unit 17 consists of a processor and drives and controls each component according to a pre-set control program. The control unit 17 creates a questionnaire, which will be described later, analyzes the questionnaire results answered by the customer, and creates preference information.

[0019] Next, the order terminal 3 will be described. The information gathering device 1 may include multiple order terminals 3, one of which is shown as a representative example in Figure 1. The order terminal 3 is placed on or near each customer's table and takes food orders via customer operation. The order terminal 3 is, for example, a tablet with a display screen, and connects to the store server 2 via wireless communication, and has functions such as taking food orders, making inquiries, requesting payment, and inputting responses to the questionnaire described later. Here, the order terminal 3 that uses wireless communication is described as an example, but of course, it may also be connected by wire.

[0020] The order terminal 3 comprises a communication unit 21 equipped with a wireless antenna 22 for connecting to the store server 2, an input unit 23, a memory 24, a processor 25, and a display unit 26. These components are connected by a bus 27, which serves as a transmission path.

[0021] The communication unit 21 has the function of wirelessly communicating with the communication unit 12, converting control signals and image data into wireless signals that can be communicated and transmitted from the wireless antenna 22, and the wireless antenna 22 converts the received wireless signals into control signals and image data and outputs them to the processor 25. Furthermore, the order terminal 3 conducts a customer survey based on customer operations, as described later, and outputs the survey results to the store server 2.

[0022] The input unit 23 is equipped with a touchpad and key switches for customer input. Additionally, a camera unit may be added as part of the input unit 23's functionality to provide a shooting function. This shooting function is used for reading identification information such as barcodes, and for photographing the person performing the input operation to create image data for person identification. The memory 24 stores, in addition to operating information or control programs for driving and controlling the device, at least menu information and guide information related to ordering operations in a rewritable format.

[0023] The processor 25 is, for example, a CPU (Central Processing Unit) that controls all components within the order terminal 3 according to a control program stored in the memory 24. The display unit 26 includes a liquid crystal display panel and an image processing circuit that converts image signals into display signals. In this embodiment, a touch panel combining the touch pad of the input unit 23 and the liquid crystal display panel may be used. The display unit 26 displays at least menu information, guide information regarding ordering operations, in-store information, and survey information.

[0024] The payment machine 4 uses, for example, a POS (Point of Sales) terminal. The payment machine 4 has the function of registering payment information, including the name, quantity, and price of the dishes served, when a customer who has registered their ID pays for a meal, and the function of settling the transaction with that customer. When payment for food and drinks is made, the payment machine 4 creates receipt information showing the details of the food and drinks served and issues a receipt. The receipt may be issued as a paper receipt or as an electronic receipt using the receipt information described later. When issuing a paper receipt, the payment machine 4 controls its built-in printer to print the receipt information on a predetermined receipt paper. When issuing an electronic receipt, the payment machine 4 sends the digitized receipt information along with the authenticated customer identification information, which is the ID, to the store server 2. Note that the payment machine 4 may be a payment machine that does not have a registration processing function.

[0025] The external user terminal 7 can be a wirelessly wireless-enabled portable device, such as a mobile phone or smartphone. The user terminal 7 has wireless communication and display functions, and by configuring various applications, it can perform a wide range of functions and processes, similar to a personal computer. For example, by downloading and using any search application, it becomes easy to find candidates for desired restaurants or items to purchase.

[0026] [First Information Gathering] Next, referring to the sequence diagram shown in Figure 2, we will explain the first information gathering method, which is the collection of information from customers by the information gathering device 1, i.e., a questionnaire survey. Here, we will explain an example in which a user enters a restaurant where the system is set up for ordering using the order terminal 3.

[0027] First, the user terminal 7 downloads a search application and an electronic receipt application of their choice in advance and registers a password and identification number (ID). Alternatively, the user may register a password and identification number (ID) set by the store they are using. By using the identification number (ID), the store server 2 or the server system 31 (described later) can identify the user terminal 7. The server system 31 includes a central server 32 and a receipt information server 33.

[0028] Next, the user operates the order terminal 3 in the store and enters their own ID. This ID can be entered using keys on the order terminal 3, by displaying an identification code such as a barcode on the user terminal 7's screen for the order terminal 3 to read, or by accessing the store server 2 from the user terminal 7 to recognize the ID. The entered ID is sent to the store server 2 for authentication. Furthermore, when a group visits the store, there may be users who have not registered their IDs. If a user without an ID places an order, the number of people present must be entered into the order terminal 3. If this number is not set, the number of characters displayed in the diagram shown in Figure 4A (described later) may differ.

[0029] Next, the customer operates the order terminal 3 to select and set food and drinks from the displayed menu. For example, they select and set one pizza, one paella, and one loaf of bread, along with two drinks, as shown in Figure 4A later. They then adjust the order by adding or removing other dishes.

[0030] Once the food and beverages to be ordered (such as the name of the dish and the quantity) have been decided, the customer operates the confirmation key on the order terminal 3 to send the decided food and beverages as order data from the communication unit 21 to the store server 2. At this time, the order data will include, for example, the ID information of two customers, as will be described later. When the control unit 17 of the store server 2 receives the order data via the communication unit 12, it stores the order data in the storage unit 13 and also sends the order data via the output unit 16 to the kitchen terminal 5 installed in the kitchen. The kitchen terminal 5 receives the order data and displays the name of the ordered dish, the quantity, etc. on its display screen.

[0031] Subsequently, the control unit 17 of the store server 2 creates a questionnaire corresponding to the ordered food and beverage. For example, for pizza, the questionnaire items would include the degree of doneness, size, and taste, while for pasta, the questionnaire items would include the degree of doneness, quantity, and taste. These questionnaire items are pre-stored in the memory unit 13 for each dish name corresponding to the menu, and are read from the control unit 17 based on the entered order data (dish name), and a questionnaire suitable for the order data is created as needed.

[0032] Next, the control unit 17 of the store server 2 sends the created questionnaire to the order terminal 3 when a predetermined period of time has elapsed, which is assumed to be when the customer has finished eating and drinking, or when it receives a payment request signal from the order terminal 3.

[0033] Upon receiving the questionnaire, the order terminal 3 displays the questionnaire as a pop-up on its screen, including the questionnaire content and an operation guide showing the operating procedures. The questionnaire content may include, for example, the quantity (or amount), taste (mild, strong, salty, sweet, etc.), condition (hard, soft, size (large or small), hot, lukewarm, etc.), and appearance for each dish. Various survey items may be in a selectable format using checkboxes or similar methods.

[0034] The customer follows the operation guide and answers the questionnaire as described below to create their questionnaire response. The completed questionnaire response is sent to the store server 2 by touching the "Complete" button displayed on the order terminal 3's screen.

[0035] The store server 2 receives the survey responses, and the control unit 17 processes the responses to convert them into preference data. Based on the survey responses, the control unit 17 categorizes the dish name into the dish genre, quantity (amount), taste (mild, strong, salty, sweet, etc.), condition (hard, soft, size (large / small), hot, lukewarm, etc.), and appearance, and converts this data into numerical values ​​by comparing them with a scale or threshold, which can then be used, for example, for ranking. For example, if the dish name is "pizza," it will be categorized into the genre of wheat - Italian cuisine - Western cuisine. Similarly, if the dish name is "paella," it will be categorized into the genre of rice - Spanish cuisine - Western cuisine. These classifications are, of course, not limited to those above, and may be classified according to an index for searching. In addition, the quantity is assigned to each user terminal 7 ID and may not necessarily be "1," but may be categorized as a fraction or percentage such as "2 / 3" or "1 / 2."

[0036] Next, the control unit 17 sorts the digitized questionnaire responses into preference information categorized into a predetermined number of categories, and adds to or updates the list. At this time, it may perform both adding to or updating the preference information formed for all users with an ID, and adding to or updating the preference information for each individual managed by the ID associated with the questionnaire response.

[0037] Subsequently, the store server 2 checks whether it has already received a payment request signal from the order terminal 3. If it has already received the payment request signal, it instructs the payment machine 4 (POS terminal) to perform the payment process. If the store server 2 has not yet received a payment request signal from the order terminal 3, it remains in standby mode. Upon receiving the payment instruction, the payment machine 4 performs the payment using the payment method specified by the customer from among various payment methods such as cash payment and card payment. During the payment process, the payment machine 4 creates receipt information (electronic receipt) consisting of various payment information and reports it to the store server 2. The customer may be given a paper receipt with the receipt information printed on it. As mentioned above, the receipt information includes at least the date and time of payment, store name, food and beverage name, quantity, amount (unit price, total amount and tax), payment method, and name of the person in charge of payment.

[0038] Next, we will explain how to search for preference information using user terminal 7. First, the user terminal 7 launches the search application, accesses the store server 2, and begins searching for preference information. The user terminal 7 selects the desired food and drink and sends a search command to the store server 2. The store server 2 receives the search command, and the control unit 17 reads the relevant information from the preference information stored in the memory unit 13 and sends it to the user terminal 7 as a search result. The user terminal 7 receives the search result and displays it. The search result reflects, for example, the results of surveys answered by other users, and includes evaluations and comments about the food, etc.

[0039] Next, we will explain an example of how to answer the questionnaire, referring to the flowchart shown in Figure 3 and the screen display examples shown in Figures 4A to 4D. Order terminal 3 receives the questionnaire from store server 2 (ACT1). First, order terminal 3 displays a screen indicating the questionnaire request (ACT2). Next, it displays an operation guide for creating the questionnaire response (ACT3). Note that the operation guide is not limited to a display; it may also be an audio operation guide.

[0040] Next, a questionnaire screen created based on the order details shown in Figure 4A is displayed on the display unit 26 (ACT4). In the questionnaire screen shown in Figure 4A, the two customers whose IDs have been entered are converted into character images 101 and 102 shown at the bottom of the screen and displayed as avatars. These character images may be selected and displayed as needed from a large number of characters stored in advance, according to the ID, or specific characters may be set when registering the ID to the store server 2. Furthermore, the bottom of the screen displays an order list of food and drinks ordered so far, and the total amount 104, which is the payment amount for the orders so far. In addition, the ordered dishes are displayed in the center of the questionnaire screen. Here, pizza 201, paella 202, bread 203, and drinks 204 and 205 are displayed. Note that if there are customers among the multiple customers whose IDs have not been registered, there is an item in the order data to set the number of customers, so when creating a composition like the one shown in Figure 4A, the number of created character images can be matched with the actual number of customers.

[0041] Next, the customer selects the dish to be surveyed by touching a dish displayed on the survey screen, for example, pizza 201 (ACT 5). The customer then touches a character image 101 displayed as their avatar to specify the amount (size) of pizza 201 they ate, as shown in Figure 4B (ACT 6). For example, they place their finger on the image of pizza 201 and trace it, drawing a line L to represent the portion of pizza 201A that they ate. Here, the example shows the customer having eaten 2 / 3 of pizza 201, and the line L is used to divide the eaten area.

[0042] Please note that all responses to the survey, from the time you touch character statue 101 until you touch another character statue, will be counted as responses for character statue 101. Therefore, to switch to character statue 102, you need to touch pizza 201 first, and then character statue 102. The order in which you touch pizza 201 and character statue 102 is not limited; you can also touch character statue 102 first, and then pizza 201.

[0043] Next, as shown in Figure 4C, the customer places their finger on pizza 201A, which is the area of ​​pizza 201 that has been eaten, on the display screen and moves it by dragging and dropping to move it to the display area of ​​their character 101 or nearby (ACT7). By performing this operation, the amount eaten can be easily set by moving the character 101 as if the customer is feeding it the amount they have eaten. Alternatively, an input box for setting the percentage numerically may be displayed on the display screen. For example, when the customer places their finger on pizza 201, the dish is selected and an input box is displayed. By writing a numerical value in this input box with their finger, a line L is drawn on pizza 201 and it is cut. Similarly, the customer places their finger on pizza 201A and moves it to the display area of ​​their character 101 or nearby by dragging and dropping.

[0044] When pizza 201A is moved to character image 101, the display area of ​​pizza 201A shrinks. After pizza 201A has moved, the remaining pizza 201B, which was cut by line L, remains displayed.

[0045] Next, another customer corresponding to character 102 touches character 102, then places their finger on the remaining pizza 201B and moves it by dragging and dropping it to either the display area of ​​their character 102 or its vicinity. As shown in Figure 4D, the moved pizza 201B is displayed in a reduced state near character 102. This drag-and-drop operation is repeated for all ordered food and drinks. Once all ordered food and drinks have been distributed (ACT8), the survey response is completed and sent from the order terminal 3 to the store server 2 as a survey response (ACT9).

[0046] The process of digitizing the portion sizes of food served via drag-and-drop will be explained using pizza 201 as an example. The area of ​​the initial image of pizza 201 displayed on the order terminal 3's screen before serving will be used as the base area. The area of ​​the portion of the pizza that the customer has cut arbitrarily on the screen and moved onto character image 101 will be compared to the base area, and the ratio will be calculated numerically. In this way, the image area of ​​each dish before the meal is used as the reference area, and by comparing the area of ​​a portion of the dish that the customer has served and moved with the reference area, the amount served is converted into a numerical value. This makes it possible to digitize the amount of food served. In this way, survey responses can be easily converted into data by moving the amount of food consumed by the customer as if feeding it to a character figure.

[0047] [Second Information Gathering] Next, referring to the sequence diagram shown in Figure 5, we will explain the second method of information gathering, which involves adding receipt information from the receipts created during payment, in addition to the customer questionnaire survey conducted by the information gathering device 1. Here, operations and actions equivalent to those described in the sequence diagram in Figure 2 will be explained in a simplified manner.

[0048] First, the user terminal 7 is configured with a search application and an electronic receipt application, and the password and identification number (ID) are registered with the store server 2. Next, after entering the restaurant, multiple customers (users) operate the order terminal 3 and enter their customer IDs. The entered IDs are sent to the store server 2 for authentication. Subsequently, customers select food and drinks from the menu, adjust the dishes and quantities they wish to order, and finalize their order. The food and drink order is then sent from the order terminal 3 to the store server 2.

[0049] The control unit 17 of the store server 2 stores the received order data in the storage unit 13 and places an order with the kitchen terminal 5. The kitchen terminal 5 displays the name and quantity of the ordered dishes. Subsequently, the control unit 17, as described above, creates a questionnaire corresponding to the ordered food and beverages by combining pre-set survey items.

[0050] Next, the control unit 17 of the store server 2 sends the questionnaire to the order terminal 3 when a predetermined time has elapsed or when it receives a payment request signal from the order terminal 3. The order terminal 3 displays the questionnaire as a pop-up on its display screen. The customer follows the questionnaire operation guide, completes the questionnaire, and sends it to the store server 2. The store server 2 receives the questionnaire response, processes it using the control unit 17, and converts it into data as described above.

[0051] Next, the control unit 17 sorts the digitized survey responses into preference information categorized into multiple categories. This new preference information is added to or updates the preference information for all users with an ID, as well as the preference information for each individual with an ID associated with the survey response. Subsequently, the store server 2 checks whether it has already received a payment request signal from the order terminal 3. If it has already received the payment request signal, it instructs the payment machine 4 to perform the payment process. If the store server 2 has not yet received a payment request signal from the order terminal 3, it remains in standby mode.

[0052] Upon receiving a payment instruction, the payment machine 4 performs the payment using the payment method specified by the customer from among various payment methods such as cash payment and card payment. At this time, the payment machine 4 creates transaction information containing various payment details during the payment process and transmits it to the store server 2. The store server 2 extracts the payment date and time, store name, food and beverage name, quantity, amount (unit price, total amount and tax), payment method, and name of the person in charge of payment from the transaction information as receipt information. This receipt information is stored in the receipt information storage unit 14. This receipt information can be output as an electronic receipt to the user terminal 7, making it accessible to the user.

[0053] The control unit 17 estimates, based on receipt information, at least the user's time spent in the store, the level of congestion, the number of customers by time of day, the number of orders, and the ranking of sales of items, and can also estimate the estimated sales amount. The time spent in the store can be calculated from the time from the time of the first order to the time of payment processing. In addition to preference information obtained from customer surveys, the control unit 17 adds receipt information created through payment processing, thereby adding time information (such as time spent in the store and busy times) and cost information, creating preference information that is easier for users to use.

[0054] Furthermore, the second information gathering device 1 can access the store server 2 from the user terminal 7, read the electronic receipt stored in the receipt information storage unit 14 to the user terminal 7, and use it for viewing, printing, and other purposes.

[0055] [Third Information Gathering] Next, referring to the sequence diagram shown in Figure 6, we will explain the third method of information gathering, which involves information gathering using image data of customers captured by the information gathering device 1. Here, operations and actions equivalent to those described in the sequence diagram shown in Figure 2 will be explained in a simplified manner.

[0056] First, the user terminal 7 is configured with a search application and an electronic receipt application, and the password and identification number (ID) are registered with the store server 2. Next, multiple customers (users) enter the restaurant and sit down at the tables. Sensors placed near the tables detect the people, and camera 6, positioned above the tables, begins recording. At this time, camera 6 adjusts its field of view (shooting area) so that all customers who have entered the restaurant are captured.

[0057] Next, the customer operates the order terminal 3 and enters their own ID, as described above in Figure 2. The entered ID is sent to the store server 2 for authentication. At this time, the camera 6 captures each customer entering their ID, and during the image analysis described later, the time of ID entry (clock) and the time of capture (clock) are compared to identify the customer who entered their ID, and it is possible to estimate or recognize the customer's seating position.

[0058] Next, the customer selects food and drinks from the menu, adjusts the dishes and quantities to finalize their order. The order is sent from the order terminal 3 to the store server 2. The control unit 17 of the store server 2 stores the received order data in the storage unit 13 and outputs the order to the kitchen terminal 5 from the output unit 16. The kitchen terminal 5 displays the names and quantities of the ordered dishes on its display screen. Subsequently, the control unit 17, as described above, combines pre-set survey items to create questionnaire items corresponding to the ordered food and beverages.

[0059] Next, when the customer finishes eating and drinking, and the store server 2 receives a payment request signal from the order terminal 3, the control unit 17 instructs the payment machine 4 to perform the payment process. At this time, the control unit 17 also stops the camera 6 from taking pictures. The image data captured by the camera 6 is transmitted to the store server 2 and temporarily stored in the storage unit 13. The payment machine 4, upon receiving the payment instruction, then performs the payment using the payment method specified by the customer from among various payment methods such as cash payment and card payment.

[0060] Next, the control unit 17 reads image data from the storage unit 13 and performs image analysis. This image analysis, for example, involves image matching between the ordered dishes and sample images of the dishes arranged on a table to identify the dishes, and further, to detect the position and size of the dishes on the table. At the same time, as mentioned above, the seating position of the customer whose ID was entered is identified, and a screen with a composition like that shown in Figure 4A is created. In the image analysis, as eating and drinking progresses, the captured images of the dishes are partially cropped, and the size of the dishes becomes smaller. The amount of change at this time is calculated, and each time the cropped partial image of the dish moves to the character image, the amount of food held by the character image is added. Thus, by analyzing the image data, it is possible to identify the customer by ID, the name of the ordered dish, the number of dishes ordered, and the amount of food consumed by each customer.

[0061] Next, the control unit 17 of the store server 2, similar to the questionnaire responses, converts the data based on the questionnaire items that use survey items other than customer feedback, as described above. The control unit 17 then sorts the converted questionnaire items into preference information classified into multiple categories to create new preference information. This new preference information is added to or updates the preference information for all users with an ID, and the preference information for each individual ID identified by image analysis.

[0062] [Fourth Information Gathering] Next, referring to the sequence diagram shown in Figure 7, we will explain the fourth information collection method, which involves combining image data of the customer captured by the information collection device 1 with receipt information contained in the receipt created during payment. Here, operations and actions equivalent to those described in the sequence diagrams shown in Figures 2 and 6 will be explained in a simplified manner.

[0063] First, the user terminal 7 is configured with a search application and an electronic receipt application, and the password and identification number (ID) are registered with the store server 2. Next, when multiple customers (users) enter the restaurant and sit down at the tables, the sensor detects them, and camera 6 begins recording. At this time, camera 6 adjusts its field of view (shooting area) so that all customers who have entered the restaurant are captured.

[0064] Next, the customer operates the order terminal 3 and enters their own ID. The entered ID is sent to the store server 2 for authentication. At this time, the camera 6 captures each customer entering their ID, and during the image analysis described later, the time of ID entry and the time of the photo are compared to identify the customer who entered their ID, and it is also possible to recognize the seating position of that customer.

[0065] Next, the customer selects food and drinks from the menu, adjusts the dishes and quantities to finalize their order. The order is sent from the order terminal 3 to the store server 2. The control unit 17 of the store server 2 stores the received order data in the storage unit 13 and outputs the order to the kitchen terminal 5 from the output unit 16. The kitchen terminal 5 displays the names and quantities of the ordered dishes on its display screen. Subsequently, the control unit 17 of the store server 2, as described above, combines pre-configured survey items to create questionnaire items corresponding to the ordered food and beverages. Next, when the customer finishes eating and drinking, and the store server 2 receives a payment request signal from the order terminal 3, the control unit 17 instructs the payment machine 4 to process the payment. At this time, the control unit 17 also instructs the camera 6 to stop taking pictures. The image data captured by the camera 6 is transmitted to the store server 2 and temporarily stored in the storage unit 13.

[0066] The payment machine 4 receives a payment instruction and performs payment according to the payment method specified by the customer. At this time, the payment machine 4 creates the aforementioned receipt information during the payment process and sends it to the store server 2. The customer may be given a paper receipt with the receipt information printed on it. The store server 2 stores the receipt information contained in the received electronic receipt in the receipt information storage unit 14. Based on the receipt information, the control unit 17 can estimate at least the user's time spent in the store, the level of congestion, the number of customers by time of day, the number of orders, and the ranking of sales of items, as well as the estimated sales amount.

[0067] Next, the control unit 17 reads image data from the storage unit 13 and performs image analysis. This image analysis, for example, identifies dishes arranged on a table using image matching processing, and further detects the position and size of the dishes on the table. At the same time, it identifies the seating position of the customer whose ID was entered as described above, and creates a screen with a composition as shown in Figure 4A. In the image analysis, as eating and drinking progresses, the captured images of the dishes are partially cropped, and the size of the dishes becomes smaller. The amount of change at this time is calculated, and each time the cropped partial image of the dish moves to the character image, the amount of dish held by the character image is added.

[0068] The control unit 17 analyzes the image data to estimate and quantify the dishes and quantities consumed by the customer identified by the ID input, and then converts this data into data based on the survey items, similar to the survey responses, as described above. Next, the control unit 17 sorts the converted survey items into preference information categorized into multiple categories. This new preference information is added to or updates the preference information for all users with an ID, as well as the preference information for each individual with an ID identified by image analysis.

[0069] Furthermore, by adding receipt information generated during the settlement process, time-related information (such as length of stay and busy times) and cost-related information can be added, providing users with more user-friendly preference information.

[0070] [Fifth Information Gathering] Next, referring to the sequence diagram shown in Figure 8, we will explain the fifth information collection method, which involves information collection from customers by the information collection device 1, specifically the collection of information from questionnaires, image data of customers, and receipt information contained in receipts created during payment. Here, operations and actions equivalent to those described in the sequence diagrams shown in Figures 2, 5, and 6 will be explained in a simplified manner.

[0071] First, the user terminal 7 is configured with a search application, an electronic receipt application, and an identification number (ID), and the password and ID are registered with the store server 2. Next, multiple customers (users) enter the restaurant and sit down at the tables. Sensors placed near the tables detect the people, and camera 6, positioned above the tables, begins recording. At this time, camera 6 adjusts its field of view (shooting area) so that all customers who have entered the restaurant are captured.

[0072] Next, the customer operates the order terminal 3 and enters their own ID, as described above in Figure 2. The entered ID is sent to the store server 2 for authentication. At this time, the camera 6 captures each customer entering their ID, and during the image analysis described later, the time of ID entry (clock) and the time of capture (clock) are compared to identify the customer who entered their ID, and it is possible to estimate or recognize the customer's seating position.

[0073] Next, the customer selects food and drinks from the menu, adjusts the dishes and quantities they want to order, and finalizes their order. The order is then sent from the order terminal 3 to the store server 2. The control unit 17 of the store server 2 stores the received order data in the storage unit 13 and places an order with the kitchen terminal 5. The kitchen terminal 5 displays the name and quantity of the ordered dishes. Subsequently, the control unit 17, as described above, creates a questionnaire corresponding to the ordered food and beverages by combining pre-set survey items.

[0074] Next, the control unit 17 of the store server 2 sends the questionnaire to the order terminal 3 when a predetermined time has elapsed or when it receives a payment request signal from the order terminal 3. The order terminal 3 displays the questionnaire as a pop-up on its display screen. The customer follows the operation guide for the questionnaire, completes the questionnaire, and sends it to the store server 2. The store server 2 stores the questionnaire answers in the storage unit 13.

[0075] Subsequently, the store server 2 checks whether it has already received a payment request signal from the order terminal 3. If it has already received the payment request signal, it instructs the payment machine 4 to perform the payment process. At this time, the control unit 17 instructs the camera 6 to stop taking pictures. The image data captured by the camera 6 is transmitted to the store server 2 and temporarily stored in the storage unit 13. Meanwhile, if the store server 2 has not received a payment request signal from the order terminal 3, it continues taking pictures with the camera 6 and remains in standby mode.

[0076] The payment machine 4 receives a payment instruction and performs payment according to the payment method specified by the customer. At this time, the payment machine 4 creates the aforementioned receipt information during the payment process and sends it to the store server 2. The customer may be given a paper receipt with the receipt information printed on it. The store server 2 stores the receipt information contained in the received electronic receipt in the receipt information storage unit 14. Based on the receipt information, the control unit 17 can estimate at least the user's time spent in the store, the level of congestion, the number of customers by time of day, the number of orders, and the ranking of sales of items, as well as the estimated sales amount.

[0077] Next, the control unit 17 reads the questionnaire responses from the storage unit 13, processes the questionnaire responses, and converts them into data as described above. The control unit 17 then sorts the converted questionnaire responses into preference information categorized into multiple categories. This new preference information is added to or updates the preference information for all users with an ID, and the preference information for each individual ID associated with the questionnaire responses. Images captured by the camera 6 can be used as attached images when this preference information is viewed.

[0078] Furthermore, the control unit 17 adds information from receipts generated during payment to the preference information obtained from customer surveys, thereby adding time information (such as length of stay and busy times) and cost information, making it possible to create preference information that is more user-friendly.

[0079] Furthermore, the image data may be analyzed using the aforementioned image analysis method. The image data is analyzed to recognize the ordered dishes and identify the seating location of the customer who entered their ID. A screen with a composition similar to that shown in Figure 4A may be created, and as the meal progresses, the captured image of the food may be partially cropped, showing the food becoming smaller, and the character image eating the cropped portion of the food.

[0080] Furthermore, even if not all customers respond to the questionnaire, image analysis of the image data can be used to obtain personal data such as who ate what within the group and the amount and number of items distributed, thus supplementing the questionnaire responses.

[0081] As described above, according to the first embodiment, personal data such as who ate what within the group and the amount and number of portions distributed can be collected. Furthermore, the method of answering the questionnaire involves cutting out a portion of the food on an image and serving it to a character (avatar), allowing customers to answer with a simple and unique operation.

[0082] <Second Embodiment> Referring to the drawings, an information collection device for constructing the information collection system according to the second embodiment will be described. Hereinafter, referring to Figure 9, the network-based information collection system 300 according to the second embodiment will be described. In this embodiment, components that are the same as those in the first embodiment described above are given the same reference numerals, and their descriptions are simplified.

[0083] In this embodiment, the information gathering system 300, when collecting information through a questionnaire survey, displays multiple users (so-called avatars), each with an assigned ID and characterized, along with the ordered dishes (images) on the screen. On this screen, by dragging and dropping part or all of the image area of ​​the dishes onto or near the characterized users, the system associates the dishes with individual users. By performing the same operation for multiple users, personal data on the dishes consumed by each user can be created.

[0084] This information gathering method allows for the collection of individual data even within groups of multiple people, enabling the realization of preference information using surveys and other collected data. By managing various types of information, including preference information, on a server system connected to the network, it can be searched from external user terminals 7.

[0085] The information gathering system 300 consists of an information gathering device 1, which comprises a store server 2, an order terminal, a payment machine (POS terminal) 4, a kitchen terminal 5, and a camera 6, all connected via a network 41 such as the Internet, and a server system 31. In addition, a user terminal 7 can communicate wirelessly with the information gathering system 300 using a base station 42 provided on the network 41.

[0086] The store server 2 is, for example, a personal computer and includes a communication unit 12 equipped with a wireless antenna 11, a storage unit 13, an image processing unit 15, an output unit 16, a control unit (processor) 17, and a communication interface 19. Here, the store server 2 functions as a control device. These components are connected by a bus 18 which serves as a transmission path. The wireless antenna 11, communication unit 12, storage unit 13, image processing unit 15, output unit 16, control unit (processor) 17, and bus 18 are as described in the first embodiment. However, in this embodiment, the control unit 17 has the function of transmitting the created preference information and electronic receipt information from the payment machine (POS terminal) 4 to the server system 31.

[0087] The server system 31 includes a central server 32 and a receipt information server 33. The central server 32 stores preference information and other data for numerous registered users. It also has the functionality to respond to search applications configured on user terminals 7.

[0088] The receipt information server 33 stores receipt information transmitted from the store server 2 and printed on the electronic receipt created when payment is made by the payment machine (POS terminal) 4. It can also receive and store receipt information read from a paper receipt by the user terminal 7. In this embodiment, the receipt information includes at least the date and time of payment, the store name, the name of the food and beverage (item name), the quantity, the amount (unit price, total amount, and tax), the payment method, and the name of the person who made the payment. The receipt information server 33 has the function to respond to the electronic receipt application set up on the user terminal 7. The control unit 17 can estimate, based on receipt information, at least the user's time spent in the store, the level of congestion, the number of customers by time of day, the number of orders, and the ranking of items in sales, and can also estimate the estimated sales amount. The time spent in the store can be calculated from the time from the initial order time to the time of payment processing.

[0089] [Sixth Information Gathering] Next, referring to the sequence diagram shown in Figure 10, we will explain the sixth information gathering method, which is the collection of information from customers by the information gathering system 300, i.e., a questionnaire survey. Here, we will explain an example in which a user enters a restaurant where the system is set up for ordering using the order terminal 3.

[0090] First, the user terminal 7 downloads a search application and an electronic receipt application in advance, and registers a password and identification number (ID) with the server system 31 (central server 32, receipt information server 33) and the store server 2. Alternatively, the user may register a password and identification number (ID) unique to the store. By using such an identification number (ID), the store server 2 or the server system 31 can identify the user terminal 7. The server system 31 includes the central server 32 and the receipt information server 33.

[0091] Next, the user operates the order terminal 3 in the store and enters their own ID. This ID can be entered by operating the keys on the order terminal 3, by displaying an identification code such as a barcode on the display screen of the user terminal 7 and having the order terminal 3 read it, or by accessing the store server 2 from the user terminal 7 to recognize the ID. The entered ID is sent to the store server 2 for authentication processing.

[0092] Next, the customer operates the order terminal 3 to select and set food and drinks from the displayed menu. For example, they select and set one pizza, one paella, and one loaf of bread, along with two drinks, as shown in Figure 4A above. They then adjust the order by adding or removing other dishes.

[0093] Once the food and beverages to be ordered (such as the name of the dish and the quantity) have been decided, the customer operates the confirmation key on the order terminal 3 to send the decided food and beverages as order data from the communication unit 21 to the store server 2. At this time, the order data will include, for example, the ID information of two customers, as will be described later. When the control unit 17 of the store server 2 receives the order data via the communication unit 12, it stores the order data in the storage unit 13 and also sends the order data via the output unit 16 to the kitchen terminal 5 installed in the kitchen. The kitchen terminal 5 receives the order data and displays the name of the ordered dish, the quantity, etc. on its display screen.

[0094] Subsequently, the control unit 17 of the store server 2 creates a questionnaire corresponding to the ordered food and beverage. For example, for pizza, the questionnaire items would include the degree of doneness, size, and taste, while for pasta, the questionnaire items would include the degree of doneness, quantity, and taste. These questionnaire items are pre-stored in the memory unit 13 for each dish name corresponding to the menu, and are read from the control unit 17 based on the entered order data (dish name), and a questionnaire suitable for the order data is created as needed.

[0095] Next, the control unit 17 of the store server 2 sends the created questionnaire to the order terminal 3 when a predetermined period of time has elapsed, which is assumed to be when the customer has finished eating and drinking, or when it receives a payment request signal from the order terminal 3.

[0096] Upon receiving the questionnaire, the order terminal 3 displays the questionnaire as a pop-up on its screen, including the questionnaire content and an operation guide showing the operating procedures. The questionnaire content may include, for example, the quantity (or amount), taste (mild, strong, salty, sweet, etc.), condition (hard, soft, size (large or small), hot, lukewarm, etc.), and appearance for each dish. Various survey items may be in a selectable format using checkboxes or similar methods.

[0097] The customer follows the operation guide and answers the questionnaire as described below to create their questionnaire response. The completed questionnaire response is sent to the store server 2 by touching the "Complete" button displayed on the order terminal 3's screen.

[0098] The store server 2 receives the survey responses, and the control unit 17 processes the survey responses. Based on the survey responses, the control unit 17 categorizes the dish name into the dish genre, quantity (or amount), taste (mild, strong, salty, sweet, etc.), condition (hard, soft, size (large or small), hot, lukewarm, etc.), and appearance, and converts them into numerical data by comparing them with pre-set thresholds, which can be used, for example, for ranking. For example, if the dish name is "pizza," it will be categorized into the wheat-Italian-Western cuisine genre. Similarly, if the dish name is "paella," it will be categorized into the rice-Spanish-Western cuisine genre. These classifications are, of course, not limited to those above, and may be classified according to the index for searching. In addition, the quantity is assigned to each user terminal 7 ID and is not necessarily a single value, but is digitized as a fraction or percentage such as 2 / 3 or 1 / 2.

[0099] Next, the control unit 17 sorts the digitized questionnaire responses into preference information categorized into a predetermined number of categories to create new preference information. This new preference information is transmitted to the center server 32 via the network 41. The center server 32 adds the new preference information to the previously accumulated preference information or updates the list. At this time, it may perform both adding to or updating the preference information formed for all users with an ID, and adding to or updating the preference information for each individual managed by the ID associated with the questionnaire response.

[0100] Subsequently, the store server 2 checks whether it has already received a payment request signal from the order terminal 3. If it has already received the payment request signal, it instructs the payment machine 4 (POS terminal) to perform the payment process. If the store server 2 has not yet received a payment request signal from the order terminal 3, it remains in standby mode. Upon receiving the payment instruction, the payment machine 4 performs the payment using the payment method specified by the customer from among various payment methods such as cash payment and card payment. During the payment process, the payment machine 4 creates receipt information (electronic receipt) consisting of various payment information and reports it to the store server 2. The customer may be given a paper receipt with the receipt information printed on it. As mentioned above, the receipt information includes at least the date and time of payment, store name, food and beverage name, quantity, amount (unit price, total amount and tax), payment method, and name of the person in charge of payment.

[0101] Next, we will explain how to search for preference information using user terminal 7. First, the user terminal 7 launches a search application and accesses the central server 32 from the nearest base station 42 to begin searching for preference information. The user terminal 7 selects the desired food or drink and sends a search command to the central server 32. The central server 32 receives the search command and reads the relevant information from the preference information stored in the server's memory. The retrieved search results are sent to the user terminal 7 via the network 41. The user terminal 7 receives the search results and displays them. These search results may include, for example, the results of surveys answered by other users, and may include evaluations and comments about the food, etc.

[0102] The method for answering the questionnaire in this embodiment is the same as in the first embodiment described above. Using the screen display examples shown in Figures 4A to 4D, the amount of food eaten can be easily set by dragging and dropping part or all of the image representing the food, as if the customer were feeding the amount of food to the character image 101. [Information gathering #7] Next, referring to the sequence diagram shown in Figure 11, we will explain the seventh information gathering method, which adds receipt information from the receipts created after payment, in addition to the customer survey conducted by the store server 2. Here, operations and actions equivalent to those described in the sequence diagram of Figure 10 will be explained in a simplified manner.

[0103] First, the user terminal 7 downloads a search application and an electronic receipt application in advance, and registers a password and identification number (ID) with the server system 31 (central server 32, receipt information server 33) and the store server 2.

[0104] Next, after entering the restaurant, multiple customers (users) operate the order terminal 3 and enter their customer IDs. The entered IDs are sent to the store server 2 for authentication. Subsequently, customers select food and drinks from the menu, adjust the dishes and quantities they wish to order, and finalize their order. The food and drink order is then sent from the order terminal 3 to the store server 2.

[0105] The control unit 17 of the store server 2 stores the received order data in the storage unit 13 and places an order with the kitchen terminal 5. The kitchen terminal 5 displays the name and quantity of the ordered dishes. Subsequently, the control unit 17, as described above, creates a questionnaire corresponding to the ordered food and beverages by combining pre-set survey items.

[0106] Next, the control unit 17 of the store server 2 sends the questionnaire to the order terminal 3 when a predetermined time has elapsed or when it receives a payment request signal from the order terminal 3. The order terminal 3 displays the questionnaire as a pop-up on its display screen. The customer follows the operation guide for the questionnaire, creates their answers using the procedure shown in Figures 4A to 4D, and sends them to the store server 2. The store server 2 receives the questionnaire answers, processes them using the control unit 17, and converts them into data as described above.

[0107] Next, the control unit 17 sorts the digitized questionnaire responses into preference information classified into a predetermined number of categories to create new preference information. This new preference information is transmitted to the center server 32 via the network 41. The center server 32 adds the new preference information to the previously accumulated preference information or updates the list. In this example, it is also possible to perform both adding to or updating the preference information formed for all users with an ID, and adding to or updating the preference information for each individual managed by the ID associated with the questionnaire response.

[0108] Subsequently, the store server 2 checks whether it has already received a payment request signal from the order terminal 3. If it has already received the payment request signal, it instructs the payment machine 4 to perform the payment process. If the store server 2 has not yet received a payment request signal from the order terminal 3, it remains in standby mode.

[0109] The payment machine 4, upon receiving a payment instruction, performs payment using one of various payment methods, such as cash or card payment, as specified by the customer. At this time, the payment machine 4 creates a receipt (electronic receipt) containing various payment information during the payment process and sends it to the store server 2. The store server 2 extracts the payment date and time, store name, food and beverage names, quantity, amount (unit price, total amount and tax), payment method, and name of the person in charge of payment from the electronic receipt as receipt information. This receipt information is stored in the storage unit 13 of the store server 2 and is also transmitted to the receipt information server 33 via the network 41, where it is stored as receipt information associated with an ID in a designated storage unit. The receipt information server 33 has a function to respond to electronic receipt applications. When a request for an electronic receipt is received from the user terminal 7, the receipt information server 33 creates an electronic receipt based on the stored receipt information and sends it to the user terminal 7.

[0110] The control unit 17 estimates, based on receipt information, at least the user's time spent in the store, the level of congestion, the number of customers by time of day, the number of orders, and the ranking of sales of items, and can also estimate the estimated sales amount. By adding receipt information created through payment to preference information obtained from customer surveys, the control unit 17 adds time information (such as time spent in the store and busy times) and cost information, creating preference information that is easier for users to use.

[0111] Furthermore, the seventh information collection system 300 can access the server system 31 from the user terminal 7, read the electronic receipts stored in the receipt information server 33 to the user terminal 7, and use them for viewing, printing, and other purposes.

[0112] [Eighth Information Gathering] Next, referring to the sequence diagram shown in Figure 12, we will explain the eighth information collection method, which involves collecting information using image data of customers captured by the information collection system 300. Here, operations and actions equivalent to those described in the sequence diagram shown in Figure 10 will be explained in a simplified manner.

[0113] First, the user terminal 7 downloads a search application and an electronic receipt application in advance, and registers a password and identification number (ID) with the server system 31 (central server 32, receipt information server 33) and the store server 2.

[0114] Next, multiple customers (users) enter the restaurant and take seats in the seating area arranged around the table. Sensors placed near the table detect the people, and camera 6, positioned above the table, begins recording. At this time, camera 6 adjusts its field of view (shooting area) so that all customers who have entered the restaurant are captured.

[0115] Next, the customer operates the order terminal 3 and enters their own ID, as described above in Figure 10. The entered ID is sent to the store server 2 for authentication. At this time, the camera 6 captures each customer entering their ID, and during the image analysis described later, the time of ID entry (clock) and the time of capture (clock) are compared to identify the customer who entered their ID, and it is possible to estimate or recognize the customer's seating position.

[0116] Next, the customer selects food and drinks from the menu, adjusts the dishes and quantities to finalize their order. The order is sent from the order terminal 3 to the store server 2. The control unit 17 of the store server 2 stores the received order data in the storage unit 13 and outputs the order to the kitchen terminal 5 from the output unit 16. The kitchen terminal 5 displays the names and quantities of the ordered dishes on its display screen. Subsequently, the control unit 17, as described above, combines pre-set survey items to create questionnaire items corresponding to the ordered food and drinks.

[0117] Next, when the customer finishes eating and drinking, and the store server 2 receives a payment request signal from the order terminal 3, the control unit 17 instructs the payment machine 4 to perform the payment process. At this time, the control unit 17 also instructs the camera 6 to stop taking pictures. The image data captured by the camera 6 is transmitted to the store server 2 and temporarily stored in the storage unit 13.

[0118] Furthermore, upon receiving a payment instruction, the payment machine 4 performs the payment according to the payment method specified by the customer. At this time, the payment machine 4 creates transaction information containing various payment details and transmits it to the store server 2. The store server 2 extracts the aforementioned receipt information included in the transaction information and transmits it to the receipt information server 33 via the network 41, where it is stored as receipt information associated with an ID in a designated storage unit.

[0119] Next, the control unit 17 reads image data from the storage unit 13 and performs image analysis. This image analysis, for example, involves image matching between the ordered dishes and sample images of the dishes arranged on a table to identify the dishes, and further, to detect the position and size of the dishes on the table. At the same time, as mentioned above, the seating position of the customer whose ID was entered is identified, and a screen with a composition like that shown in Figure 4A is created. In the image analysis, as eating and drinking progresses, the captured images of the dishes are partially cropped, and the size of the dishes becomes smaller. The amount of change at this time is calculated, and each time the cropped partial image of the dish moves to the character image, the amount of food held by the character image is added. Thus, by analyzing the image data, it is possible to identify the customer by ID, the name of the ordered dish, the number of dishes ordered, and the amount of food consumed by each customer.

[0120] Next, the control unit 17 of the store server 2 digitizes the survey items as described above, similar to the survey responses. The control unit 17 then sorts the digitized survey items into preference information categorized into multiple categories to create new preference information. This new preference information is transmitted to the center server 32 via the network 41. The center server 32 adds the new preference information to the previously accumulated preference information or updates the list. At this time, it may perform both adding to or updating the preference information formed for all users with an ID, and adding to or updating the preference information for each individual managed by the ID associated with the survey response.

[0121] [9th Information Gathering] Next, referring to the sequence diagram shown in Figure 13, we will explain the ninth information collection method, which involves combining image data of the customer captured by the information collection device 1 with receipt information contained in the receipt created during payment. Here, operations and actions equivalent to those described in the sequence diagrams shown in Figures 11 and 12 will be explained in a simplified manner.

[0122] First, the user terminal 7 downloads a search application and an electronic receipt application in advance, and registers a password and identification number (ID) with the server system 31 (central server 32, receipt information server 33) and the store server 2. Next, when multiple customers (users) enter the restaurant and sit down at the tables, the sensor detects them, and camera 6 begins recording. At this time, camera 6 adjusts its field of view (shooting area) so that all customers who have entered the restaurant are captured.

[0123] Next, the customer operates the order terminal 3 and enters their own ID. The entered ID is sent to the store server 2 for authentication. At this time, the camera 6 captures each customer entering their ID, and during the image analysis described later, the time of ID entry and the time of the photo are compared to identify the customer who entered their ID, and it is also possible to recognize the seating position of that customer.

[0124] Next, the customer selects food and drinks from the menu, adjusts the dishes and quantities to finalize their order. The order is sent from the order terminal 3 to the store server 2. The control unit 17 of the store server 2 stores the received order data in the storage unit 13 and outputs the order to the kitchen terminal 5 from the output unit 16. The kitchen terminal 5 displays the names and quantities of the ordered dishes on its display screen.

[0125] Subsequently, the control unit 17 of the store server 2, as described above, combines pre-configured survey items to create questionnaire items corresponding to the ordered food and beverages. Next, when the customer finishes eating and drinking, and the store server 2 receives a payment request signal from the order terminal 3, the control unit 17 instructs the payment machine 4 to process the payment. At this time, the control unit 17 also instructs the camera 6 to stop taking pictures. The image data captured by the camera 6 is transmitted to the store server 2 and temporarily stored in the storage unit 13.

[0126] The payment machine 4 receives a payment instruction and performs payment according to the payment method specified by the customer. At this time, the payment machine 4 creates a receipt (electronic receipt) containing various payment information during the payment process and sends it to the store server 2. The store server 2 extracts the payment date and time, store name, food and beverage name, quantity, amount (unit price, total amount and tax), payment method, and name of the person in charge of payment from the transaction information as receipt information. This receipt information is stored in the storage unit 13 of the store server 2 and is also sent to the receipt information server 33 via the network 41, where it is stored as receipt information associated with an ID in a designated storage unit. The receipt information server 33 has a function to respond to electronic receipt applications. When a request for an electronic receipt is received from the user terminal 7, the receipt information server 33 creates an electronic receipt based on the stored receipt information and sends it to the user terminal 7.

[0127] Next, the control unit 17 reads image data from the storage unit 13 and performs image analysis. This image analysis, for example, identifies dishes arranged on a table using image matching processing, and further detects the position and size of the dishes on the table. At the same time, it identifies the seating position of the customer whose ID was entered as described above, and creates a screen with a composition as shown in Figure 4A. In the image analysis, as eating and drinking progresses, the captured images of the dishes are partially cropped, and the size of the dishes becomes smaller. The amount of change at this time is calculated, and each time the cropped partial image of the dish moves to the character image, the amount of dish held by the character image is added.

[0128] The control unit 17 analyzes the image data and estimates the dishes and quantities consumed by the customer identified by ID input, quantifying the data and converting it into data based on the survey items, similar to the survey responses. Next, the control unit 17 sorts the converted survey items into preference information categorized into multiple categories.

[0129] The control unit 17 adds receipt information generated from payment to preference information obtained from customer surveys, thereby adding time information (such as length of stay and busy times) and cost information, creating preference information that is more user-friendly. This preference information is added to or updates the preference information list for all users with an ID, as well as the preference information list for each individual ID identified by image analysis.

[0130] The control unit 17 estimates, based on receipt information, at least the user's time spent in the store, the level of congestion, the number of customers by time of day, the number of orders, and the ranking of sales of items, and can also estimate the estimated sales amount. Furthermore, by adding receipt information created through settlement, time information (such as time spent in the store and busy times) and cost information are added, making it possible to provide more user-friendly preference information.

[0131] [Information gathering #10] Next, referring to the sequence diagram shown in Figure 14, we will explain the tenth information collection method, which involves information collection from customers by the information collection device 1, that is, information collection combining questionnaire surveys, image data of customers, and receipt information contained in receipts created through payment. Here, operations and actions equivalent to those described in the sequence diagrams shown in Figures 10 to 12 will be explained in a simplified manner.

[0132] First, the user terminal 7 downloads a search application and an electronic receipt application in advance, and registers a password and identification number (ID) with the server system 31 (central server 32, receipt information server 33) and the store server 2.

[0133] Next, multiple customers (users) enter the restaurant and sit down at the tables. Sensors placed near the tables detect the people, and camera 6, positioned above the tables, begins recording. At this time, camera 6 adjusts its field of view (shooting area) so that all customers who have entered the restaurant are captured.

[0134] Next, the customer operates the order terminal 3 and enters their own ID. The entered ID is sent to the store server 2 for authentication. At this time, the camera 6 captures each customer entering their ID, and during the image analysis described later, the time of ID entry (clock) and the time of the image capture (clock) are compared to identify the customer who entered their ID, and it is possible to estimate or recognize that customer's seating position.

[0135] Next, the customer selects food and drinks from the menu, adjusts the dishes and quantities they want to order, and finalizes their order. The order is then sent from the order terminal 3 to the store server 2. The control unit 17 of the store server 2 stores the received order data in the storage unit 13 and places an order with the kitchen terminal 5. The kitchen terminal 5 displays the name and quantity of the ordered dishes. Subsequently, the control unit 17, as described above, creates a questionnaire corresponding to the ordered food and beverages by combining pre-set survey items.

[0136] Next, the control unit 17 of the store server 2 sends the questionnaire to the order terminal 3 when a predetermined time has elapsed or when it receives a payment request signal from the order terminal 3. The order terminal 3 displays the questionnaire as a pop-up on its display screen. The customer follows the operation guide for the questionnaire, completes the questionnaire, and sends it to the store server 2. The store server 2 stores the questionnaire answers in the storage unit 13.

[0137] Subsequently, the store server 2 checks whether it has already received a payment request signal from the order terminal 3. If it has already received the payment request signal, it instructs the payment machine 4 to perform the payment process. At this time, the control unit 17 instructs the camera 6 to stop taking pictures. The image data captured by the camera 6 is transmitted to the store server 2 and temporarily stored in the storage unit 13. Meanwhile, if the store server 2 has not received a payment request signal from the order terminal 3, it continues taking pictures with the camera 6 and remains in standby mode.

[0138] The payment machine 4 receives a payment instruction and performs payment according to the payment method specified by the customer. At this time, the payment machine 4 creates a receipt (electronic receipt) containing various payment information during the payment process and sends it to the store server 2. The store server 2 extracts the aforementioned receipt information contained in the electronic receipt. This receipt information is stored in the storage unit 13 of the store server 2 and is also transmitted to the receipt information server 33 via the network 41. The receipt information server 33 associates the ID registered in the user terminal 7 with the received receipt information and stores it in a predetermined storage unit. When the store settles the payment, the customer may be given a paper receipt with the receipt information printed on it. The receipt information server 33 has a function to respond to electronic receipt applications. When the user terminal 7 requests an electronic receipt, the receipt information server 33 creates an electronic receipt based on the stored receipt information and sends it to the user terminal 7.

[0139] Next, the control unit 17 reads the questionnaire responses from the storage unit 13, processes the questionnaire responses, and converts them into data as described above. The control unit 17 then sorts the converted questionnaire responses into preference information classified into multiple categories to create new preference information. This new preference information is transmitted to the center server 32 via the network 41. The center server 32 adds the new preference information to the previously accumulated preference information or updates the list. At this time, it may perform both adding to or updating the preference information formed for all users with an ID, and adding to or updating the preference information for each individual managed by the ID associated with the questionnaire response. Images captured by the camera 6 can be used as attached images when this preference information is viewed.

[0140] Based on receipt information, the control unit 17 can estimate at least the user's time spent in the store, the level of congestion, the number of customers by time of day, the number of orders, and the ranking of sales of items, and can also estimate the estimated sales amount. Furthermore, by adding receipt information created through payment to preference information obtained from customer surveys, the control unit 17 can add time information (such as time spent in the store and busy times) and cost information, thereby creating preference information that is more user-friendly.

[0141] Furthermore, the image data may be analyzed using the aforementioned image analysis method. The image data is analyzed to recognize the ordered dishes and identify the seating location of the customer who entered their ID. A screen with a composition similar to that shown in Figure 4A may be created, and as the meal progresses, the captured image of the food may be partially cropped, showing the food becoming smaller, and the character image eating the cropped portion of the food.

[0142] Furthermore, even if not all customers respond to the questionnaire, image analysis of the image data can be used to obtain personal data such as who ate what within the group and the amount and number of items distributed, thus supplementing the questionnaire responses.

[0143] As described above, according to the second embodiment, it is possible to serve a large number of customers using a network and a central server, and personal data such as who ate what within a group of people, and the amount and number of portions distributed, can be collected. Furthermore, the method of answering the questionnaire involves cutting out a portion of the food on an image and serving it to a character (so-called avatar), allowing customers to answer with a simple and unique operation.

[0144] As mentioned above, by analyzing image data, the seating position of a customer who has entered their ID can be identified, and a screen with a composition like that shown in Figure 4A can be created. In the image analysis, as the customer eats, parts of the food image are cropped, and the size of the food decreases. The amount of change at this time is calculated, and each time the cropped portion of the food image moves to the character image, the amount of food held by the character image is added. In this way, by analyzing image data, it is possible to identify customers by ID, find the names of the dishes ordered, the number of dishes ordered, and the amount of food consumed by each customer. Therefore, although it is not possible to survey customer feedback or comments, it is possible to save the effort of having customers create questionnaires.

[0145] Although several embodiments have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. The original claims of this application are included below. [C1] An order terminal that displays a character image associated with user-specific identification information and an image of an item on a screen, and allows the image of the item to be moved on or near the display area of ​​the character image on the screen, When the image of the item is moved to or near the display area for each character image, a control unit generates preference information for the item for each user corresponding to the character image. An information gathering system equipped with the following features. [C2] The aforementioned information gathering system further, A storage unit that stores the aforementioned identification information and the aforementioned preference information in association, The information collection system according to C1, which transmits the preference information stored in the memory unit to an external user terminal on which the identification information is set. [C3] The aforementioned information gathering system further, The payment machine is equipped with a system that generates receipt information, including the total amount, to be added to the preference information when payment processing is performed for the aforementioned items. The information gathering system described in C1. [C4] The aforementioned information gathering system is equipped with a camera, The information gathering system according to C1, wherein the control unit estimates the location of the user who input identification information from the image captured by the camera, and associates the user with the character image. [C5] An order terminal that displays a character image associated with user-specific identification information alongside an image of an item on a screen, and allows the image of the item to be moved on or near the display area of ​​the character image on the screen, A control device that, when the image of the item is moved onto or near the display area for each character image, generates user preference information corresponding to the character image and transmits the identification information and the preference information in association; A server equipped with a memory unit that stores user preference information, An information gathering system equipped with the following features. [C6] The aforementioned information gathering system further, A payment machine that generates receipt information including the total amount to be added to the preference information when payment processing is performed for the aforementioned items, A receipt information server that stores the receipt information generated by the payment machine, An information gathering system described in C5, which is equipped with the following features. [Explanation of symbols]

[0146] 1... Information gathering device, 2... Store server, 3... Order terminal, 4... Payment machine (POS terminal), 5... Kitchen terminal, 6... Camera, 7... User terminal, 11, 22... Wireless antenna, 12, 21... Communication unit, 13... Memory unit, 14... Receipt information memory unit, 15... Image processing unit, 16... Output unit, 17... Control unit, 18, 27... Bus, 19... Communication interface, 23... Input unit, 24... Memory, 25... Processor, 26... Display unit, 31... Server system, 32... Center server, 33... Receipt information server, 41... Network, 42... Base station, 101, 102... Character image, 201... Pizza, 202... Paella, 203... Bread, 204, 205... Drinks, 300... Information gathering system.

Claims

1. An order terminal that displays a character image associated with user-specific identification information and an image of an item on a screen, allows the user to move part or all of the image area of ​​the item on or near the display area of ​​the character image, and transmits personal data indicating the quantity of the item distributed to each user based on such movement operation. A control unit that generates preference information for the item for each user corresponding to the character image, based on the personal data transmitted from the order terminal, An information gathering system equipped with the following features.

2. The aforementioned information gathering system further, A storage unit that stores the aforementioned identification information and the aforementioned preference information in association, The information collection system according to claim 1, wherein the preference information stored in the storage unit is transmitted to an external user terminal on which the identification information is set.

3. The aforementioned information gathering system further, The payment machine is equipped with a system that generates receipt information, including the total amount, to be added to the preference information when payment processing is performed for the aforementioned items. The information collection system according to claim 1.

4. The aforementioned information gathering system is equipped with a camera, The information gathering system according to claim 1, wherein the control unit estimates the location of the user who input the identification information from the image captured by the camera, and associates the user with the character image.

5. An order terminal that displays a character image associated with user-specific identification information and an image of an item on a screen, allows the user to move part or all of the image area of ​​the item on or near the display area of ​​the character image, and transmits personal data indicating the quantity of the item distributed to each user based on such movement operation. A control device that generates user preference information corresponding to the character image based on the personal data transmitted from the order terminal, and transmits the identification information and the preference information in association; A server equipped with a memory unit for storing the user's preference information, An information gathering system equipped with the following features.

6. The aforementioned information gathering system further, A payment machine that generates receipt information including the total amount to be added to the preference information when payment processing is performed for the aforementioned items, A receipt information server that stores the receipt information generated by the payment machine, The information gathering system according to claim 5, comprising the above.

Images