Multi-screen interaction system for restaurant and control method thereof

Abstract

The invention provides a multi-screen interaction system for a restaurant and a control method of the multi-screen interaction system. The multi-screen interaction system comprises at least one mobile terminal with a touch type first display screen; the at least one second display screen is mounted in the dining space of the restaurant; and a server communicatively coupled with the mobile terminal and the second display screen, wherein the mobile terminal is configured to: present menu information to a customer on the first display screen; detecting an interaction action executed by the customer on the first display screen; identifying a first interaction event based on the detected interaction action; and sending the identified first interaction event to a server; the server is configured to: in response to a received first interaction event, determine a display mode of a second display screen and display content corresponding to the first interaction event on the second display screen, wherein the display mode of the second display screen at least comprises a first display mode for displaying the same content as the first display screen and a second display mode for displaying different contents.

Description

Technical Field

[0001] This invention relates to the field of digital technology, and more specifically, to a multi-screen interactive system for restaurants and its control method. Background Technology

[0002] With the rapid development of the social economy and the continuous improvement of people's living standards, the catering industry has become a highly competitive market. When choosing a restaurant, customers not only pay attention to the taste and quality of the food, but also increasingly value the overall dining experience. Customers are increasingly inclined to seek unique and personalized dining experiences; they hope to relax and enjoy their time in a comfortable, distinctive, and creative environment while enjoying delicious food. This experience includes, but is not limited to, the restaurant's ambiance, service quality, and the convenience of the dining process.

[0003] However, most restaurants on the market still fall short in these aspects, failing to meet the increasingly diverse needs of customers. Restaurant ambiance often lacks innovation and personalization, resulting in a similar dining atmosphere across different restaurants, lacking appeal. Furthermore, traditional ordering methods rely heavily on paper menus or verbal explanations from waiters. This approach is not only inefficient but also lacks intuitive information delivery, making it difficult for customers to quickly and comprehensively understand the menu, thus impacting ordering efficiency and customer satisfaction. While some restaurants have begun to introduce intelligent equipment, such as self-service ordering machines and intelligent recommendation systems, these devices and systems often have limited functionality, fail to achieve effective integration, and offer inconsistent user experiences, failing to truly enhance the overall dining experience.

[0004] Furthermore, the traditional ordering process is quite cumbersome for groups of people. Generally, only one person (usually the host) can order, and everyone else agrees. In this case, it is very inconvenient to ask for the opinions of other diners. Multiple menus or passing them around are needed, and sometimes the person taking the order often needs to describe the dishes. Finally, it is necessary to reach a consensus among everyone, which is very time-consuming and inconvenient. Summary of the Invention

[0005] This summary is provided to introduce, in a simplified form, some concepts that will be further described in the following detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0006] To address the problems existing in the prior art, this disclosure provides a solution based on multi-screen interaction, bringing customers a brand-new, unique, and personalized ordering and dining experience.

[0007] According to a first aspect of this disclosure, a multi-screen interactive system for a restaurant is provided, the multi-screen interactive system comprising: at least one mobile terminal, each mobile terminal having a first display screen, wherein the first display screen is a touch display screen; at least one second display screen installed in the dining space of the restaurant; and a server, the server being communicatively coupled to the at least one mobile terminal and the at least one second display screen, wherein the mobile terminal is configured to: present menu information to a customer on the first display screen; detect an interactive action performed by the customer on the first display screen; identify a first interactive event based on the detected interactive action; and send the identified first interactive event to the server; wherein the server is configured to: in response to receiving a first interactive event from the mobile terminal, determine a display mode of the second display screen and display content corresponding to the first interactive event on the second display screen, wherein the display mode of the second display screen includes at least a first display mode displaying the same content as the first display screen and a second display mode displaying different content than the first display screen.

[0008] According to a further embodiment of this disclosure, the first interactive event is a customer request to view details of a menu item contained in the menu information or to select a menu item, and the server is configured to display content corresponding to the menu item on a second display screen.

[0009] According to further embodiments of this disclosure, menu items include one or more of the following: food, setting, service, and merchandise; the content corresponding to the menu items includes one or more of the following: raw materials, preparation process, cultural background, customer reviews, and availability related to the food; audio or video corresponding to the setting; audio or video corresponding to the service; and introductions and sales information related to the merchandise.

[0010] According to a further embodiment of this disclosure, the multi-screen interactive system further includes: at least one environmental sensing device installed in the dining space of a restaurant, and the server is further configured to: identify a second interactive event based on environmental sensing of the dining space by the at least one environmental sensing device; and in response to the identified second interactive event, determine the display mode of a second display screen and display content corresponding to the second interactive event on the second display screen.

[0011] According to further embodiments of this disclosure, the second interaction event includes one or more of the following: customer actions, voice and facial expressions, food served, and dining progress.

[0012] According to a further embodiment of this disclosure, the first interactive event is that a customer initiates a joint order, and the server is configured to: in response to the first interactive event, display the joint order interface on a second display screen.

[0013] According to a further embodiment of this disclosure, in response to entering a shared ordering interface, the server is further configured to: identify the selection intentions of multiple customers in the dining space regarding food based on environmental sensing data provided by at least one environmental sensing device, and determine the food selection result based on the identified selection intentions.

[0014] According to a further embodiment of this disclosure, the intention of multiple customers in a dining space to select food is identified based on one or more of the following information obtained from environmental sensing data: the customer's identity, gestures, gaze, and facial expressions.

[0015] According to a further embodiment of this disclosure, identifying the selection intentions of multiple customers in a dining space regarding food, and determining the food selection result based on the identified selection intentions, further includes: identifying the primary customer among the multiple customers, and assigning a higher weight to the primary customer's selection intentions than that of other customers; and weighting the food selection intentions of the multiple customers to determine the food selection result.

[0016] According to a second aspect of this disclosure, a control method for a multi-screen interactive system in a restaurant is provided. The control method includes: presenting menu information to a customer on a first display screen, wherein the first display screen is a touch screen of a mobile terminal; detecting an interactive action performed by the customer on the first display screen; identifying a first interactive event based on the detected interactive action; determining a display mode of a second display screen based on the identified first interactive event and displaying content corresponding to the first interactive event on the second display screen, wherein the second display screen is installed in the dining space of the restaurant, and the display mode of the second display screen includes at least a first display mode displaying the same content as the first display screen and a second display mode displaying different content than the first display screen.

[0017] According to a further embodiment of this disclosure, the first interactive event is a customer request to view details of a menu item contained in the menu information or to select a menu item, and displaying content corresponding to the first interactive event on the second display screen further includes displaying content corresponding to the menu item on the second display screen.

[0018] According to further embodiments of this disclosure, menu items include one or more of the following: food, setting, service, and merchandise; content associated with menu items includes one or more of the following: raw materials, preparation process, cultural background, customer reviews, and availability related to food; audio or video corresponding to the setting; audio or video corresponding to the service; and descriptions and sales information related to merchandise.

[0019] According to a further embodiment of this disclosure, the control method further includes: identifying a second interactive event based on environmental sensing of the dining space by at least one environmental sensing device installed in the dining space of the restaurant; and in response to the identified second interactive event, determining a display mode of a second display screen and displaying content corresponding to the second interactive event on the second display screen.

[0020] According to further embodiments of this disclosure, the second interaction event includes one or more of the following: customer actions, voice and facial expressions, food served, and dining progress.

[0021] According to a further embodiment of this disclosure, the first interactive event is that a customer initiates a joint ordering, and the control method further includes: in response to the first interactive event, displaying a joint ordering interface on a second display screen.

[0022] According to a further embodiment of the present disclosure, the control method further includes: in response to entering a shared ordering interface, identifying the selection intentions of multiple customers in the dining space based on environmental sensing data provided by at least one environmental sensing device, and determining the selection result of the food based on the identified selection intentions.

[0023] According to a further embodiment of this disclosure, the intention of multiple customers in a dining space to select food is identified based on one or more of the following information obtained from environmental sensing data: the customer's identity, gestures, gaze, and facial expressions.

[0024] According to a further embodiment of this disclosure, identifying the selection intentions of multiple customers in a dining space regarding food, and determining the food selection result based on the identified selection intentions, further includes: identifying the primary customer among the multiple customers, and assigning a higher weight to the primary customer's selection intentions than that of other customers; and weighting the food selection intentions of the multiple customers to determine the food selection result.

[0025] Compared with the prior art, the multi-screen interactive system and control method for restaurants provided in this disclosure have at least the following advantages:

[0026] 1. It can conveniently and intuitively display detailed information about the dishes and other items through multiple screens, significantly improving the convenience of ordering for customers;

[0027] 2. It enables convenient multi-person ordering and provides interactive features such as food voting, enhancing the convenience and fun of multi-person ordering;

[0028] 3. It can create a digital and personalized immersive dining environment, and can be switched at will according to customer preferences or to match the food, overcoming the problem of the past dining environment being monotonous;

[0029] 4. Multi-screen solutions can better showcase food or other goods and services, reduce labor costs while increasing the probability of marketing success, enrich the restaurant's service offerings, and enhance the restaurant's appeal to customers.

[0030] These and other features and advantages will become apparent from the following detailed description and with reference to the accompanying drawings. It should be understood that the foregoing general description and the following detailed description are illustrative only and do not limit the scope of the claims. Attached Figure Description

[0031] To gain a more detailed understanding of the manner in which the features of this disclosure are described above, reference can be made to the various embodiments for a more specific description of the above-briefly summarized aspects, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only certain typical aspects of this disclosure and should not be considered as limiting its scope, as this description may allow for other equivalent and effective aspects.

[0032] Figure 1 This is an example architecture diagram of a multi-screen interactive system for a restaurant according to an embodiment of the present disclosure.

[0033] Figure 2 This is an example flowchart of a control method for a multi-screen interactive system for a restaurant according to an embodiment of the present disclosure. Detailed Implementation

[0034] The present invention will now be described in detail with reference to the accompanying drawings, and its features will become further apparent from the following specific description. In this detailed description, numerous specific details are set forth to provide a thorough understanding of the exemplary embodiments described. However, it will be apparent to those skilled in the art that the described embodiments may be practiced without some or all of these specific details. In other exemplary embodiments, well-known structures or processing steps have not been described in detail to avoid unnecessarily obscuring the concepts of this disclosure.

[0035] In this specification, unless otherwise stated, the term "A or B" as used herein means "A and B" and "A or B", and does not mean that A and B are excluded.

[0036] Figure 1 This is an example architecture diagram of a multi-screen interactive system 100 for a restaurant according to an embodiment of this disclosure. Figure 1 As shown, the multi-screen interactive system 100 may include at least one mobile terminal 101, at least one second display screen 102 installed in the dining space of the restaurant, and a server 103 communicatively coupled to the mobile terminal 101 and the second display screen 102.

[0037] Mobile terminal 101 can be any portable electronic device, which typically has wireless communication capabilities, allowing users to access the internet, receive and send data while on the move. In the restaurant scenario described in this disclosure, mobile terminal 101 can be a smartphone or tablet, or any other portable electronic device. Each mobile terminal 101 may have a first display screen 104, and the first display screen 104 is typically a touch screen to facilitate user interaction and input via touch.

[0038] The second display screen 102 can be any display device, including but not limited to televisions, monitors, and screens receiving projections from projectors (including screens, walls, etc.), and is not limited to the display technologies used, such as LCD, LED, OLED, MEMS, etc. Compared to the first display screen 104 on the mobile terminal 101, the second display screen 102 can be fixedly installed in the dining space of the restaurant, for example, by wall mounting or hanging in the lobby or private rooms. Of course, the second display screen 102 can also be movable, such as a television mounted on a mobile stand with wheels. Furthermore, the size of the second display screen 102 is not limited, but it is generally significantly larger than the size of the first display screen 104. For example, it can be the 32-85 inch size commonly used in televisions, or a large monitor or screen exceeding 100 inches. Such large-size displays can be arranged to cover large areas of walls, which is beneficial for providing visually impactful images and creating an immersive experience. Optionally, the second display screen 102 may also include a larger-size display device composed of multiple display devices. The location and number of the second display screens 102 installed in the restaurant can be arbitrarily set as needed, for example, in a private room of the restaurant.

[0039] Server 103 can communicate with mobile terminal 101 via wireless communication technology (e.g., Wi-Fi) and with second display screen 102 via wireless or wired communication technology. When server 103 is connected to second display screen 102 via a wired connection, second display screen 102 can serve as one of the external display devices for server 103 to display content specified by server 103. When server 103 communicates with second display screen 102 wirelessly, server 103 can control the content to be displayed on second display screen 102 via technologies such as wireless projection.

[0040] Server 103 can run and maintain the restaurant's digital management system, which may include multiple modules for implementing various functions such as reservations, ordering, payment, kitchen management, and warehouse management. Taking ordering as an example, mobile terminal 101 can be configured as a client for the ordering module, while server 103 can act as the server for the ordering module. Server 103 may further include a storage unit 105 for storing the program code of the digital management system and multiple databases. The databases may store various digital information related to the restaurant, including text and / or multimedia information related to the food, which will be further described later.

[0041] According to one embodiment of the present disclosure, a mobile terminal 101 may be configured to: present menu information to a customer on a first display screen 104; detect interactive actions performed by the customer on the first display screen 104; identify a first interactive event based on the detected interactive actions; and send the identified first interactive event to a server 103.

[0042] As mentioned earlier, mobile terminal 101 can be a tablet computer provided by the restaurant for customers to use, with the restaurant's ordering application installed on the tablet, allowing customers to access and operate the ordering application interface. Alternatively, mobile terminal 101 can also be the customer's own smartphone, accessing the ordering app interface by scanning a QR code provided by the restaurant or through other means. In response to the customer accessing and launching the ordering interface, the restaurant's menu information can be displayed on the screen.

[0043] The menu information may include an introduction to the restaurant, as well as various menu items available for customers to choose from. Menu items may include not only the food offered by the restaurant, but also, but not limited to, the setting, services, and merchandise.

[0044] Customers can use touchscreens to navigate the menu on the first display screen 104. Upon discovering an item of interest (e.g., a meal), they can click to view detailed information, add it to their cart, or place an order. To this end, the mobile terminal 101 is configured to detect customer interactions on the first display screen 104 (e.g., swiping, tapping), and determine whether a pre-defined first interaction event has been executed based on the detected interactions. The first interaction event may include, but is not limited to: browsing, viewing details, adding to cart, placing an order, and checking out. Once these interaction events are detected, the mobile terminal 101 reports them to the server 103, which then executes a response.

[0045] Accordingly, server 103 may be configured to: in response to receiving a first interactive event from mobile terminal 101, determine the display mode of second display screen 102 and display content corresponding to the interactive event on second display screen 102.

[0046] As a multi-screen system, the display modes of the second display screen 102 may include at least a first display mode that displays the same content as the first display screen 104 and a second display mode that displays different content from the first display screen 104. According to one example, the first and second display modes can be switched in response to a customer clicking a separately configured display mode switching button in the application interface on the first display screen 104. According to another example, as will be described in further detail below, the first and second display modes switch automatically and intelligently, for example, in response to a first interaction event performed by the customer on the first display screen 104.

[0047] As an example scenario, imagine a customer sitting in a private dining room, ordering food via a touchscreen on a first display screen 104 of a mobile terminal 101 (tablet). A large second display screen 102 (television) is mounted on one wall of the room. The second display screen 102 operates by default in a second display mode, which can include displaying a default screen and displaying a screen pushed by a server 103. The default screen can be a preset background interface, such as the restaurant's welcome screen, or a static or dynamic background that matches the current theme of the private room. When the customer taps on a dish on the first display screen 104, the tap is recognized as an event requesting details about that dish. This request is then sent to the server 103, which retrieves the content corresponding to the dish from its storage unit 105 and displays it on the second display screen 102. When the menu item is a dish, the content corresponding to the dish may include, but is not limited to, information about the dish's ingredients, preparation process, cultural background, customer reviews, and availability. Content can be presented in various formats, including images, audio, video, or any other multimedia format.

[0048] Compared to the traditional method where menu details can only be viewed on a small screen on the customer's own mobile device, this method allows menu details to be displayed on a large second screen. This more effectively showcases the details of the dishes, creating a more visually impactful and engaging experience. It is especially beneficial in group dining settings, allowing other customers present to easily and intuitively view the menu details, thus bringing a completely new ordering experience.

[0049] In one alternative implementation, in response to the completion of playback of content corresponding to a menu item, or when the customer returns to the previous screen in the ordering interface, the second display screen can return to the previous default screen. Furthermore, in response to the customer clicking the display mode switch button on the interface, the second display screen can switch at any time to a first display mode that presents the same content as the first display screen.

[0050] In another example, menu items can be scenes. A scene refers to the layout of the dining space, which can be used with the help of facilities including the second display screen 102 to create an atmosphere corresponding to the selected scene, providing immersive experiences for diners. Scenes can include various environments, atmospheres, themes, etc., such as sunny beaches, underwater worlds, forest wonderlands, exotic settings, futuristic science fiction, art galleries, historical reenactments, seasonal changes, etc. When a customer selects a scene from the multiple scenes provided on the menu, the server 103 can control the second display screen 102 to display content corresponding to that scene, including static images or dynamic videos. In addition, the server 103 can also control the lighting, background music, temperature, humidity, odor, airflow, etc. in the dining space in conjunction with the second display screen 102 to further enhance the immersive experience. Furthermore, to enhance the immersive effect, multiple second display screens 102 can be set up in the dining space as needed, and can cover multiple walls, ceilings, and / or floors.

[0051] In another embodiment of this disclosure, the scene can also be a virtual shared dining scenario. In this scenario, a second display screen 102 mounted on the wall can display and create the effect of an extended dining table, while the other end of the table can be other customers in another dining location, such as the customer's relatives or friends. The parties in the scene can see and talk to each other in a manner similar to a video conference. The other party's table in the image is virtual, while the food displayed on the table can be the other party's real food. In this way, an immersive shared dining experience can be achieved for multiple people in different physical locations. Furthermore, it is conceivable that the other party in the shared dining scenario can also be a digital virtual character. In this case, digital animation generation technology can be used to display the food on the current customer's table on the virtual table, creating a realistic effect of the digital virtual character dining with the customer.

[0052] In yet another example, menu items can be services. Here, services can be services offered by the restaurant other than food and beverage, such as music / video on demand, birthday / holiday / anniversary celebrations, or any other service suitable for presentation via the second display screen 102. As an example, in response to a customer selecting a piano piece in the menu interface, the audio or video of that piano piece can be played via the second display screen 102. In response to a customer selecting a birthday or anniversary celebration in the menu interface, corresponding greetings, videos of well wishes from friends and family, a birthday song, a collection of memorable photos, etc., can be displayed on the second display screen.

[0053] In another example, menu items can be merchandise. Merchandise can be any item offered by the restaurant for customers to purchase, including but not limited to select ingredients, beverages, merchandise bearing the restaurant's logo, limited-edition restaurant souvenirs, etc. In response to a customer selecting a product, detailed product information, a purchase link, or a payment QR code can be displayed on the second display screen 102.

[0054] Compared to traditional restaurants, this invention, through the aforementioned multi-screen interactive method, not only provides a more convenient ordering experience, but also offers customers richer services and a more personalized and immersive dining experience.

[0055] According to a further embodiment of this disclosure, the multi-screen interaction system 100 may further include at least one environmental sensing device installed in the dining space of a restaurant. The environmental sensing device may include, but is not limited to, a camera, a microphone, or any other sensor suitable for sensing and identifying objects and events in the environment within the dining space. In some embodiments, the environmental sensing device, such as the camera and microphone, may be standalone hardware or integrated with the second display screen 102 or other hardware devices in the dining space. The camera may be configured to identify customers in the dining space, perform facial recognition to identify customers, recognize customer facial expressions, recognize customer hand gestures, track customer gaze, etc. The microphone may be configured to process customer speech to recognize customer voice commands, or understand context and situation based on customer dialogue to assist the multi-screen interaction system 100 in providing a more accurate and intelligent response to customer intentions.

[0056] In an embodiment equipped with an environmental sensing device, the server 103 may be further configured to identify a second interactive event based on environmental sensing of the dining space by at least one environmental sensing device, and in response to the identified second interactive event, determine a display mode 105 of the second display screen 102 and display content corresponding to the second interactive event on the second display screen 102.

[0057] Unlike the first interaction event, which is identified by the customer's touch on the first display screen 104, the second interaction event is identified by sensing via an environmental sensing device. As a non-limiting example, the second interaction event may include, but is not limited to: the customer's facial expressions, hand gestures, gaze, and voice, food being served, dining progress, etc.

[0058] As an example, server 103 can identify the dish being served by capturing images from a camera, and then control the display of a video introduction and background information of that dish on the second display screen 102. Compared to traditional restaurants where waiters usually verbally introduce dishes when serving, this method provides richer information and a more intuitive and unique experience. Additionally or alternatively, depending on the type of dish being served, server 103 can also control the automatic switching of the aforementioned immersive scenes, with the second display screen 102 displaying different content accordingly. For example, when serving a traditional Chinese dish, the scene can be switched to a traditional Chinese theme, while when serving a representative French dish, the scene can be switched to a Parisian street scene or a medieval castle theme. Similarly, depending on the origin of the wine or tea the customer is tasting, the display on the second display screen 102 can be automatically switched to provide a corresponding immersive experience.

[0059] Similarly, based on the recognition of other secondary interactive events, the display mode and content of the second display screen 102 can also be controlled. For example, when a customer is detected frowning, displeased, serious, or shaking their head, it may indicate that the customer does not like the currently displayed content; some hand gestures can be set to facilitate customers to quickly switch display modes and content, such as customers using a T-shaped gesture to pause the playback, a cross gesture to stop the playback, or waving one hand to the side to indicate returning, etc.; voice commands can also be set, and when customers speak these voice commands, the corresponding content is displayed; when it is recognized that the meal is finished, an interactive interface can be displayed on the second display screen 102 in a timely manner, inviting users to evaluate today's meal, asking users if they want to purchase the same food or drinks, etc.; and so on.

[0060] In reality, it's common for customers ordering food to consult with other customers, which often requires multiple menus or handing the menus or ordering terminals to other customers, wasting time and being inefficient. This disclosed multi-screen interactive system 100 improves the convenience and enjoyment of multi-person ordering. In this scenario, a shared ordering function button can be set on the ordering interface. Customers clicking this button initiate a shared order. In response to this shared ordering request, server 103 controls a second display screen 102 to show the shared ordering interface. As an example, displaying the shared ordering interface on the second display screen 102 can include switching the second display screen 102 to a first display mode, i.e., displaying the same content as the first display screen 104, so that all customers present can see the menu and menu items through the second display screen 102. Clicking on a menu item allows for further details to be displayed on the second display screen 102.

[0061] When equipped with environmental sensing devices, the collaborative ordering function can also provide a candidate menu voting feature. For example, multiple dishes can be selected as candidates, compared among them, and the final order can be made. For instance, for similar dishes (such as meat), two or three dishes can be selected as candidates, and then one dish can be chosen from each candidate, say steak or lamb chops. A voting interface can then be presented, prompting customers to raise their hands to vote for different candidates in turn, or simultaneously point to a dish, or look at a dish. The camera identifies the intentions of multiple customers and tallies the voting results to determine the final choice. Alternatively, a similar function can be implemented based on the facial expressions of the customers present. For example, when two candidate dishes are introduced sequentially on the second display screen 102, the facial expressions of the customers can be observed and recognized, and the facial expression recognition results can be used to indicate the dish that most customers are more interested in as the order. Furthermore, in another example, when the camera can identify the diners, such as recognizing a particular customer as the main guest for the day, the voting process can consider that customer's opinion more in the background, for example, by giving them more weight in the vote. This feature not only provides convenience for multiple people ordering food together but also adds a lot of fun, turning ordering into a collaborative activity that is both interesting and efficient, while also alleviating awkward situations in certain contexts.

[0062] Figure 2 This is an example flowchart of a control method 200 for a multi-screen interactive system in a restaurant according to an embodiment of the present disclosure. The multi-screen interactive system can be combined with... Figure 1 The multi-screen interactive system 100 described herein, and the control method 200, can be executed by the server 103 within the multi-screen interactive system 100 to control the coordinated display and interaction of the first display screen 104 and the second display screen 102. For example... Figure 2 As shown, method 200 begins at step 202, presenting menu information to the customer on a first display screen. As previously described, the first display screen may be a touch display screen 104 of the mobile terminal 101 of the multi-screen interaction system 100.

[0063] In step 204, the interactive action performed by the customer on the first display screen is detected. As previously described, the interactive action can be an action performed by the customer on the first display screen 104, including swiping, tapping, etc.

[0064] In step 206, a first interaction event is identified based on the detected interaction action. As described previously, a specific first interaction event can be identified based on the interaction action, including browsing, viewing details, adding to cart, placing an order, checking out, etc., of menu items contained in the menu information. Menu items may include food items, scenarios, services, goods, or similar terms.

[0065] In step 208, based on the identified first interaction event, the display mode of the second display screen is determined, and content corresponding to the first interaction event is displayed on the second display screen. The second display screen is installed in the dining space of the restaurant, and the display modes of the second display screen include at least a first display mode displaying the same content as the first display screen and a second display mode displaying different content. As previously described, the first interaction event could be a customer requesting to view details of a menu item included in the menu information or selecting a menu item. Accordingly, content corresponding to the menu item can be displayed on the second display screen, which may include: ingredients, preparation process, cultural background, customer reviews, and availability related to the dish; audio or video corresponding to the scene; audio or video corresponding to the service; product descriptions and sales information; and so on.

[0066] When at least one environmental sensing device is installed in the dining space, the control method 200 may further include: identifying a second interactive event based on environmental sensing of the dining space by the at least one environmental sensing device installed in the dining space of the restaurant; and, in response to the identified second interactive event, determining the display mode of the second display screen and displaying content corresponding to the second interactive event on the second display screen. As previously described, the second interactive event may include customer actions, voice and facial expressions, food being served, dining progress, etc. In response to the identified second interactive event, the second display screen can be controlled to automatically and intelligently switch modes and display content, such as introductions or scenes corresponding to the food being served.

[0067] In another example scenario, the first interaction event could be a customer initiating a shared order. Accordingly, the control method 200 could further include displaying a shared order interface on a second display screen. Further, in response to entering the shared order interface, the intentions of multiple customers in the dining space to select dishes are identified based on environmental sensing data provided by at least one environmental sensing device, and the dish selection result is determined based on the identified intentions. As previously described, the customers' intentions to select dishes can be identified based on each customer's identity, gestures, gaze, or facial expressions, etc.

[0068] The foregoing description includes examples of various aspects of the claimed subject matter. It is certainly impossible to describe every conceivable combination of components or methods for the purpose of depicting the claimed subject matter, but those skilled in the art will recognize that many further combinations and arrangements of the claimed subject matter are possible. Thus, the disclosed subject matter is intended to cover all such changes, modifications, and variations that fall within the spirit and scope of the appended claims.

Claims

1. A multi-screen interactive system for restaurants, characterized in that, The multi-screen interaction system includes: At least one mobile terminal, each of the mobile terminals having a first display screen, wherein the first display screen is a touch display screen; At least one second display screen installed in the dining area of ​​the restaurant; and A server, communicatively coupled to the at least one mobile terminal and the at least one second display screen. The mobile terminal is configured to: The menu information is displayed to the customer on the first display screen; Detect the interactive actions performed by the customer on the first display screen; The first interaction event is identified based on the detected interaction action; and Send the first identified interaction event to the server; The server is configured as follows: In response to receiving a first interactive event from the mobile terminal, the display mode of the second display screen is determined and content corresponding to the first interactive event is displayed on the second display screen, wherein the display mode of the second display screen includes at least a first display mode that displays the same content as the first display screen and a second display mode that displays different content from the first display screen.

2. The multi-screen interactive system as described in claim 1, characterized in that, The first interactive event is a customer request to view details of a menu item contained in the menu information or to select a menu item, and the server is configured to display content corresponding to the menu item on the second display screen.

3. The multi-screen interactive system as described in claim 2, characterized in that, The menu items include one or more of the following: food, setting, service, and merchandise; The content corresponding to the menu items includes one or more of the following: raw materials, preparation process, cultural background, customer reviews, and availability related to the food; audio or video corresponding to the scenario; audio or video corresponding to the service; and introductions and sales information related to the product.

4. The multi-screen interactive system as described in claim 1, characterized in that, The multi-screen interactive system further includes: at least one environmental sensing device installed in the dining space of the restaurant, and the server is further configured to: The second interactive event is identified based on environmental sensing of the dining space by the at least one environmental sensing device; and In response to the identified second interactive event, the display mode of the second display screen is determined and content corresponding to the second interactive event is displayed on the second display screen.

5. The multi-screen interactive system as described in claim 4, characterized in that, The second interactive event includes one or more of the following: customer actions, voice and facial expressions, food served, and dining progress.

6. The multi-screen interactive system as described in claim 4, characterized in that, The first interactive event is when a customer initiates a joint order, and the server is configured to display the joint order interface on the second display screen in response to the first interactive event.

7. The multi-screen interaction system as described in claim 6, characterized in that, In response to entering the shared ordering interface, the server is further configured to: identify the selection intentions of multiple customers in the dining space based on environmental sensing data provided by the at least one environmental sensing device, and determine the selection results based on the identified selection intentions.

8. The multi-screen interaction system as described in claim 7, characterized in that, The intentions of multiple customers in the dining space to select food are identified based on one or more of the following information obtained from the environmental sensing data: The customer's identity, gestures, gaze, and facial expressions.

9. The multi-screen interactive system as described in claim 8, characterized in that, Identifying the food selection intentions of multiple customers in the dining space, and determining the food selection results based on the identified intentions, further includes: Identify the primary customer among the plurality of customers and assign a higher weight to the primary customer's selection intent than that of other customers; and The food selection intentions of the multiple customers are weighted to determine the food selection results.

10. A control method for a multi-screen interactive system in a restaurant, characterized in that, The control method includes: Menu information is presented to the customer on a first display screen, wherein the first display screen is a touch screen of a mobile terminal; Detect the interactive actions performed by the customer on the first display screen; The first interaction event is identified based on the detected interaction actions; Based on the identified first interaction event, a display mode of the second display screen is determined and content corresponding to the first interaction event is displayed on the second display screen, wherein the second display screen is installed in the dining space of the restaurant, and the display mode of the second display screen includes at least a first display mode that displays the same content as the first display screen and a second display mode that displays different content from the first display screen.

11. The control method as described in claim 10, characterized in that, The first interactive event is a customer request to view details of a menu item contained in the menu information or to select a menu item, and the content corresponding to the first interactive event is displayed on the second display screen, further including displaying content corresponding to the menu item on the second display screen.

12. The control method as described in claim 11, characterized in that, The menu items include one or more of the following: food, setting, service, and merchandise; The content associated with the menu items includes one or more of the following: raw materials, preparation process, cultural background, customer reviews, and availability related to the food; audio or video corresponding to the scenario; audio or video corresponding to the service; and product descriptions and sales information.

13. The control method as described in claim 10, characterized in that, The control method further includes: The second interactive event is identified based on environmental sensing of the dining space by at least one environmental sensing device installed in the dining space of the restaurant; and In response to the identified second interactive event, the display mode of the second display screen is determined and content corresponding to the second interactive event is displayed on the second display screen.

14. The control method as described in claim 10, characterized in that, The second interactive event includes one or more of the following: customer actions, voice and facial expressions, food served, and dining progress.

15. The control method as described in claim 13, characterized in that, The first interactive event is a customer initiating a joint order, and the control method further includes: In response to the first interactive event, a shared ordering interface is displayed on the second display screen.

16. The control method as described in claim 15, characterized in that, The control method further includes: In response to entering the shared ordering interface, the system identifies the selection intentions of multiple customers in the dining space based on environmental sensing data provided by the at least one environmental sensing device, and determines the selection result based on the identified selection intentions.

17. The control method as described in claim 16, characterized in that, The intentions of multiple customers in the dining space to select food are identified based on one or more of the following information obtained from the environmental sensing data: The customer's identity, gestures, gaze, and facial expressions.

18. The control method as described in claim 17, characterized in that, Identifying the food selection intentions of multiple customers in the dining space, and determining the food selection results based on the identified intentions, further includes: Identify the primary customer among the plurality of customers and assign a higher weight to the primary customer's selection intent than that of other customers; and The food selection intentions of the multiple customers are weighted to determine the food selection results.

Images