Restaurant intelligent seat searching system and method based on ESP32

Abstract

The application discloses a restaurant intelligent seat searching system and method based on ESP32, which comprises a controller, a collecting module installed in a restaurant, an output module installed in the restaurant, a cloud server and a mobile applet, wherein the controller is an ESP32 development board, the collecting module communicates with the controller through ESP-NOW, the controller communicates with the output module through a serial port, the controller transmits data to the cloud server through an MQTT protocol, and the applet interacts with the cloud server by calling an API interface provided by an official website. The system is assembled by different modules with PCB boards, is convenient to install, debug and maintain, has low cost, low power consumption and multiple functions, and is safe in communication without connection. The ESP-NOW passing mode can support that the master control board simultaneously communicates with more than 20 slave machines, that is, the master control and the camera communicate with each other, so that the purpose of simultaneously identifying multiple seats is achieved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent seat finding technology, specifically relating to an intelligent seat finding system and method for restaurants based on ESP32. Background Technology

[0002] For universities, the greater openness of university campuses and the larger number of students make it difficult for students to find seats in public places such as libraries, restaurants, and study rooms. This often results in students taking detours and wasting time, which reduces the efficiency of their daily life, work, and study.

[0003] Xu Tao et al. published a paper, Design and Implementation of a Smart Library Seating System Based on Raspberry Pi [J]. Computer Knowledge and Technology, 2019, 15(28): 104-106. The system in this paper uses a combination of hardware and software, employs facial recognition technology, and utilizes cloud and IoT platforms to assist in data visualization to achieve real-time monitoring of the seating situation inside the library and to feed the latest dynamic data of library seating information back to the user's WeChat mini-program through the cloud platform. However, since the Raspberry Pi needs to be connected to an ultrasonic ranging sensor and a camera to monitor the seating situation, considering the wiring and size of each module, it is not convenient to install in practice. At the same time, the Raspberry Pi needs to connect to WIFI through a wireless receiver to feed the collected data back to the cloud in real time. If used extensively, it will greatly occupy network resources and cause network congestion. And Li Wenjia et al. published a paper, Design of a Smart Library Management System Based on WeChat Mini-Program [J]. Special Technology, 2021.09. The system discussed in this paper mainly relies on students' self-awareness and controls venue resources through a WeChat mini-program by having them proactively report reservations. However, this design is too idealistic. When someone fails to report proactively or in a timely manner, it will waste resources and affect others. Summary of the Invention

[0004] This invention provides a smart restaurant seating system and method based on ESP32, which solves the technical problems existing in the prior art.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following solution:

[0006] The restaurant smart seating system based on ESP32 includes a controller, a data acquisition module installed in the restaurant, an output module installed in the restaurant, a cloud server, and a mobile app.

[0007] The controller is an ESP32 development board. The acquisition module communicates with the controller through ESP-NOW, the controller communicates with the output module through a serial port, and the controller transmits data to the cloud server through the MQTT protocol. The mini-program interacts with the cloud server by calling the official API interface. The battery module powers the controller.

[0008] Further optimization involves the acquisition module comprising multiple ESP32-CAM surveillance cameras.

[0009] Further optimization involves dividing the restaurant into multiple areas, each equipped with 20 ESPCAMs and one ESP32 master station. The 20 ESPCAMs and the ESP32 master station are electrically connected, and the ESP32 master station in each area sequentially sends the seating information of that area to the main ESP32 development board of the system.

[0010] Further optimization includes an LD3322 voice module and an OLED display.

[0011] Further optimizations include a speech recognition submodule and an autonomous broadcasting submodule.

[0012] Further optimization: During off-peak dining hours, the restaurant's smart seating system is in a dormant state and is activated by the voice recognition submodule.

[0013] When the voice recognition submodule receives the wake-up command, it will reply "Monitoring is on" and the main development board, monitoring camera and OLED display will be woken up, and the seat search system will operate normally.

[0014] When the voice command "Turn off monitoring" is received, the voice reply "Monitoring is off" will be executed, and the main development board, monitoring camera and OLED display will be turned off, putting them into a deep sleep state.

[0015] When the voice recognition module does not receive any instructions for an extended period of time, it will announce "Call me again if needed" and enter a sleep state until it is woken up again.

[0016] Further optimization was performed using the Edge Impulse Lab platform for training, with over two thousand samples from Kaggle's dining table dataset and DFT's profile dataset.

[0017] First, EON Turner was used to test a small number of samples, and finally the best machine learning framework suitable for running ESP32CAM was obtained.

[0018] Then, through transfer learning on the dataset, corresponding feature points are generated, and a lightweight TensorFlow Lite model is produced.

[0019] Further optimization involves using a 9V rechargeable lithium battery, which is regulated by a 5V voltage regulator module to power the ESP32CAM. Simultaneously, the voltage of the voltage regulator module is sampled via an ADC through the GPIO PIN14 pin of the ESP32CAM. When the sampled voltage is less than 5V, the ESP32CAM transmits the power information to the main control board through ESP-NOW and announces it via a voice module to remind the user to charge the battery.

[0020] The restaurant intelligent seat-finding method based on ESP32, based on the above system, specifically includes the following steps:

[0021] Step 1: Set up a restaurant seating system: Divide the restaurant into multiple areas, each area is equipped with 20 ESPCAMs and one ESP32 master station. The 20 ESPCAMs and the ESP32 master station are electrically connected. The ESP32 master station of each area sends the seating information of that area to the main ESP32 development board of the system in turn.

[0022] An OLED display screen and a voice module are installed in the restaurant. The ESP32 master station, OLED display screen, and voice module are all electrically connected to the ESP32 development board. The ESP32 development board transmits data to the cloud server via the MQTT protocol, and the mini-program interacts with the cloud server by calling the official API interface.

[0023] Step 2: Check restaurant seating occupancy:

[0024] Step 2.1: Before diners arrive at the restaurant, check the restaurant's seating occupancy status through a mobile app to determine the restaurant's foot traffic; connect the mobile app to the restaurant's seating system via the OneNet cloud server, and the OneNet cloud service uploads the restaurant's seating information to the mobile app in real time via the MQTT protocol.

[0025] Step 2.2: When diners arrive at the restaurant, they can check the seating availability inside the restaurant through the OLED display or by making a voice inquiry.

[0026] Further optimization, step 2.2 specifically includes,

[0027] The OLED display screen is installed in the restaurant, and the current seating information is displayed on the OLED display screen, allowing diners to see the current seating situation in real time and intuitively.

[0028] The voice recognition module communicates with the user and announces the table occupancy status, such as:

[0029] User: Status of Table 2; Reply: Table 2 is full (or empty);

[0030] User: Status of table one; Reply: Table one is full (or empty);

[0031] ......”

[0032] The autonomous broadcasting submodule automatically broadcasts the current status of the dining tables based on their occupancy, such as: "Table 1 is full" or "Table 2 is empty".

[0033] Compared with the prior art, the beneficial effects of this application are as follows:

[0034] 1. The modules selected in this invention all have the characteristics of low power consumption. For example, the ESP32-S3 chip has a load current of only 7-240uA in low power mode. Moreover, the device can monitor the working status of the system through voice control. During the off-peak period, the device can be turned off by voice command to put it into a deep sleep state, which can further reduce power consumption. Therefore, the overall power consumption of the device is very low, and it has the characteristics of low power consumption.

[0035] 2. Edge AI technology addresses the challenge of processing data generated by IoT devices locally, requiring uploading to the cloud or data center for analysis, which results in high data latency and significant bandwidth pressure. Edge computing directly applies the powerful capabilities of AI to the device, offering advantages such as low latency, high security, and low cost.

[0036] 2. The system described in this invention has five very practical functions for users to use: voice monitoring system, voice Q&A, autonomous broadcast, real-time display of seat information on OLED screen, and real-time query of seat information on mobile terminal. It is multifunctional.

[0037] 3. The ESP32-CAM transmits the recognition results to the main development board via ESP-NOW communication. This provides a feasible connectionless and secure communication solution while offering advantages such as low power consumption, ease of use, and no network occupation. Furthermore, the ESP-NOW communication mode can support the main control board simultaneously communicating with up to 20 slave devices, enabling data communication between the main control board and the cameras, thus achieving the goal of simultaneously recognizing multiple seats.

[0038] 4. This system is assembled from different modules using PCB boards. It is small in size, easy to install, debug and maintain, and low in cost. Attached Figure Description

[0039] Figure 1 This is a block diagram of the restaurant intelligent seat-finding system based on ESP32 described in this invention;

[0040] Figure 2 This is a diagram illustrating the overall framework of the ESP32-based intelligent seating system for restaurants as described in this invention.

[0041] Figure 3 Here is a flowchart of the workflow for a restaurant smart seating system based on ESP32;

[0042] Figure 4 Here is a flowchart of the ESP32-CAM workflow;

[0043] Figure 5 Here is a flowchart of the ESP32-DevKitc-1 workflow;

[0044] Figure 6 This is a schematic diagram showing the seat occupancy status of the OLED display.

[0045] Figure 7 The mobile app displays a diagram showing the restaurant's occupancy status.

[0046] Figure 8 This is a diagram of a neural network framework.

[0047] Figure 9 This is a schematic diagram of the training results based on a neural network training model;

[0048] Figure 10 This is a graph showing the training results of a neural network-based training model. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] like Figure 1 As shown, the restaurant smart seating system based on ESP32 includes a controller, a data acquisition module installed in the restaurant, an output module installed in the restaurant, a cloud server, and a mobile app.

[0051] The controller is an ESP32 development board. The acquisition module communicates with the controller via ESP-NOW, the controller communicates with the output module via a serial port, and the controller transmits data to the cloud server via the MQTT protocol. The mini-program interacts with the cloud server by calling the official API interface. The battery module powers the controller. The acquisition module includes multiple ESP32-CAM surveillance cameras. The output module includes an LD3322 voice module and an OLED display. The voice module includes a voice recognition submodule and an autonomous broadcast submodule.

[0052] In this embodiment, the mobile device is a smartphone, and the mini-program is a WeChat app. In other embodiments, the mobile device can be a tablet computer, etc.

[0053] The ESP32 development board, designed and developed by Espressif Systems, is a device that can run applications as a standalone system or as a slave device to a host MCU. It provides Wi-Fi and Bluetooth functionality via SPI / SDIO or I2C / UART interfaces. Designed for mobile devices, wearable electronics, and IoT applications, this chip boasts industry-leading low-power performance, including fine-grained clock gating, power-saving modes, and dynamic voltage regulation. The ESP32 integrates an antenna switch, RF balun, power amplifier, low-noise receiver amplifier, filter, and power management module. With minimal external components, the ESP32 achieves powerful processing performance, reliable security, and Wi-Fi & Bluetooth functionality.

[0054] This invention is based on a smart seating system built upon an IoT platform, cloud platform, and WeChat mini-program. Utilizing edge AI, image recognition, and voice recognition technologies, it employs an ESP32-CAM camera to detect seat occupancy in the application scenario. The detection results are then uploaded in real-time to the main development board ESP32-S3-DevKitC-1 via ESP-NOW wireless communication. The main development board then uploads the seating information to the OneNET cloud platform via the MQTT protocol. The seating information is visualized using the API interfaces of the WeChat mini-program and the OneNET cloud platform. Meanwhile, diners in the restaurant can check seating availability via an OLED display or by voice inquiry. Specifically, when seating arrangements change, the voice announcement system will announce the current seat number in real-time to facilitate seating. Figure 2 As shown.

[0055] This system is suitable for epidemic prevention and control in campus restaurants during the pandemic. By using a modular product design approach, combining various emerging technologies, and employing embedded devices, the system has a wide range of applications, a certain degree of flexibility and applicability, and can be adjusted in a timely manner based on customer needs. At the same time, its low power consumption and low cost also give the product a broader market space.

[0056] This invention provides regionalized management of restaurant seating, dividing the restaurant into multiple areas such as A, B, C, D, etc. Each area has an ESP32 master station responsible for managing the seat information identified by 20 ESPCAMs within that area. Then, each area's ESP32 master station sequentially sends the seat information within its area to the system's main ESP32 development board. The main ESP32 development board can use a polling method to receive information from each area's ESP32 master station to avoid data collisions, thus solving the deployment problem for medium-sized restaurants. The ESP32-CAM transmits the recognition results to the main development board via ESP-NOW communication. This provides a feasible solution for connectionless and secure communication, while also offering advantages such as low power consumption, ease of use, and no need to occupy a network. More importantly, the ESP-NOW communication method can support the main control board simultaneously communicating with up to 20 slave devices, i.e., the main control board and the cameras, thereby achieving the goal of simultaneously recognizing multiple seats. The workflow of the ESP32-CAM is as follows: Figure 4 As shown.

[0057] In this invention, based on artificial intelligence pattern recognition, the Internet of Things, and big data, five highly practical functions are realized on the basis of embedded devices: voice-controlled seat finding system, voice question and answer, autonomous broadcast, real-time display of seat information on OLED screen, and real-time query of seat information on mobile terminal. It also has many other functions such as automatic recognition and analysis and uploading data to the cloud.

[0058] In this invention, the sensing devices used include the OV2640 image sensor of the ESP32-CAM module and the onboard microphone of the LD3322 module. The onboard microphone of the LD3322 module converts sound signals into electrical signals. These electrical signals are then processed by the main control chip of the module to achieve voice recognition. The OV2640 image sensor features high sensitivity, low voltage, support for image compression, and a built-in embedded microprocessor. As a surveillance camera, it uses a TensorFlow Lite model trained on the Edge Impulse platform through edge learning to identify seat occupancy, achieving an accuracy of over 95%. The recognition results are then transmitted to the main development board via ESP-NOW communication.

[0059] The ESP32-CAM transmits the recognition results to the main development board via ESP-NOW communication. After receiving and processing the information from the ESP32-CAM, the main development board communicates with the LD3322 speech recognition module via serial port to enable the LD3322 to perform voice broadcast. Simultaneously, the main development board transmits data to the cloud via the MQTT protocol. The mini-program interacts with the cloud (OneNet cloud platform) by calling the official API interface and uploads the data to the WeChat mini-program.

[0060] The ESP-NOW communication utilizes IEEE 802.11 Action Vendor frame technology, combined with Espressif's proprietary IE function and CCMP encryption technology. This provides a feasible connectionless and secure communication solution while offering advantages such as low power consumption and ease of use. More importantly, it can support simultaneous data communication between the main control board and up to 20 slave devices (cameras), enabling the simultaneous identification of multiple seats.

[0061] This invention employs an MQTT protocol data transmission method based on the OneNet platform. In the MQTT protocol, there are three roles on both the client and server sides: publisher, broker, and subscriber. The main development board transmits data as a publisher, the server as a broker, and the monitoring computer as a subscriber (i.e., the client). MQTT transmitted data consists of two parts: data type and data content. When application data is sent through the MQTT network, MQTT associates the associated Quality of Service (QoS) with the data type. After subscribing, subscribers will receive data of that type. A server can distribute multiple data sets to different clients, and each client can process them independently.

[0062] The connection between WeChat Mini Programs and the OneNet cloud platform primarily utilizes the official API interfaces to interact with the OneNet platform. By subscribing to a server with the hardware, the Mini Program can synchronously receive data on the server after the hardware uploads data.

[0063] The LD3322's voice recognition function controls the main development board ESP32-S3-DevKitC-1, switching it between normal operation and deep sleep modes. The main development board ESP32-S3-DevKitC-1 controls the OLED module and ESP32-CAM, and communicates with the LD3322's voice recognition module. The programs for both ESP32-CAM and ESP32-S3-DevKitC-1 were written in the Arduino development environment, and the model is a TensorFlow Lite model trained on the Edge Impulse platform using edge learning.

[0064] WeChat Mini Programs are developed using the official WeChat Developer Tools. These tools utilize WXML's built-in conditional rendering syntax to visualize data. If the WeChat Mini Program receives the data "FULLY!", a black icon is rendered (representing the table is occupied); if it receives the data "EMPTY!", a white icon is rendered (representing the table is empty).

[0065] In terms of model building, this invention uses the Edge Impulse Lab platform for training, employing over two thousand samples from Kaggle's table dataset and DFT's profile dataset. First, EON Torner is used to test on a small number of samples, ultimately determining the optimal machine learning framework suitable for running ESP32CAM. Figure 8 As shown. Then, transfer learning is used to generate corresponding feature points and a lightweight TensorFlow Lite model. During sample training, it was found that the accuracy of camera recognition results is mainly affected by the camera's installation position and angle. Since the DFP side face samples used during model training are frontal side faces, and the table samples and side face samples are independent of each other, the training results are as follows. Figure 9 As shown in Figure 10. In reality, the image captured by the camera often contains both a profile view and a dining table, and the difference in angle between the profile view and the camera causes fluctuations in the accuracy of the model during actual use. Therefore, when actually installing the camera, a location installation program should be used to determine the optimal installation position to improve accuracy, or customized model training should be conducted according to local conditions to increase the model's precision and stability.

[0066] A field test was conducted in the applicant's university cafeteria. Test participants were randomly seated at any table, and images were captured using ESP32CAM. The test showed that the model's accuracy reached approximately 90%, with the highest accuracy achieved when the test participant's face was turned to the side of the camera or when their seat was close to the camera.

[0067] Regarding the test method for the accuracy of recognizing diners and tables, under the condition that the lighting conditions in the test scene remain unchanged, ten people with different appearance features were randomly selected for the test. The recognition test was conducted by taking pictures from different angles, such as the front and side. Table 1 shows the recognition results of the target objects from different angles in the test.

[0068] Table 1. Recognition results of the target object from different angles.

[0069] Number 1 2 3 4 5 6 7 8 9 10 Angle 90° 90° 90° 90° 90° 90° 90° 90° 90° 90° Profile probability 0.93 0.49 0.80 0.89 0.92 0.94 0.91 0.88 0.87 0.91 Table probability 0.07 0.51 0.13 0.11 0.08 0.06 0.09 0.12 0.13 0.09 Number 1 2 3 4 5 6 7 8 9 10 Angle 45° 45° 45° 45° 45° 45° 45° 45° 45° 45° Profile probability 0.77 0.80 0.73 0.76 0.74 0.80 0.71 0.44 0.47 0.42 Table probability 0.23 0.20 0.27 0.24 0.26 0.2 0.29 0.56 0.53 0.58 Number 1 2 3 4 5 6 7 8 9 10 Angle 0° 0° 0° 0° 0° 0° 0° 0° 0° 0° Profile probability 0.71 0.86 0.43 0.76 0.44 0.70 0.78 0.73 0.74 0.41 Table probability 0.29 0.14 0.57 0.24 0.56 0.3 0.22 0.27 0.26 0.59

[0070] Analysis of Table 1 shows that the recognition success rate is highest when the angle is 90°, which is consistent with the side face training model used in this product and the training results are good, meeting the needs of practical applications.

[0071] Since restaurants are not always in peak hours, keeping the seat-finding system constantly operational would waste resources and increase costs. Therefore, the system should be shut down during off-peak hours to reduce usage, costs, and power consumption. Voice control of the system further simplifies operation; a single sentence can control the system and achieve the desired effect.

[0072] The seat-finding system consists of the main development board ESP32-S3-DevKitC-1, the surveillance camera ESP32-CAM, an OLED display module, and the LD3322 voice recognition module. The entire seat-finding system is activated by the voice recognition submodule, such as... Figure 3 , 5 As shown.

[0073] When the voice recognition module receives a wake-up command, such as "monitoring on" or "turn on monitoring", it will reply "monitoring is on". At the same time, the main development board, the monitoring camera and the OLED display will be woken up and the seat-finding system will operate normally.

[0074] When the voice command "Turn off monitoring" is received, the system will reply "Monitoring is off" and shut down the main development board, monitoring camera, and OLED display, putting them into a deep sleep state. Simultaneously, if the voice recognition module does not receive a command for an extended period, it will announce "Call me again if needed" and enter sleep mode until it is woken up again, thereby reducing power consumption and saving costs and resources.

[0075] Finding a seat in a school cafeteria is crucial, especially during peak hours. With so many people, finding a seat can become extremely difficult, often resulting in students carrying their utensils while searching for a place to sit, severely impacting their dining experience. Therefore, to make finding a seat simple and quick for students, this invention utilizes a "voice response" function, making it easy and fun, and solving the long-standing "seat-finding problem" for students.

[0076] When the seating system is activated, the voice recognition module acts as a voice assistant, interacting with the user and announcing the table's status when someone inquires about it. The effect is as follows:

[0077] User: Hi Xiaomei! Reply: How can I help you?

[0078] User: Status of Table Two. Reply: Table Two is full (or empty).

[0079] User: Status of table one. Reply: Table one is full (or empty). ......

[0081] (When the user stops asking) Reply: Call me again if needed.

[0082] The voice response function can easily help students find seats and eat, improving their dining experience.

[0083] Although the aforementioned voice response function already exists, in reality, the seating arrangement of students is dynamic and the real-time nature of voice responses is not high, so an automatic broadcast function is needed.

[0084] Like the voice response function, the automatic announcement function will only work properly when the seat-finding system is activated. When the restaurant is in peak dining hours, if the status of a table changes (from empty to full or from full to empty), the system will automatically announce the current status of that table, such as: "Table 1 is full", "Table 2 is empty".

[0085] By broadcasting table information, students in the restaurant can clearly understand the current seating situation without any operation. The problem of finding a seat has been effectively solved based on voice response.

[0086] By displaying the current seating information on a large screen in the restaurant, students can see the seating situation in real time and intuitively. This invention uses an OLED module to replace the large screen in the restaurant, achieving the desired effect. Figure 6 As shown, when tables one and two are unoccupied, the status of both Desk 1 and Desk 2 on the OLED display is Empty. When someone is seated at table two, the OLED display shows Desk 1 as Empty and Desk 2 as Full. The ESP32-S3-DevKitc-1 synchronizes time from the Internet via the NTP protocol.

[0087] The introduction of this function makes the solution to the "seat-finding problem" of this invention more complete and more effective, further enhancing the user's dining experience!

[0088] By connecting mobile phones to the restaurant's seating system through OneNet cloud, restaurant seating information is uploaded to a WeChat mini-program in real time via the MQTT protocol, allowing users to conveniently view seating information on their mobile devices. The physical effect is as follows: Figure 7 As shown, when a seat at the table is occupied, its color will change from white to black, indicating that the seat is occupied. Simultaneously, the blue area at the top of the mini-program will display the current number of occupants at the table, the region, and a slogan reminding everyone to dine politely.

[0089] This feature is primarily aimed at students planning to dine in the cafeteria. Of course, students already in the cafeteria can also use this feature to check seating availability on their phones and quickly find a seat, solving the "seat-finding problem." Students planning to dine in can check seating availability in advance on their phones and estimate the current crowd density based on the information, then decide whether to go to the cafeteria. This aims to achieve staggered dining times, reduce crowding in the cafeteria, and improve the student dining experience.

[0090] Since the ESP32CAM operates at a normal power supply voltage of 5V, this invention uses a 9V rechargeable lithium battery regulated by a 5V voltage regulator module to power the ESP32CAM. Simultaneously, the voltage of the voltage regulator module is sampled using an ADC via the ESP32CAM's GPIO PIN14 pin. When the sampled voltage is less than 5V, the ESP32CAM transmits power information to the main control board via ESP-NOW and announces it via a voice module to remind the user to charge the battery. The technical specifications of each module are shown in Table 2.

[0091] Table 2 Technical Specifications of Each Module

[0092]

[0093] Example 1:

[0094] In this embodiment, three rounds of testing were conducted. In each round, students sat in front of the dining table in turn, and the camera was placed next to the aisle, facing the student's profile. The probability values ​​of the model recognizing each student's profile and the dining table, as well as the OLED display status, were recorded each time using an Arduino serial port assistant. The angle of the camera relative to the student's profile and the table was changed after each round of testing. Finally, the average probability of recognizing the profile and the dining table at different angles, along with the recognition accuracy, were calculated using the recorded data. Some of the data is shown in Table 3.

[0095] Table 3 Test Results

[0096]

[0097] The data in Table 3 is partial record data. Since the feature points identified in this project are different when there are people and when there are no people, that is, the probability of the dining table is the probability when there is no one, while the probability of the profile is the probability when there are people, so the sum of the probabilities of the two is 1.

[0098] Conclusion: Based on the complete data, the corresponding recognition accuracy was calculated for different camera angles.

[0099] Camera facing the side of the face (90°): 92.5%

[0100] Camera angled at the side of the face (45°): 88.9%

[0101] Camera facing the side of the face (0°): 87.2%

[0102] Therefore, the actual recognition accuracy of the model is not much different from the ideal accuracy of 95%. Furthermore, the data calculated through testing shows that the recognition accuracy is the highest when the camera is facing the side of the face, while the accuracy is relatively low at other angles. Therefore, in practical applications, the recognition effect is best when the camera is facing the side of the face.

[0103] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A restaurant intelligent seating system based on ESP32, characterized in that, This includes a controller, a data acquisition module installed in the restaurant, an output module installed in the restaurant, a cloud server, and a mobile app. The controller is an ESP32 development board. The acquisition module communicates with the controller through ESP-NOW, the controller communicates with the output module through a serial port, the controller transmits data to the cloud server through the MQTT protocol, and the mini-program interacts with the cloud server by calling the official API interface; the battery module powers the controller. The acquisition module includes multiple ESP32-CAM surveillance cameras; The restaurant is divided into multiple areas, each of which is equipped with 20 ESP32-CAMs and one ESP32 master station. The 20 ESPCAMs and the ESP32 master station are electrically connected. The ESP32 master station of each area sends the seating information of that area to the main ESP32 development board of the system in turn. The Edge Impulse Lab platform was used for training, and the dataset used for training consisted of more than two thousand samples from Kaggle's table dataset and DFT's profile dataset. First, EON Turner was used to test a small number of samples, and finally the best machine learning framework suitable for running ESP32CAM was obtained. Then, through transfer learning on the dataset, corresponding feature points are generated, and a lightweight TensorFlow Lite model is produced.

2. The restaurant intelligent seating system based on ESP32 according to claim 1, characterized in that, The output module includes an LD3322 voice module and an OLED display.

3. The restaurant intelligent seating system based on ESP32 according to claim 2, characterized in that, The voice module includes a voice recognition submodule and an autonomous broadcasting submodule.

4. The restaurant intelligent seating system based on ESP32 according to claim 3, characterized in that, During off-peak dining hours, the restaurant's smart seating system is in a dormant state and is activated by the voice recognition submodule. When the voice recognition submodule receives the wake-up command, it will reply "Monitoring is on", which will wake up the main development board, the monitoring camera and the OLED display, and the seat-finding system will operate normally. When the voice command "Turn off monitoring" is received, the voice reply "Monitoring is off" will be executed, and the main development board, monitoring camera and OLED display will be turned off, putting them into a deep sleep state. When the voice recognition module does not receive instructions for an extended period of time, it will announce "Call me again if needed" and enter a sleep state until it is woken up again.

5. The restaurant intelligent seating system based on ESP32 according to claim 4, characterized in that, The battery module is a 9V rechargeable lithium battery, which is regulated by a 5V voltage regulator module to power the ESP32CAM. At the same time, the voltage of the voltage regulator module is sampled by an ADC through the GPIOPIN14 pin of the ESP32CAM. When the sampled voltage is less than 5V, the ESP32CAM will transmit the power information to the main control board through ESP-NOW and announce it through the voice module to remind the battery to be charged.

6. A restaurant intelligent seat-finding method based on ESP32, characterized in that, The system based on claim 5 specifically includes the following steps: Step 1: Set up a restaurant seating system: Divide the restaurant into multiple areas, each area is equipped with 20 ESPCAMs and one ESP32 master station. The 20 ESPCAMs and the ESP32 master station are electrically connected. The ESP32 master station of each area sends the seating information of that area to the main ESP32 development board of the system in turn. An OLED display and a voice module are installed in the restaurant. The ESP32 master station, OLED display, and voice module are all electrically connected to the ESP32 development board. The ESP32 development board transmits data to the cloud server via the MQTT protocol, and the mini-program interacts with the cloud server by calling the official API interface. Step 2: Check restaurant seating occupancy: Step 2.1: Before diners arrive at the restaurant, check the restaurant's seating occupancy status through a mobile app to determine the restaurant's foot traffic; connect the mobile app to the restaurant's seating system via the OneNet cloud server, and the OneNet cloud service uploads the restaurant's seating information to the mobile app in real time via the MQTT protocol. Step 2.2: When diners arrive at the restaurant, they can check the seating availability inside the restaurant through the OLED display or by making a voice inquiry.

7. The restaurant intelligent seat-finding method based on ESP32 according to claim 6, characterized in that, Step 2.2 specifically includes, The OLED display screen is installed in the restaurant, and the current seating information is displayed on the OLED display screen, so that diners can see the current seating situation in real time and intuitively. The voice recognition module communicates with the user and announces the table occupancy status. The autonomous broadcasting submodule automatically broadcasts the current status of the dining tables based on their occupancy.

Images