Non-contact interactive display intelligent recipe recommendation system device and method

Abstract

The invention discloses a non-contact interactive display intelligent recipe recommendation system, device and method. The device comprises an upper shell, a lower shell, a suspension touch control + E-Ink display screen, an ESP32-S3 dual-core processor and the like, non-contact interaction is achieved through a capacitive suspension touch control sensor, and the E-Ink display screen provides low-power-consumption display. The method comprises the steps of non-contact interaction data acquisition, user intention analysis, recipe recommendation algorithm, interface dynamic rendering, system linkage, energy conservation and the like. The E-Ink display screen and the capacitive suspension touch control technology are combined, non-contact operation, low-power-consumption operation and accurate recommendation are achieved, the method is suitable for high-oil-stain scenes such as kitchens, the cooking experience of users is improved, and the method has practicability and technical innovation.

Description

Technical Field

[0002] The present invention relates to the technical field of smart home, and particularly to an intelligent recipe recommendation system device and method with non-contact interaction display. Background Art

[0004] Smart home integrates lighting, security, home appliances and other devices in a home into a system through technology integration using comprehensive wiring technology, network communication technology, automatic control technology, etc., to achieve centralized control and automated management. Its core is to connect devices through the Internet of Things (IoT) and optimize the user experience with the help of artificial intelligence (AI) and cloud computing. With the rapid development and progress of information technology and machine learning, recipe recommendation services are becoming increasingly popular among the public. The recipe recommendation service is a service that provides personalized dietary suggestions for users through intelligent algorithms. This service intelligently recommends recipes suitable for users based on their dietary preferences, nutritional needs and other relevant information. For example, existing intelligent cooking assistants, AI virtual assistants (such as Mr. ROKI), cooking robots (such as Peng Xiaoxian, Mijia cooking robot), etc., provide a full-process cooking solution through the collaboration of hardware, software and content ecosystem, including: the creation stage: voice recipe broadcast, intelligent recommendation; the recording stage: generating a dedicated cooking curve and digital recipe; the restoration stage: replicating the "taste in memory"; the sharing stage: social platform interaction and experience exchange. Features: lower the cooking threshold and enhance the fun of the experience. The core goal is to simplify the cooking process, improve efficiency or experience. Through these meticulous services, the recipe recommendation service aims to enable every user to enjoy the pleasure of cooking, not only learn how to cook, but also know how to enjoy every meal in life. Whether a novice cook or a seasoned enthusiast, everyone can find their own cooking world here and turn every cooking session into a wonderful life experience.

[0005] However, the existing technologies have the following defects: Traditional cooking assistants rely on touchscreens or mobile applications, which are inconvenient to operate or cannot recommend recipes in real time according to time. At the same time, existing products lack integration or have poor user experience. Summary of the Invention

[0007] The present invention aims to provide an intelligent recipe recommendation system device and method with non-contact interaction display to solve the problems existing in the prior art.

[0008] The intelligent recipe recommendation system device of the present invention includes a system device and a system implementation method. The system device includes an upper shell, which is a rectangular hollow shell, on which a floating touch + E-Ink display screen, a multi-modal gesture sensor, and multiple expansion buttons are installed; a lower shell, which is a corresponding rectangular bottom cover of the shell, with ventilation holes left on it, and multiple magnetic adsorption components or suction cup components are installed around it. The upper shell and the lower shell are snap-fitted or screwed together to form a hollow shell, and an ESP32-S3 dual-core processor is installed inside the hollow shell; the floating touch + E-Ink display screen integrates a capacitive floating touch sensor above or below the display screen; the ESP32-S3 dual-core processor includes a dual-core Xtensa® 32-bit LX7 CPU, a clock and power management module, a memory configuration, a peripheral interface, and a security module, and a lightweight machine learning framework (TensorFlow Lite Micro) is installed inside, which supports gesture recognition, multi-mode sensor interfaces, and intelligent kitchen equipment control.

[0009] The floating touch + E-Ink display screen, based on the microcapsule electrophoresis technology, includes an ESP32-S3 chip, controls the display screen through the SPI / I²C interface, and a flexible circuit board connects the driving board microcapsule layer to transmit control signals and power supply, and a backlight source; the capacitive floating touch sensor realizes the floating touch and multi-touch functions by simultaneously operating the mutual capacitance and self-capacitance sensors on the touch screen. The mutual capacitance is used to realize multi-touch detection, and the self-capacitance is used to detect the position of a farther finger to distinguish between floating touch and contact touch.

[0010] The multi-modal gesture sensor includes: a capacitive floating touch sensor and a millimeter-wave radar gesture recognition module. The capacitive floating touch sensor combines at least two sensors and data sources among a camera, a depth sensor, an inertial sensor, and an electromagnetic sensor to recognize gestures; the millimeter-wave radar gesture recognition uses the millimeter-wave frequency band (30 - 300 GHz) to detect by transmitting millimeter-wave signals through an antenna, and calculates information such as the distance and speed of the target based on information such as the time difference and frequency change between the transmitted signal and the received signal.

[0011] An intelligent recipe recommendation system method for non-contact interaction display, the system includes a data layer, an algorithm layer, and an application layer. The intelligent recipe recommendation system device described above executes the following steps through the system: S1 Non-contact interaction data acquisition: Extract gesture motion features through millimeter-wave radar gesture recognition, and train a lightweight CNN model to recognize various gestures; use the self-capacitance detection of capacitive floating touch to achieve non-contact clicks, and filter out environmental interference through an anti-mis-touch algorithm; S2 User intention parsing: Integrate context by combining time / scene, ingredient inventory, and historical behavior, and predict preferences through the user profile model; S3 Recipe recommendation algorithm: Integrate the local database and cloud data sources, and adopt a hybrid recommendation model, including collaborative filtering and content filtering, and dynamically adjust the recommendation weights through a model-free reinforcement learning algorithm (Q-Learning); S4 Floating touch + E-Ink interface dynamic rendering: Adopt a hierarchical display strategy for low-power UI design and content preloading; S5 System linkage and energy saving: Adaptively adjust according to the environment for power management and deep sleep settings.

[0012] In the non-contact interaction data acquisition step, the millimeter-wave radar gesture recognition extracts gesture motion features by analyzing the radar echo frequency change through the fast Fourier transform (FFT) efficient and fast algorithm. The recognized gestures include swiping left, swiping right, hovering, making a fist, and double-clicking.

[0013] In the user intention parsing step, the time period is judged by the RTC clock, the ingredient inventory data is obtained through RFID tags or image recognition, and the user profile model is constructed through collaborative filtering + Embedding.

[0014] In the floating touch + E-Ink interface dynamic rendering step, the static layer displays basic information such as time and temperature for a long time, and the dynamic layer only updates the recipe content. The E-Ink local refresh technology is adopted with a refresh rate <2Hz; 4-level gray scale is used to render graphics and texts to avoid full-screen refresh; the next recipe data is downloaded in advance in the background.

[0015] In the system linkage and energy saving step, the light sensor triggers backlight adjustment, and unnecessary peripherals are automatically turned off when the temperature > 30°C; the millimeter-wave radar enters low-power scanning in the standby mode, and the ESP32-S3 enters Modem-Sleep; the RTC wakes up regularly, and deep sleep is triggered after 10 minutes without user operation.

[0016] The beneficial effect of the present invention is that in the above way, the E-Ink display screen and the capacitive floating touch technology can be combined and used on the same device, which not only provides an interactive experience with fast response and multi-touch, but also has an E-Ink display screen for displaying content that does not need to be updated frequently, reducing the overall power consumption and flexibly adjusting the display content and interaction method according to needs. Brief Description of the Drawings

[0017] In the figure, the markings are: system device 100, upper housing 10, multimodal gesture sensor 11, multiple expansion buttons 12, lower housing 20, magnetic adsorption component or suction cup component 21, ventilation hole 22, floating touch + E-Ink display screen 30, ESP32-S3 chip 31, SPI / I²C interface 32, flexible printed circuit board (FPC) 33, driving board microcapsule layer 34, and backlight 35, capacitive floating touch sensor 36, millimeter wave radar gesture recognition module 37, ESP32-S3 dual-core processor 40, system implementation method 200, data layer 201, algorithm layer 202, application layer 203.

[0018] Figure 1 Schematic structural diagram of the present system; Figure 2 Device composition diagram of the present system; Figure 3 Method step diagram of the present system. Specific implementation manners

[0019] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0020] As Figure 1 、 2 shown, a non-contact interactive display intelligent recipe recommendation system device 100 includes: an upper housing 10, the upper housing 10 is a rectangular hollow housing, on which a floating touch + E-Ink display screen 30, a multimodal gesture sensor 11 and multiple expansion buttons 12 are installed; a lower housing 20, the lower housing 20 is a corresponding rectangular housing bottom cover plate, on which ventilation holes 22 are provided, and multiple magnetic adsorption components or suction cup components 21 are installed around it. The upper housing and the lower housing are snap-fitted or screwed together to form a hollow housing, and an ESP32-S3 dual-core processor 40 is built in the hollow housing; the floating touch + E-Ink display screen 30 integrates a capacitive floating touch sensor 36 above or below the display screen; the ESP32-S3 dual-core processor 40 includes a dual-core Xtensa® 32-bit LX7 CPU, a clock and power management module, a memory configuration, a peripheral interface, and a security module, and a lightweight machine learning framework (TensorFlow Lite Micro) is built in, which supports gesture recognition, multi-mode sensor interfaces, and intelligent kitchen device control.

[0021] The floating touch + E-Ink display screen 30, based on the microcapsule electrophoresis technology, includes an ESP32-S3 chip 31, controls the display screen through an SPI / I²C interface 32, a flexible circuit board 33 is connected to the driving board microcapsule layer 34 to transmit control signals and power supply, and a backlight 35; the capacitive floating touch sensor 36 realizes the floating touch and multi-touch functions by simultaneously operating mutual capacitance and self-capacitance sensors on the touch screen. The mutual capacitance is used to realize multi-touch detection, and the self-capacitance is used to detect the position of a farther finger to distinguish between floating touch and contact touch.

[0022] The multi-modal gesture sensor 11 includes: a capacitive floating touch sensor 36 and a millimeter-wave radar gesture recognition module 37. The capacitive floating touch sensor 36 combines at least two sensors and data sources among a camera, a depth sensor, an inertial sensor, and an electromagnetic sensor to identify gestures; the millimeter-wave radar gesture recognition uses the millimeter-wave frequency band (30 - 300 GHz) to detect by transmitting millimeter-wave signals through an antenna, and calculates information such as the distance and speed of the target according to information such as the time difference and frequency change between the transmitted signal and the received signal.

[0023] Such as Figure 3 An intelligent recipe recommendation system method 200 for non-contact interaction display as shown, the system includes a data layer 201, an algorithm layer 202, and an application layer 203, and is applied to the intelligent recipe recommendation system device described above. It is characterized in that the following steps are executed by the system: S1 Non-contact interaction data collection: Extract gesture motion features through millimeter-wave radar gesture recognition, and train a lightweight CNN model to recognize multiple gestures; use the self-capacitance detection of capacitive floating touch to realize non-contact clicking, and filter out environmental interference through an anti-mis-touch algorithm; S2 User intention parsing: Combine time / scene, ingredient inventory, and historical behavior for context fusion, and predict preferences through a user portrait model; S3 Recipe recommendation algorithm: Integrate the local database and cloud data sources, adopt a hybrid recommendation model, including collaborative filtering and content filtering, and dynamically adjust the recommendation weight through a model-free reinforcement learning algorithm (Q-Learning); S4 Floating touch + E-Ink interface dynamic rendering: Adopt a hierarchical display strategy for low-power UI design and content preloading; S5 System linkage and energy saving: Adjust adaptively according to the environment for power management and deep sleep settings.

[0024] In the non-contact interaction data acquisition step, the millimeter-wave radar gesture recognition extracts gesture motion features by analyzing the radar echo frequency change through the efficient and fast algorithm of fast Fourier transform (FFT). The recognized gestures include swiping left, swiping right, hovering, making a fist, and double-clicking.

[0025] In the user intention parsing step, the time period is judged through the RTC clock, the ingredient inventory data is obtained through RFID tags or image recognition, and a user portrait model is constructed through collaborative filtering + Embedding.

[0026] In the floating touch + E-Ink interface dynamic rendering step, the static layer displays basic information such as time and temperature for a long time, and the dynamic layer only updates the recipe content. The E-Ink local refresh technology is adopted, and the refresh rate < 2Hz; 4-level gray scale is used to render graphics and texts to avoid full-screen refresh; the next recipe data is downloaded in advance in the background.

[0027] In the system linkage and energy saving step, the light sensor triggers the backlight adjustment, and unnecessary peripherals are automatically turned off when the temperature > 30°C; in the standby mode, the millimeter-wave radar enters the low-power scanning mode, and the ESP32-S3 enters the Modem-Sleep; the RTC wakes up regularly, and deep sleep is triggered after 10 minutes without user operation.

[0028] Usage Scenario Examples Scenario 1: The user approaches the kitchen ① The floating touch detects that the user is approaching (distance < 0.3m).

[0029] ② The system wakes up, and the millimeter-wave radar enters the gesture detection mode.

[0030] ③ The floating touch + E-Ink screen displays the welcome interface and quick access (such as "Today's Recommendation").

[0031] Scenario 2: Non-contact flipping of recipes ① The user's right-swipe gesture triggers the recipe switch.

[0032] ② The screen is locally refreshed to display the new recipe picture and step summary.

[0033] ③ When the hovering gesture exceeds 2 seconds, the detailed steps and nutritional information are popped up.

[0034] Through the above steps, the system realizes the core functions of non-contact operation, low-power operation, and accurate recommendation, which is applicable to high-oil and multi-interference scenarios such as kitchens, and has practicality and technological innovation.

Claims

1. An intelligent recipe recommendation system device (100) for non-contact interaction display, characterized in that, Comprising: An upper housing (10), the upper housing (10) being a rectangular hollow housing, on which a floating touch + E-Ink display screen (30), a multi-modal gesture sensor (11) and a plurality of expansion buttons (12) are installed; a lower housing (20), the lower housing (20) being a corresponding rectangular housing bottom cover plate, on which ventilation holes (22) are provided, and a plurality of magnetic adsorption members or suction cup members (21) are installed around it. The upper housing and the lower housing are snap-fitted or screwed together to form a hollow housing, and an ESP32-S3 dual-core processor (40) is installed inside the hollow housing; the floating touch + E-Ink display screen (30) integrates a capacitive floating touch sensor (36) above or below the display screen; the ESP32-S3 dual-core processor (40) includes a dual-core Xtensa® 32-bit LX7 CPU, a clock and power management module, a memory configuration, a peripheral interface, and a security module, and a lightweight machine learning framework (TensorFlowLite Micro) is installed inside, supporting gesture recognition, multi-mode sensor interfaces, and intelligent kitchen device control.

2. The system device (100) according to claim 1, characterized in that, The floating touch + E-Ink display screen (30), based on the microcapsule electrophoresis technology, includes an ESP32-S3 chip (31), controls the display screen through an SPI / I²C interface (32), a flexible circuit board (33) connects to the driving board microcapsule layer (34) to transmit control signals and power supply, and a backlight (35); the capacitive floating touch sensor (36) realizes floating touch and multi-touch functions by simultaneously operating mutual capacitance and self-capacitance sensors on the touch screen. The mutual capacitance is used to realize multi-touch detection, and the self-capacitance is used to detect the position of a farther finger to distinguish between floating touch and contact touch.

3. The system device (100) according to any one of the preceding claims, wherein the multimodal gesture sensor (11) comprises: A capacitive floating touch sensor (36) and a millimeter-wave radar gesture recognition module (37), the capacitive floating touch sensor (36) combines at least two sensors and data sources among a camera, a depth sensor, an inertial sensor, and an electromagnetic sensor to recognize gestures; the millimeter-wave radar gesture recognition uses the millimeter-wave frequency band (30 - 300 GHz) to detect by transmitting millimeter-wave signals through an antenna, and calculates information such as the distance and speed of the target according to information such as the time difference and frequency change between the transmitted signal and the received signal.

4. A method for an intelligent recipe recommendation system with non-contact interaction display (200), the system comprising a data layer (201), an algorithm layer (202), and an application layer (203), applied to the intelligent recipe recommendation system device according to any one of claims 1 - 3, characterized in that, By the system performing the following steps: S1 Non-contact interaction data acquisition: Extract gesture motion features through millimeter-wave radar gesture recognition, and train a lightweight CNN model to recognize various gestures; use the self-capacitance detection of capacitive floating touch to realize non-contact clicking, and filter out environmental interference through an anti-mis-touch algorithm; S2 User intention parsing: Combine time / scenario, ingredient inventory, and historical behavior for context fusion, and predict preferences through a user portrait model; S3 Recipe Recommendation Algorithm: Integrate the local database and cloud data sources, adopt a hybrid recommendation model, including collaborative filtering and content filtering, and dynamically adjust the recommendation weights through a model-free reinforcement learning algorithm (Q-Learning); S4 Floating Touch + E-Ink Interface Dynamic Rendering: Adopt a hierarchical display strategy for low-power UI design and content preloading; S5 System Linkage and Energy Saving: Automatically adjust according to the environment for power management and deep sleep settings.

5. The method (200) of the non-contact interaction display intelligent recipe recommendation system according to claim 4, characterized in that, In the non-contact interaction data collection step, the millimeter-wave radar gesture recognition extracts gesture motion features by analyzing the radar echo frequency change through the fast Fourier transform (FFT) efficient and fast algorithm. The recognized gestures include swiping left, swiping right, hovering, making a fist, and double-clicking.

6. The non-contact interaction display intelligent recipe recommendation system method (200) according to claim 4, characterized in that, In the user intention parsing step, judge the time period through the RTC clock, obtain the ingredient inventory data through RFID tags or image recognition, and build a user portrait model through collaborative filtering + Embedding.

7. The method (200) of the non-contact interaction display intelligent recipe recommendation system according to claim 4, characterized in that, In the floating touch + E-Ink interface dynamic rendering step, the static layer displays basic information such as time and temperature for a long time, and the dynamic layer only updates the recipe content. Adopt the E-Ink local refresh technology with a refresh rate <2Hz; use 4-level gray scale to render text and images to avoid full-screen refresh; download the next recipe data in advance in the background.

8. The method (200) of the non-contact interaction display intelligent recipe recommendation system according to claim 4, characterized in that, In the system linkage and energy saving step, the light sensor triggers backlight adjustment, and automatically turns off unnecessary peripherals when the temperature > 30°C; the millimeter-wave radar enters low-power scanning in the standby mode, and the ESP32-S3 enters Modem-Sleep; the RTC wakes up regularly, and triggers deep sleep after 10 minutes of no user operation.

Images