Face acquisition and display method, system and device based on LVGL and storage medium
By creating screen objects, setting face frame buffers, and utilizing DMA controllers to process image data in parallel and dirty region management techniques within the LVGL system framework, the complexities of face image display and page switching in the LVGL GUI system were solved, achieving efficient fusion display and fast switching while avoiding memory conflicts and image tearing.
Patent Information
- Application Number
- CN202511074832.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-01
- Publication Date
- 2025-11-14
AI Technical Summary
The existing LVGL GUI system framework lacks a solution for displaying facial images, which leads to complex image display and page switching, as well as serious problems with resource contention and memory conflicts.
Within the LVGL system framework, efficient display of face images and page switching are achieved by creating screen objects, setting face frame buffers, using DMA controllers to process image data in parallel, and employing double buffering mechanisms and dirty area management techniques, thus avoiding memory conflicts and image tearing.
It achieves efficient fusion display of LVGL interface and face image, is independent of LVGL architecture, has fast page switching speed, reduces resource consumption, and improves system real-time performance and response speed.
Smart Images
Figure CN120950169A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of GUI system frameworks, and in particular to LVGL-based face capture and display methods, systems, devices, and storage media. Background Technology
[0002] Currently, electronic devices using embedded systems primarily employ two GUI solutions: high-end devices use the Android GUI framework, while low-end devices use the miniGUI framework. The Android GUI framework has a large codebase, making it unsuitable for storage on devices with limited space. The miniGUI framework suffers from outdated functionality, insufficient visual effects, and a still relatively large codebase.
[0003] LVGL (Light and Versatile Graphics Library) is a new GUI system framework with advantages such as small code size and ease of UI (user interface) development. It can load the firmware of electronic devices using only 16MB of flash memory.
[0004] However, LVGL is a newly emerging GUI system framework and does not yet have a solution for external face frame video streams, requiring the development of new functionalities on top of it. For example, the LVGL system framework only supports image display within its own architecture, and is based on the active screen layer, lacking additional interfaces to display images from face-capturing peripherals. Furthermore, placing the application at the top and system layers leads to complex handling issues in some pop-up blocking scenarios. Summary of the Invention
[0005] To enable the display of facial images and page switching within the LVGL system framework, this application provides a face acquisition and display method, system, device, and storage medium based on LVGL.
[0006] Firstly, this application provides a face capture and display method based on LVGL, employing the following technical solution: In the active screen layer of the face acquisition and display system, a screen object containing screen sub-objects is created, and an LVGL control is inserted into the screen sub-object. The LVGL control includes a face acquisition and recognition thread. A face frame buffer is set in the face acquisition and display system. The face UVC video stream device is used to acquire and recognize face images according to the face acquisition and recognition thread and store them in the face frame buffer. Receive the target LVGL frame buffer and redraw the target LVGL frame buffer in the ACT layer of the face acquisition and display system; The face frame coordinate data of the face image is drawn in the face frame buffer, and the face frame coordinate data is fused into the target LVGL frame buffer; The flush_cb function is called to point the display image pointer of the face capture and display system to the target LVGL frame buffer.
[0007] Through the above technical solution, the present invention achieves efficient fusion display and page switching of LVGL interface and face image, realizes face image display and dynamic tracking of face box independently of LVGL, does not conflict with LVGL architecture, has fast page switching speed and does not cause memory conflict or image tearing.
[0008] In one specific implementation, the method is further based on a DMA controller, wherein the step of using the face UVC video stream device to acquire and recognize face images according to the face acquisition and recognition thread and storing them in the face frame buffer includes: The face image is acquired using the face UVC video stream device according to the face acquisition and recognition thread; Configure the face frame buffer and storage location information of the face image; The DMA controller stores the face image into the face frame buffer according to the storage location information, and the face image is recognized by the face algorithm to obtain face information data and then stored.
[0009] In one specific implementation, the face frame buffer includes a first face frame buffer and a second face frame buffer, the face acquisition and display system includes an acquisition and storage pointer, and the step of using the DMA controller to store the face image into the face frame buffer according to the storage location information, and using a face recognition algorithm to recognize the face image to obtain face information data and store it includes: The storage location of the face image is determined based on the storage location information; If the storage location of the face image is the first face frame buffer, then the acquisition and storage pointer is pointed to the first face frame buffer, and the face image is stored in the first face frame buffer using the DMA controller; The face image is identified using a face recognition algorithm to obtain face information data, which is then stored. Mark the first face frame buffer as completed, and set the acquisition and saving pointer to the second face frame buffer; If the storage location of the face image is the second face frame buffer, then the acquisition and storage pointer is pointed to the second face frame buffer, and the face image is stored in the second face frame buffer using the DMA controller; The face image is identified using a face recognition algorithm to obtain face information data, which is then stored. Mark the second face frame buffer as completed and set the acquisition and save pointer to the first face frame buffer.
[0010] The above technical solution, through a double-buffering mechanism, allows the system to acquire the next frame of image data while processing the previous frame. This parallel processing method improves the system's real-time performance and efficiency, ensures the continuity of face acquisition and recognition, and reduces data processing latency.
[0011] In one specific implementation, the target LVGL frame buffer includes: Receive LVGL frame buffer, wherein the LVGL frame buffer includes a first LVGL frame buffer and a second LVGL frame buffer; Determine the direction of the display image pointer of the face capture and display system; If the displayed image pointer points to the first LVGL frame buffer, then the second LVGL frame buffer is determined to be the target LVGL frame buffer; If the display image pointer points to the second LVGL frame buffer, then the first LVGL frame buffer is determined to be the target LVGL frame buffer.
[0012] Through the above technical solution, the present invention achieves efficient fusion display and page switching of LVGL interface and face image, realizes face image display and dynamic tracking of face box independently of LVGL, does not conflict with LVGL architecture, has fast page switching speed and does not cause memory conflict or image tearing.
[0013] In one specific implementation, redrawing the target LVGL frame buffer in the ACT layer of the face capture and display system includes: Polling is performed through LVGL periodic events in the LVGL system framework; When a change in the properties of the screen object is detected, the area of the screen object where the change has occurred is marked as a dirty area, and the dirty area is expanded to obtain the actual redraw range. The ACT layer of the face capture and display system redraws the target LVGL frame buffer according to the actual redraw range.
[0014] In one specific implementation, marking the area where the screen object has changed as a dirty region, and expanding the dirty region to obtain the actual redraw range includes: Determine whether the change in the screen object's attribute is unique; If the change in the attributes of the screen object is unique, then the area of the screen object that has changed is marked as a dirty area, and the dirty area is expanded to obtain the actual redraw range; If the changes in the attributes of the screen object are not unique, then determine several regions of the screen object that have changed, and mark these regions as several dirty regions. Expanding several of the aforementioned dirty regions yields several actual redrawn sub-ranges; The actual redrawing range is obtained by merging several of the dirty regions, and the actual redrawing range includes several of the actual redrawing sub-ranges.
[0015] By employing the aforementioned technical solutions, dirty area management and the redraw mechanism significantly improve UI update efficiency by updating only the areas that need to change, rather than the entire screen. This method reduces unnecessary redraw operations and lowers system resource consumption, especially in scenarios involving complex UIs or frequent updates, where it can significantly improve system responsiveness and smoothness.
[0016] In one specific implementation, the face frame coordinate data for drawing the face image in the face frame buffer includes: Set the drawing color of the face image; Based on the drawing color, face coordinates are drawn in the face frame buffer according to the face image to obtain face frame coordinate data.
[0017] Through the above technical solution, this invention effectively integrates the LVGL interface and face image display while avoiding memory conflicts and resource contention. This method not only preserves the original functions of LVGL (such as pop-ups and mouse cursors) but also achieves real-time face display and tracking, providing an efficient and flexible GUI solution for embedded systems.
[0018] Secondly, this application provides a face capture and display system, which is based on a face capture and display system using the LVGL system framework and a face UVC video stream device, and adopts the following technical solution: The system includes: A screen object creation module is used to create a screen object containing screen sub-objects in the active screen layer of the face acquisition and display system, and to insert an LVGL control in the screen sub-object, wherein the LVGL control includes a face acquisition and recognition thread. A face acquisition and storage module is used to set a face frame buffer in the face acquisition and display system, and to acquire and recognize face images according to the face acquisition and recognition thread using the face UVC video stream device and store them in the face frame buffer. The frame buffer redraw module is used to receive the target LVGL frame buffer and redraw the target LVGL frame buffer in the ACT layer of the face acquisition and display system. The fusion module is used to draw the face frame coordinate data of the face image in the face frame buffer and fuse the face frame coordinate data into the target LVGL frame buffer; The face image display module is used to call the flush_cb function to point the display image pointer of the face acquisition and display system to the target LVGL frame buffer.
[0019] Thirdly, this application provides a computer device that adopts the following technical solution: it includes a memory and a processor, wherein the memory stores a computer program that can be loaded and executed by the processor, such as the above-described LVGL-based face capture and display method.
[0020] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution: storing a computer program that can be loaded by a processor and executed as described above for a face capture and display method based on LVGL.
[0021] In summary, this application has the following beneficial technical effects: (1) This invention realizes efficient fusion display and page switching of LVGL interface and face image. It independently realizes the display of face image and dynamic tracking of face box without conflict with LVGL architecture. The page switching speed is fast and there is no memory conflict or image tearing. It provides an efficient and flexible GUI solution for embedded systems.
[0022] (2) This application adopts a double buffering mechanism (BUFF1 and BUFF2) to allow the system to acquire the next frame of image data while processing one frame of image data. This parallel processing method improves the real-time performance and efficiency of the system, ensures the continuity of the face acquisition and recognition process, and reduces the delay in data processing.
[0023] (3) Dirty area management and redraw mechanism greatly improve the efficiency of UI updates by updating only the areas that need to change, rather than the entire screen. This method reduces unnecessary redraw operations and lowers the consumption of system resources. Especially in scenarios involving complex UIs or frequent updates, it can significantly improve the system's response speed and smoothness. Attached Figure Description
[0024] Figure 1 This is a flowchart of a face capture and display method according to an embodiment of this application.
[0025] Figure 2 This is a rendering of the fusion of a human face image and an LVGL image.
[0026] Figure 3 This is a diagram illustrating the collection and storage of facial data.
[0027] Figure 4 This is a structural block diagram of a face capture and display method according to an embodiment of this application.
[0028] Reference numerals: 401, Screen object creation module; 402, Face acquisition and storage module; 403, Frame buffer redraw module; 404, Fusion module; 405, Face image display module. Detailed Implementation
[0029] The following is in conjunction with the appendix Figures 1-4 This application will be described in further detail.
[0030] This application discloses a face capture and display method, system, device, and storage medium based on LVGL. The method is used to display face images and switch pages within the LVGL system framework.
[0031] Currently, electronic devices using embedded systems primarily employ two GUI solutions: high-end devices use the Android GUI framework, while low-end devices use the miniGUI framework. The Android GUI framework has a large codebase, making it unsuitable for storage on devices with limited space. The miniGUI framework suffers from outdated functionality, insufficient visual effects, and a still relatively large codebase.
[0032] LVGL (Light and Versatile Graphics Library) is a new GUI system framework with advantages such as small code size and ease of UI (user interface) development. It can load the firmware of electronic devices using only 16MB of flash memory.
[0033] However, LVGL is a newly emerging GUI system framework and does not yet have a solution for external face frame video streams, requiring the development of new functionalities on top of it. For example, the LVGL system framework only supports image display within its own architecture, and is based on the active screen layer, lacking additional interfaces to display images from face-capturing peripherals. Furthermore, placing the application at the top and system layers leads to complex handling issues in some pop-up blocking scenarios.
[0034] Therefore, this application proposes a face capture and display method based on LVGL, which enables the display of face images and page switching within the LVGL system framework. like Figure 1 As shown, the method includes: S10: Create a screen object containing screen sub-objects in the active screen layer of the face capture and display system, and insert an LVGL control into the screen sub-object. The LVGL control includes the face capture and recognition thread.
[0035] Specifically, the face capture and display system uses LVGL as the GUI system framework. The system creates an `obj_s` object (screen object) based on the active screen layer. On top of this `obj_s` object, another `obj_a` object (screen sub-object) is placed; `obj_a` is a sub-object of `obj_s`. Then, LVGL controls such as horizontal bar effects, font effects, and the face capture and recognition thread, face coordinate output callback thread, and the face display buffer enable flag (set to normal, timeout, exit, and destroy callback) are created. All the LVGL-related controls to be used are placed in the `obj_a` object, and the display logic is set. The preparation work is then complete.
[0036] S20, a face frame buffer is set in the face acquisition and display system, and the face UVC video stream device is used to acquire and recognize face images according to the face acquisition and recognition thread and store them in the face frame buffer.
[0037] Specifically, a face frame buffer is set up in the face capture and display system to store face images and related information. The face capture and recognition thread is set with the trigger conditions and corresponding callback functions for the face capture and display interface. For example, the trigger condition is that the face UVC video stream device detects a face, and then the face UVC video stream device is used to capture and recognize the face image according to the face capture and recognition thread and store it in the face frame buffer through the DMA controller.
[0038] S30: Receive the target LVGL frame buffer and redraw the target LVGL frame buffer in the ACT layer of the face capture and display system.
[0039] Specifically, a frame buffer is a memory area used to store the pixel data of the current screen (i.e., the color value of each pixel). This application adopts a double buffering mechanism, that is, the frame buffer includes the currently displayed frame buffer and the next frame buffer being drawn. The acquired target LVGL frame buffer is the next frame buffer being drawn. When it is necessary to display the collected faces within the LVGL system framework, it is necessary to change the LVGL page display, modify the properties of the screen object in the active screen layer, redraw the target LVGL frame buffer in the ACT layer of the face acquisition and display system, and draw LVGL interface elements (background, text, borders, shadows, etc.) to the target frame buffer.
[0040] S40: Draw the face frame coordinate data of the face image in the face frame buffer, and fuse the face frame coordinate data into the target LVGL frame buffer.
[0041] Specifically, the face coordinates of the face image are calculated by the callback thread based on the face coordinates in the face frame buffer. The face coordinates are then passed to the LVGL drawing interface. By setting the corresponding color of Sb, the face frame coordinate data of the face image is drawn and overwritten. The face frame coordinate data in the face frame buffer, which is independent of the LVGL architecture, is then moved to the target LVGL frame buffer using a DMA unit, and the merging is completed to generate a new target LVGL frame buffer.
[0042] The steps for merging the target LVGL frame buffer and the face frame buffer are as follows: 1. Calculate the width and height (W1, H1) of the obj_s object, i.e., the screen object, and the size of the ACT layer (Ws, Hs). 2. Retain the data in frame buffer B with the first W1H1 display format (bytes 1-4) unchanged; 3. Calculate Ws*Hs*Sb-W1*H1*Sb data equal to Wy*Hy*Sb, which is the remaining area. Set each x, y coordinate x*y*Sb bit in Wy*Hy*Sb to transparent (0x00). 4. Use the DMA unit to move the image data of the face frame buffer to the transparent area, and complete the merging to generate a new frame buffer B.
[0043] S50, call the flush_cb function to point the display image pointer of the face capture and display system to the target LVGL frame buffer.
[0044] Specifically, scheduling the `flush_cb` function shifts the display image pointer from the currently displayed frame buffer to the target LVGL frame buffer, preventing image tearing and displaying faces in LVGL format as shown in the image. Figure 2 .
[0045] This invention achieves efficient fusion display and page switching of LVGL interface and face image. It independently realizes the display of face image and dynamic tracking of face box without conflict with LVGL architecture. The page switching speed is fast and there will be no memory conflict or image tearing.
[0046] In one embodiment, to implement the display of face images and page switching within the LVGL system framework, the step of using a face UVC video stream device to acquire and recognize face images according to the face acquisition and recognition thread and store them in the face frame buffer can be specifically executed as follows: First, the face image is acquired using the face UVC video stream device according to the face acquisition and recognition thread. Specifically, a face frame buffer is set in the face acquisition and display system to store the face image and related information. The face acquisition and recognition thread is set with the trigger conditions and corresponding callback functions for the face acquisition and display interface. For example, the trigger condition is that the face UVC video stream device detects a face. Then, the face UVC video stream device is used to acquire and recognize the face image according to the face acquisition and recognition thread. Then, configure the face frame buffer and storage location information of the face image. Specifically, when setting the face frame buffer, a double buffering mechanism is used. Configure the face frame buffer where the face image is stored and set the index flag, i.e., the storage location information. Next, the DMA controller stores the face image into the face frame buffer according to the storage location information. The face image is then recognized using a face recognition algorithm to obtain and store the face information data. Specifically, the face frame buffer where the face image is stored is determined according to the storage location information. The face image is then stored into the face frame buffer through the DMA controller. The face image is then recognized using a face recognition algorithm to obtain and store the face information data.
[0047] In one embodiment, to display face images and switch pages within an LVGL system framework, the DMA controller stores the face image in a face frame buffer based on storage location information. The step of using a face recognition algorithm to identify the face image, obtain face information data, and then storing it can be specifically executed as follows: The face frame buffer includes a first face frame buffer and a second face frame buffer. The face acquisition and display system includes an acquisition and storage pointer. A diagram of face data acquisition and storage is shown below. Figure 3 As shown.
[0048] The storage location of the face image is determined based on the storage location information. Specifically, a double buffering mechanism is used when setting the face frame buffer. The face frame buffer includes a first face frame buffer and a second face frame buffer. The face frame buffer where the face image is stored is determined based on the storage location information. If the storage location of the face image is the first face frame buffer, then the acquisition and save pointer is pointed to the first face frame buffer, and the face image is stored in the first face frame buffer using the DMA controller. Specifically, if the storage location of the face image is the first face frame buffer, the acquisition and save pointer is pointed to the first face frame buffer, and the face image is stored in the first face frame buffer using the DMA controller. The face information data is obtained and stored by using face recognition algorithms to identify face images. Specifically, the face information data is obtained and stored by using face recognition algorithms to identify face images. Mark the first face frame buffer as completed, and then point the acquisition and save pointer to the second face frame buffer. Specifically, after completion, mark the first face frame buffer as completed, switch the acquisition and save pointer to the second face frame buffer block and set it as incomplete. If the storage location of the face image is the second face frame buffer, then the acquisition and save pointer is pointed to the second face frame buffer, and the face image is stored in the second face frame buffer using the DMA controller. Specifically, if the storage location of the face image is the second face frame buffer, the acquisition and save pointer is pointed to the second face frame buffer, and the face image is stored in the second face frame buffer using the DMA controller. The face information data is obtained and stored by using face recognition algorithms to identify face images. Specifically, the face information data is obtained and stored by using face recognition algorithms to identify face images. Mark the second face frame buffer as completed, and then point the acquisition and save pointer to the first face frame buffer. Specifically, after completion, mark the second face frame buffer as completed, switch the acquisition and save pointer to the first face frame buffer block, and set it to incomplete.
[0049] In addition to locating the storage area by collecting and saving pointers, the storage area can also be located and stored by setting a DMA address.
[0050] This application utilizes a double-buffering mechanism to acquire and store facial images, allowing the system to acquire the next frame of image data while processing the previous one. This parallel processing method improves the system's real-time performance and efficiency, ensures the continuity of the face acquisition and recognition process, and reduces data processing latency.
[0051] In one embodiment, in order to display face images and switch pages within the LVGL system framework, the step of receiving the target LVGL frame buffer can be specifically performed as follows: First, the LVGL frame buffer is received. The LVGL frame buffer includes a first LVGL frame buffer and a second LVGL frame buffer. Specifically, the frame buffer is a memory area used to store the pixel data of the current screen (i.e., the color value of each pixel). This application adopts a double buffering mechanism (first LVGL frame buffer and second LVGL frame buffer) to obtain the LVGL frame buffer. That is, the frame buffer includes the currently displayed frame buffer and the next frame buffer being drawn. Then, the direction of the display image pointer of the face capture and display system is determined. If the display image pointer points to the first LVGL frame buffer, the second LVGL frame buffer is determined as the target LVGL frame buffer. If the display image pointer points to the second LVGL frame buffer, the first LVGL frame buffer is determined as the target LVGL frame buffer. Specifically, the currently displayed frame buffer is determined by the direction of the display image pointer of the face capture and display system. If the display image pointer points to the first LVGL frame buffer, it means that the currently displayed frame buffer is the first LVGL frame buffer, and the frame buffer to be drawn is the second LVGL frame buffer, i.e., the target LVGL frame buffer. If the display image pointer points to the second LVGL frame buffer, it means that the currently displayed frame buffer is the second LVGL frame buffer, and the frame buffer to be drawn is the first LVGL frame buffer, i.e., the target LVGL frame buffer.
[0052] In one embodiment, to implement the display of face images and page switching within the LVGL system framework, the step of redrawing the target LVGL frame buffer in the ACT layer of the face capture and display system can be specifically performed as follows: First, the LVGL system framework uses LVGL periodic events for polling. Specifically, when it is necessary to display the collected faces under the LVGL system framework, the LVGL page display needs to be changed. The LVGL interface elements (background, text, border, shadow, etc.) used to draw the face display are often modified by creating and modifying the properties of screen objects in the active screen layer. The LVGL system framework uses LVGL periodic events for polling to detect whether the properties of the screen objects have changed.
[0053] Then, when a change in the properties of a screen object is detected, the area of the screen object that has changed is marked as a dirty region. The dirty region is expanded to obtain the actual redraw range. Specifically, when a change in the properties of a screen object is detected, the area of the screen object that has changed is the area that needs to be redrawn and is marked as a dirty region. LVGL will consider the expansion boundary of the object during redrawing to ensure that style effects (such as shadows, outlines, etc.) are rendered completely.
[0054] Next, the target LVGL frame buffer is redrawn in the ACT layer of the face capture and display system according to the actual redraw range. Specifically, the target LVGL frame buffer is redrawn in the ACT layer of the face capture and display system, and LVGL interface elements (background, text, border, shadow, etc.) are drawn to the target frame buffer.
[0055] In one embodiment, to implement the display of face images and page switching within the LVGL system framework, the step of marking the area where screen objects change as the dirty region and expanding the dirty region to obtain the actual redraw range can be specifically performed as follows: First, determine whether the change in the screen object's attributes is unique. Specifically, when it is necessary to display the collected faces under the LVGL system framework, it is necessary to change the LVGL page display and draw the LVGL interface elements (background, text, border, shadow, etc.) used for face display. This is often achieved by modifying the attributes of the screen object in the active screen layer. First, determine whether the change in the screen object's attributes is unique. Then, if the change in the properties of the screen object is unique, the area of the screen object that has changed is marked as a dirty area. The dirty area is expanded to obtain the actual redraw range. Specifically, when a change in the properties of the screen object is detected, and the change in the properties of the screen object is unique, the area of the screen object that has changed is the area that needs to be redrawn and is marked as a dirty area. LVGL will consider the expansion boundary of the object when redrawing to ensure that the style effects (such as shadows, outlines, etc.) are rendered completely. Next, if the changes in the properties of the screen object are not unique, then several areas of the screen object that have changed are identified and marked as several dirty areas. Specifically, when it is detected that the properties of the screen object have changed and the changes in the properties of the screen object are not unique, the areas of the screen object that have changed are areas that need to be redrawn. If there are multiple areas that need to be redrawn, these areas are marked as dirty areas. Then, several dirty regions are expanded to obtain several actual redraw sub-ranges. Specifically, LVGL will consider the expansion boundary of the object during redrawing to ensure that style effects (such as shadows, outlines, etc.) are rendered completely. The boundary of each dirty region is expanded to obtain several actual redraw sub-ranges. Finally, several dirty regions are merged to obtain the actual redrawing range, which includes several actual redrawing sub-ranges. Specifically, merging all dirty regions means merging several actual redrawing sub-ranges to obtain the total actual redrawing range.
[0056] Dirty area management and redraw mechanisms significantly improve UI update efficiency by updating only the areas that need to change, rather than the entire screen. This approach reduces unnecessary redraw operations and lowers system resource consumption, especially in scenarios involving complex UIs or frequent updates, where it can significantly improve system responsiveness and smoothness.
[0057] In one embodiment, to implement the display of face images and page switching within the LVGL system framework, the step of drawing the face frame coordinate data of the face image in the face frame buffer can be specifically performed as follows: First, set the drawing color of the face image. Specifically, the face coordinates of the face image are calculated by the callback thread based on the face coordinates in the face frame buffer to perform face tracking. The face coordinates are then passed to the LVGL drawing interface, and the corresponding color of Sb is set. Then, based on the drawing color, the face coordinates are drawn in the face frame buffer according to the face image to obtain face frame coordinate data. Specifically, the face frame coordinate data of the face image is drawn in the face frame buffer according to the drawing color, and then the face frame data is overwritten.
[0058] It achieves efficient fusion display and page switching of LVGL interface and face image, realizes face image display and dynamic tracking of face box independently of LVGL, does not conflict with LVGL architecture, has fast page switching speed and does not cause memory conflict or image tearing, and provides an efficient and flexible GUI solution for embedded systems.
[0059] Based on the above method, this application also discloses a face capture and display system. For example... Figure 4 The system includes the following modules: The screen object creation module 401 is used to create a screen object containing screen sub-objects in the active screen layer of the face acquisition and display system, and to insert LVGL controls in the screen sub-objects. The LVGL controls include the face acquisition and recognition thread. The face acquisition and storage module 402 is used to set up a face frame buffer in the face acquisition and display system, and to acquire and recognize face images according to the face acquisition and recognition thread using the face UVC video stream device and store them in the face frame buffer. The frame buffer redrawing module 403 is used to receive the target LVGL frame buffer and redraw the target LVGL frame buffer in the ACT layer of the face capture and display system. The fusion module 404 is used to draw the face frame coordinate data of the face image in the face frame buffer and fuse the face frame coordinate data into the target LVGL frame buffer. The face image display module 405 is used to call the flush_cb function to point the display image pointer of the face acquisition and display system to the target LVGL frame buffer.
[0060] In one embodiment, the face acquisition and storage module 402 is specifically used to acquire face images using a face UVC video stream device according to a face acquisition and recognition thread; configure the face frame buffer and storage location information of the face images; use a DMA controller to store the face images into the face frame buffer according to the storage location information; and use a face algorithm to recognize the face images to obtain face information data and store it.
[0061] In one embodiment, the face acquisition and storage module 402 is specifically used to determine the storage location of the face image based on the storage location information; if the storage location of the face image is the first face frame buffer, the acquisition and storage pointer is pointed to the first face frame buffer, and the face image is stored in the first face frame buffer using the DMA controller; the face image is recognized using a face algorithm to obtain face information data and stored; the storage of the first face frame buffer is marked as complete, and the acquisition and storage pointer is pointed to the second face frame buffer; if the storage location of the face image is the second face frame buffer, the acquisition and storage pointer is pointed to the second face frame buffer, and the face image is stored in the second face frame buffer using the DMA controller; the face image is recognized using a face algorithm to obtain face information data and stored; the storage of the second face frame buffer is marked as complete, and the acquisition and storage pointer is pointed to the first face frame buffer.
[0062] In one embodiment, the frame buffer redraw module 403 is specifically used to receive an LVGL frame buffer, which includes a first LVGL frame buffer and a second LVGL frame buffer; determine the direction of the display image pointer of the face acquisition and display system; if the display image pointer points to the first LVGL frame buffer, then the second LVGL frame buffer is determined as the target LVGL frame buffer; if the display image pointer points to the second LVGL frame buffer, then the first LVGL frame buffer is determined as the target LVGL frame buffer.
[0063] In one embodiment, the frame buffer redraw module 403 is specifically used to poll the LVGL periodic events of the LVGL system framework; when a change in the attributes of a screen object is detected, the area of the screen object where the change has occurred is marked as a dirty area, and the dirty area is expanded to obtain the actual redraw range; the target LVGL frame buffer is redrawn in the ACT layer of the face acquisition and display system according to the actual redraw range.
[0064] In one embodiment, the frame buffer redraw module 403 is specifically used to determine whether the change in the attributes of the screen object is unique; if the change in the attributes of the screen object is unique, the area where the screen object has changed is marked as a dirty region, and the dirty region is expanded to obtain the actual redraw range; if the change in the attributes of the screen object is not unique, several areas where the screen object has changed are determined, and several areas are marked as several dirty regions; the several dirty regions are expanded to obtain several actual redraw sub-ranges; the several dirty regions are merged to obtain the actual redraw range, and the actual redraw range includes several actual redraw sub-ranges.
[0065] In one embodiment, the fusion module 404 is specifically used to set the drawing color of the face image; and to draw the face coordinates in the face frame buffer according to the face image based on the drawing color to obtain face frame coordinate data.
[0066] This application also discloses a computer device.
[0067] Specifically, the computer device includes a memory and a processor, and the memory stores a computer program that can be loaded and executed by the processor to perform the aforementioned LVGL-based face capture and display method.
[0068] This application also discloses a computer-readable storage medium.
[0069] Specifically, the computer-readable storage medium stores a computer program that can be loaded and executed by a processor, such as the aforementioned LVGL-based face capture and display method. The computer-readable storage medium includes, for example, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0070] This specific embodiment is merely an explanation of the present invention and is not intended to limit the invention. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of the claims of the present invention.
Claims
1. A face capture and display method based on LVGL, characterized in that, The method is based on a face capture and display system using the LVGL system framework and a face UVC video streaming device. The method includes: In the active screen layer of the face acquisition and display system, a screen object containing screen sub-objects is created, and an LVGL control is inserted into the screen sub-object. The LVGL control includes a face acquisition and recognition thread. A face frame buffer is set in the face acquisition and display system. The face UVC video stream device is used to acquire and recognize face images according to the face acquisition and recognition thread and store them in the face frame buffer. Receive the target LVGL frame buffer and redraw the target LVGL frame buffer in the ACT layer of the face acquisition and display system; The face frame coordinate data of the face image is drawn in the face frame buffer, and the face frame coordinate data is fused into the target LVGL frame buffer; The flush_cb function is called to point the display image pointer of the face capture and display system to the target LVGL frame buffer.
2. The method according to claim 1, characterized in that, The method is also based on a DMA controller, wherein the step of using the face UVC video stream device to acquire and recognize face images according to the face acquisition and recognition thread and storing them in the face frame buffer includes: The face image is acquired using the face UVC video stream device according to the face acquisition and recognition thread; Configure the face frame buffer and storage location information of the face image; The DMA controller stores the face image into the face frame buffer according to the storage location information, and the face image is recognized by the face algorithm to obtain face information data and then stored.
3. The method according to claim 2, characterized in that, The face frame buffer includes a first face frame buffer and a second face frame buffer. The face acquisition and display system includes an acquisition and storage pointer. The step of using the DMA controller to store the face image into the face frame buffer according to the storage location information, and using a face algorithm to recognize the face image to obtain and store face information data includes: The storage location of the face image is determined based on the storage location information; If the storage location of the face image is the first face frame buffer, then the acquisition and storage pointer is pointed to the first face frame buffer, and the face image is stored in the first face frame buffer using the DMA controller; The face image is identified using a face recognition algorithm to obtain face information data, which is then stored. Mark the first face frame buffer as completed, and set the acquisition and saving pointer to the second face frame buffer; If the storage location of the face image is the second face frame buffer, then the acquisition and storage pointer is pointed to the second face frame buffer, and the face image is stored in the second face frame buffer using the DMA controller; The face image is identified using a face recognition algorithm to obtain face information data, which is then stored. Mark the second face frame buffer as completed and set the acquisition and save pointer to the first face frame buffer.
4. The method according to claim 3, characterized in that, The target LVGL frame buffer includes: Receive LVGL frame buffer, wherein the LVGL frame buffer includes a first LVGL frame buffer and a second LVGL frame buffer; Determine the direction of the display image pointer of the face capture and display system; If the displayed image pointer points to the first LVGL frame buffer, then the second LVGL frame buffer is determined to be the target LVGL frame buffer; If the display image pointer points to the second LVGL frame buffer, then the first LVGL frame buffer is determined to be the target LVGL frame buffer.
5. The method according to claim 4, characterized in that, The redrawing of the target LVGL frame buffer in the ACT layer of the face capture and display system includes: Polling is performed through LVGL periodic events in the LVGL system framework; When a change in the properties of the screen object is detected, the area of the screen object where the change has occurred is marked as a dirty area, and the dirty area is expanded to obtain the actual redraw range. The ACT layer of the face capture and display system redraws the target LVGL frame buffer according to the actual redraw range.
6. The method according to claim 5, characterized in that, The area where the screen object changes is marked as a dirty region, and expanding the dirty region to obtain the actual redraw range includes: Determine whether the change in the screen object's attribute is unique; If the change in the attributes of the screen object is unique, then the area of the screen object that has changed is marked as a dirty area, and the dirty area is expanded to obtain the actual redraw range; If the changes in the attributes of the screen object are not unique, then determine several regions of the screen object that have changed, and mark these regions as several dirty regions. Expanding several of the aforementioned dirty regions yields several actual redrawn sub-ranges; The actual redrawing range is obtained by merging several of the dirty regions, and the actual redrawing range includes several of the actual redrawing sub-ranges.
7. The method according to claim 6, characterized in that, The face frame coordinate data used to draw the face image in the face frame buffer includes: Set the drawing color of the face image; Based on the drawing color, face coordinates are drawn in the face frame buffer according to the face image to obtain face frame coordinate data.
8. A face capture and display system, characterized in that, The system is based on a face capture and display system using the LVGL system framework and a face UVC video streaming device. The system includes: The screen object creation module (401) is used to create a screen object containing screen sub-objects in the active screen layer of the face acquisition and display system, and to insert an LVGL control in the screen sub-object, wherein the LVGL control includes a face acquisition and recognition thread. The face acquisition and storage module (402) is used to set a face frame buffer in the face acquisition and display system, and to acquire and recognize face images according to the face acquisition and recognition thread using the face UVC video stream device and store them in the face frame buffer. The frame buffer redrawing module (403) is used to receive the target LVGL frame buffer and redraw the target LVGL frame buffer in the ACT layer of the face acquisition and display system. The fusion module (404) is used to draw the face frame coordinate data of the face image in the face frame buffer and fuse the face frame coordinate data into the target LVGL frame buffer; The face image display module (405) is used to call the flush_cb function to point the display image pointer of the face acquisition and display system to the target LVGL frame buffer.
9. A computer device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer program is stored that can be loaded by a processor and executed according to any one of claims 1 to 7.