Robotic eye display method and system

Abstract

This invention provides a robot eye display method and system. The robot eye display method includes the following steps: S1, the main control module receives external interaction signals; the shared rendering module generates a unified rendering frame containing the left-eye and right-eye images using a single rendering; the unified rendering frame is constructed based on multiple graphics layers to simulate the structure of the human eye; S2, the binocular synchronization control module receives the unified rendering frame and generates homogeneous line synchronization and field synchronization signals; S3, the display output module drives the left-eye display module and the right-eye display module to synchronously display the left-eye and right-eye images according to the homogeneous line synchronization and field synchronization signals. This invention provides high human eye simulation accuracy, natural and dynamic display, and improved human-computer interaction friendliness; the binocular synchronization is precise, there is no image fragmentation, and the display is smooth and stable.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more specifically to a method and system for displaying robot eyes. Background Technology

[0002] With the popularization of service robots, bionic robots, and pet robots, the naturalness and friendliness of human-computer interaction have become core requirements for improving user experience. As the core visual carrier of human-computer interaction, the robot's eyes directly determine the user's perception and acceptance of the robot. Robot eye displays need to simulate human or biological eye movements and facial expressions, achieving functions such as blinking, eye movement, and emotional expression (e.g., happiness, sadness, focus, fatigue). This requires ensuring display synchronization and smoothness while also considering the low power consumption and low cost requirements of embedded platforms, and adapting to various display carriers such as OLED, LCD, LED dot matrix, and projection.

[0003] Currently, binocular display technology for robots is widely used in scenarios such as home service robots, commercial guided robots, educational robots, and bionic pet robots. Its core requirement is to convey the robot's state and emotions through the synergy of binocular displays, reducing the sense of unfamiliarity in human-computer interaction. Existing binocular display solutions for robots are mainly based on independent dual-screen driving, independent rendering, or simple image replication. However, with the increasing demands of users for interactive experiences, and the limitations of embedded platform computing power and memory resources, existing solutions can no longer meet the practical application requirements of high precision, high naturalness, and low power consumption. There is an urgent need for a binocular display implementation solution that can resolve the contradictions between synchronization, smoothness, and resource consumption.

[0004] Existing binocular display solutions for robots suffer from numerous technical flaws in both hardware architecture design and software rendering control, resulting in harsh display effects, excessive resource consumption, and poor adaptability. The specific flaws are as follows: 1. Asynchronous display between the two eyes results in a strong sense of visual disconnect and a poor interactive experience. This is the most critical flaw in existing technology, mainly manifested in two aspects: First, the left and right eyes are driven and refreshed independently, using two independent rendering channels and display control modules, each receiving rendering instructions and completing screen refresh. Due to slight differences in the clock source and rendering rhythm of the two systems, problems such as unsynchronized left and right eye frames, screen tearing, and unsynchronized blinking can easily occur. For example, the left eye may have completed the blinking action while the right eye is still open, or there may be a delay difference of 1-2 frames between the left and right eye images, which seriously damages the continuity and naturalness of eye display. Second, the screen-drive synchronization is insufficient. The horizontal and vertical synchronization signals of the left and right eye display modules have different sources, resulting in screen refresh misalignment and inconsistent brightness. In some scenarios, the left and right eye images may even shift, making it impossible to form a unified eye visual effect and greatly reducing the affinity of human-computer interaction.

[0005] Second, excessive computing power and video memory usage, poor compatibility, and susceptibility to stuttering.

[0006] Most existing solutions employ a single-eye independent rendering mode, where each eye has its own independent rendering logic, video memory, and drawing channel. This requires separate rendering, caching, and output of the left and right eye images. This mode necessitates storing two independent sets of eye image data, doubling video memory usage, and executing two rendering tasks simultaneously, significantly increasing CPU / GPU processing power. Since most robots use embedded platforms with limited computing power and video memory resources, rendering stutters and frame drops are common, especially when rendering complex expressions such as rapid eye movements and continuous blinking. The stuttering is even more pronounced when rendering complex expressions, such as rapid eye movements or multiple consecutive blinks, making it impossible to guarantee smooth display. Furthermore, excessive resource consumption increases robot power consumption, shortening battery life and hindering mass production applications of portable and home-use robots.

[0007] 3. The eye posture and facial expression control are not coordinated, resulting in a stiff appearance.

[0008] In existing technologies, the posture control of the left and right eyes (such as eyeball rotation angle and blinking speed) and the presentation of emotional expressions (such as pupil size, highlight position, and eye deformation) mostly adopt independent control logic, lacking a unified expression engine scheduling. For example, when controlling the left eyeball rotation, it cannot be guaranteed that the right eye's rotation angle and speed are completely consistent with the left eye; when presenting a happy expression, the highlight distribution and eye curvature of the left and right eyes differ, resulting in uncoordinated eye movements and stiff expressions, failing to simulate the natural state of real biological eyes. In addition, although some solutions use a simple image copying method, directly copying the left eye image to the right eye, while ensuring image consistency, they cannot achieve independent eyeball positioning and differentiated expression fine-tuning (such as slight squinting of one eye), resulting in poor flexibility and failing to meet diverse interaction needs.

[0009] Fourth, the hardware structure is complex and costly, which is not conducive to mass production.

[0010] To achieve independent rendering and driving for the left and right eyes, most existing technical solutions require two independent rendering modules, display driver chips, and video memory units. This results in complex hardware structures and an increased number of components, which not only increases the size of the robot's eye module but also significantly raises hardware costs. Furthermore, debugging two independent hardware systems is difficult, requiring separate calibration of display parameters for the left and right eyes (such as brightness, contrast, and image shift). This cumbersome calibration process reduces production efficiency and hinders the large-scale mass production of robots. In addition, some solutions employ display driving methods with poor compatibility, unable to adapt to different types of display carriers (e.g., OLED and LED dot matrix screens cannot share a single driving logic), resulting in weak versatility and increased adaptation costs.

[0011] 5. Inadequate power consumption control results in poor battery life.

[0012] Due to its dual-rendering-channel and dual-drive-module design, the current solution consistently consumes a high level of power. Even when the robot is in sleep or low-load mode, both rendering and drive units continue to operate, making synchronized sleep and screen-off control impossible. Furthermore, in independent rendering mode, invalid rendering operations increase (such as repeatedly rendering the same content in the left and right eye images), further increasing power consumption. For battery-powered portable robots and home service robots, excessive power consumption significantly shortens battery life, impacting the actual user experience and limiting the robot's application scenarios.

[0013] VI. The display effect is too simplistic and cannot meet diverse interactive needs.

[0014] Most existing solutions can only achieve simple screen display and basic blinking and eye movement functions, lacking support for complex emotional expressions. They cannot dynamically adjust the eye display effect according to the robot's working status (such as busy, idle, or malfunctioning) and user interaction scenarios (such as greetings, comforting, and reminders). At the same time, the screen rendering is not flexible enough to achieve fine presentation of eye details (such as pupil dilation, sclera reflection, and eye shadows). The display effect is monotonous and rigid, unable to convey rich emotional information, and fails to meet users' high-end demands for natural human-computer interaction.

[0015] Based on the numerous shortcomings of existing robot binocular display technologies, such as poor synchronization, high resource consumption, stiff expressions, complex hardware, excessive power consumption, and limited display effects, which cannot meet the mass production and application requirements of robots for high precision, high naturalness, low cost, and low power consumption, the inventors of this application have designed a robot eye display method and system to overcome the above-mentioned technical problems. Summary of the Invention

[0016] The technical problem to be solved by the present invention is to overcome the many defects of existing robot eye display technology, such as poor synchronization, high resource consumption, stiff expressions, complex hardware, high power consumption, and single display effect, as well as the inability to adapt to the mass production and application requirements of robots with high precision, high naturalness, low cost and low power consumption, and to provide a robot eye display method and system.

[0017] The present invention solves the above-mentioned technical problems through the following technical solution: This invention provides a robot eye display method, characterized in that the robot eye display method includes the following steps: S1, the main control module receives external interaction signals; the shared rendering module generates a unified rendering frame containing the left eye image and the right eye image using a single rendering; the unified rendering frame is constructed based on the superposition of multiple graphics layers to simulate the structure of the human eye; S2, the binocular synchronization control module receives the unified rendering frame and generates homogeneous line synchronization and field synchronization signals; S3, the display output module drives the left eye display module and the right eye display module to synchronously display the left eye image and the right eye image according to the homogeneous line synchronization and field synchronization signals.

[0018] According to one or more embodiments of the present invention, in step S1, the multi-layer pattern layer includes, from bottom to top, a sclera layer, an eyeball layer, a pupil layer, a highlight layer, an eyelid bottom layer, an eyelid middle layer, and an eyelid top layer, which are stacked sequentially from bottom to top.

[0019] According to one or more embodiments of the present invention, in step S1, the shared rendering module independently controls the display state and / or motion parameters and / or transparency of at least one of the graphics layers to generate dynamic eye expressions.

[0020] According to one or more embodiments of the present invention, in step S1, when the main control module does not receive an external interaction signal, the robot enters a silent interaction mode; when the main control module receives an external interaction signal, the robot switches to an active interaction mode.

[0021] According to one or more embodiments of the present invention, in silent interactive mode, generating the unified rendering frame in step S1 includes the following steps: S 11 Based on preset robot personality parameters, a basic expression is randomly determined; S 12 Based on the basic expression, control the dynamic rendering of the eyeball layer, pupil layer, and highlight layer; randomly trigger and control the top and bottom layers of the eyelid to perform blinking actions.

[0022] According to one or more embodiments of the present invention, in the active interaction mode, generating the unified rendering frame in step S1 includes the following steps: S 11 When an interactive object is detected, the eye layer and pupil layer are rendered to ensure that the pupil's orientation follows the position of the interactive object; 12 Based on the facial expressions of the interacting object or the frequency and duration of touch operations, the parameters of pupil size and blinking are dynamically adjusted.

[0023] According to one or more embodiments of the present invention, the following steps are further included before step S1: Step S 01 Step S: Start the robot and display the startup status screen in the unified rendering frame.02 1. Detect the working status of each hardware module; if a hardware abnormality is detected, the corresponding fault prompt information is displayed synchronously in the unified rendering frame; if no abnormality is detected, proceed to step S1.

[0024] This invention also provides a robot eye display system, characterized in that it is used to implement the robot eye display method described above. The robot eye display system includes a main control module, a shared rendering module, a binocular synchronization control module, and a display output module. The main control module is used to receive external interaction signals. The shared rendering module is electrically connected to the main control module and is used to generate a unified rendering frame containing left-eye and right-eye images using a single rendering. The binocular synchronization control module is electrically connected to the main control module and the shared rendering module and is used to receive the unified rendering frame and generate homogeneous line synchronization and field synchronization signals. The display output module includes a left-eye display module and a right-eye display module, which are respectively electrically connected to the binocular synchronization control module. The display output module drives the left-eye display module and the right-eye display module to synchronously display the left-eye and right-eye images according to the homogeneous line synchronization and field synchronization signals.

[0025] According to one or more embodiments of the present invention, the main control module adopts the Horizon X5 embedded main control chip.

[0026] According to one or more embodiments of the present invention, the main control module integrates a unified expression engine unit, which is configured with multiple preset basic expressions and eye movements, for generating expression instructions to control at least one graphics layer parameter in the shared rendering module based on external interaction signals.

[0027] According to one or more embodiments of the present invention, the shared rendering module is integrated into the main control module, and the shared rendering module is based on the LVGL graphics library or the OpenGL ES rendering engine, and constructs the unified rendering frame by superimposing multiple graphics layers.

[0028] According to one or more embodiments of the present invention, the binocular synchronization control module includes an HDMI splitter and a synchronization clock module; the HDMI splitter is used to receive HDMI signals from the main control module and convert them into two MIPI signals for output to the display output module; the synchronization clock module is used to generate horizontal and vertical synchronization signals of the same origin to ensure that the two MIPI signals are synchronized.

[0029] According to one or more embodiments of the present invention, the left eye display module and the right eye display module adopt a dot matrix color screen, and the robot eye display system includes only one main control module and one shared rendering module.

[0030] The robot eye display method and system of the present invention have at least the following advantages: With high human eye simulation accuracy, the display is natural and dynamic, enhancing the user-friendliness of human-computer interaction; precise binocular synchronization, no screen breaks, and smooth and stable display; significantly reduced computing power and video memory usage, compatible with embedded platforms, and longer battery life; simple hardware structure, low cost, and easy mass production; flexible facial expression control, adapting to diverse interaction needs; strong anti-interference capability, and stable and reliable operation. Attached Figure Description

[0031] The above and other features, properties and advantages of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings and embodiments, in which the same reference numerals always denote the same features, wherein: Figure 1 This is a schematic diagram of an embodiment of the robot eye display system of the present invention.

[0032] Figure 2 This is a schematic diagram of the installation structure of an embodiment of the robot eye display system of the present invention. Detailed Implementation

[0033] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0034] Reference will now be made in detail to preferred embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals will be used in all the drawings to denote the same or similar parts. Furthermore, although the terminology used herein is selected from commonly known and used terms, some terms mentioned in this specification may have been chosen by the applicant at his or her discretion, and their detailed meanings are explained in the relevant sections of the description herein. Moreover, the invention should be understood not only by the actual terminology used, but also by the meaning implied by each term. Also, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale.

[0035] This invention provides a method for displaying robot eyes, comprising the following steps: Step S1: The main control module 100 receives external interaction signals; the shared rendering module 200 generates a unified rendering frame containing the left eye image and the right eye image using a single rendering; the unified rendering frame is constructed based on the superposition of multiple graphics layers to simulate the structure of the human eye. Step S2: The binocular synchronization control module 300 receives the unified rendering frame and generates line synchronization and field synchronization signals of the same origin. Step S3: The display output module 400 drives the left eye display module 410 and the right eye display module 420 to synchronously display the left eye image and the right eye image according to the same horizontal and vertical synchronization signals.

[0036] In a preferred embodiment of the robot eye display method of the present invention, in step S1, the multi-layer graphic layer includes, from bottom to top, a sclera layer, an eyeball layer, a pupil layer, a highlight layer, an eyelid bottom layer, an eyelid middle layer, and an eyelid top layer.

[0037] Preferably, the multi-layer graphics layer adopts a multi-layer display method that simulates the physiological structure of the human eye, achieved through seven layers of LVGL (Lightweight and Multifunctional Graphics Library) stacked together, accurately reproducing the visual hierarchy and detail features of the human eye. The seven layers, from bottom to top, are: The first layer, the sclera layer, is used to simulate the sclera area of ​​the human eye, presenting the base color and natural texture of the sclera.

[0038] The second eyeball layer is used to simulate the human eyeball itself, carrying the eyeball's outline and basic color tone.

[0039] The third pupil layer is used to simulate the human pupil, and can achieve scaling and position shifting to reproduce the characteristics of the pupil changing with light and emotions.

[0040] The fourth highlight layer is used to simulate the highlight points on the human eye pupil, enhancing the three-dimensionality and vividness of the eyes.

[0041] The fifth layer, the bottom layer of the eyelid, is used to simulate the lower eyelid of a human eye and control the opening and closing range of the lower eyelid.

[0042] The sixth layer, the middle layer of the eyelid, is used to simulate the folds and textures of the human eyelid, enhancing the biomimetic realism.

[0043] The seventh layer, the top layer of the eyelid, is used to simulate the upper eyelid of the human eye, achieving a dynamic representation of natural blinking.

[0044] Each layer is rendered and overlaid independently, with individual control over display status, motion parameters, and transparency, thus accurately reproducing the natural shape and dynamic characteristics of the human eye. The shared rendering module 200 generates a unified rendering frame containing the left and right eye images. Each frame contains a complete seven-layer structure, completing the unified rendering of the human eye image, facial expressions, and eye movements, significantly reducing system computing power and video memory usage.

[0045] In a preferred embodiment of the robot eye display method of the present invention, in step S1, the shared rendering module 200 independently controls the display state and / or motion parameters and / or transparency of at least one of the graphics layers to generate dynamic eye expressions.

[0046] In a preferred embodiment of the robot eye display method of the present invention, in step S1, when the main control module 100 does not receive an external interaction signal, the robot enters a silent interaction mode; when the main control module 100 receives an external interaction signal, the robot switches to an active interaction mode.

[0047] In a preferred embodiment of the robot eye display method of the present invention, in silent interaction mode, generating the unified rendering frame in step S1 includes the following steps: Step S 11 Based on preset robot personality parameters, the basic expression is randomly determined.

[0048] Step S 12 Based on the basic expression, control the dynamic rendering of the eyeball layer, pupil layer, and highlight layer; randomly trigger and control the top and bottom layers of the eyelid to perform blinking actions.

[0049] Preferably, after the robot has started up and all hardware modules have been tested and found to be normal, the robot automatically enters the silent interaction mode. In this mode, when there is no external interaction (no touch operation, no face detection), the binocular display will strictly follow the robot's preset personality parameters (such as docile, lively, introverted) and randomly present the basic expressions that match them (such as focused, relaxed, lazy, etc.).

[0050] In the LVGL layering, the second eye layer, the third pupil layer, and the fourth highlight layer are rendered independently and dynamically superimposed according to the current expression, and output synchronously to the left and right eye displays. The screen refresh rate is kept stable at 120Hz, without stuttering or tearing, and can delicately and realistically display complex emotional expressions. For example, a docile robot will show a gentle pupil dilation and contraction and a soft distribution of highlights, while a lively robot will show a rapid and slight shift in pupils and a dynamic flickering of highlights, restoring the emotional expression characteristics of real human eyes.

[0051] Meanwhile, in silent mode, random blinking expressions will be interspersed. By precisely controlling the falling speed of the top layer of the seventh eyelid, the rising speed of the bottom layer of the fifth eyelid, and the opening and closing amplitude of the two eyelids, different intensities and rhythms of blinking effects (such as slow and gentle blinking, fast and light blinking, and intermittent blinking) will be presented to avoid the eyes appearing too stiff and further enhance the bionic naturalness.

[0052] In a preferred embodiment of the robot eye display method of the present invention, in active interaction mode, generating the unified rendering frame in step S1 includes the following steps: Step S 11 When an interactive object is detected, the eye layer and pupil layer are rendered so that the pupil orientation follows the position of the interactive object.

[0053] Step S 12 Based on the facial expressions of the interacting object or the frequency and duration of touch operations, the parameters of pupil size and blinking are dynamically adjusted.

[0054] Preferably, when the main control module 100 detects an external interaction signal (such as a camera detecting a face or a touch module detecting a touch operation), the robot automatically switches to the active interaction mode. The binocular display will match the corresponding eye interaction state in real time according to the robot's preset personality model and emotional strategy model to achieve a human-like linkage response.

[0055] Taking a home setting as an example, when the robot's camera detects a face, it quickly collects facial feature information and compares it with the built-in face database to accurately identify the identity of the interactive object (such as a child or an adult) and facial expressions (such as smiling, frowning, or surprised). If a child in the family is detected smiling at the robot, the main control module 100 will simultaneously send control commands. On the one hand, it drives the head motor to turn the robot's head towards the face. On the other hand, it controls the second eye layer and the third pupil layer in the binocular LVGL layer to adjust the orientation of the eyeballs in the display area in real time according to the face position (ensuring that the pupils are always aligned with the face). At the same time, it dynamically adjusts the pupil size (the pupils dilate appropriately during smiling interactions to present a pleasant state) to match the smiling expression. Simultaneously, it links the voice module to issue a friendly response voice and the motion module to perform interactive actions such as rubbing or nodding, achieving a coordinated response of vision, motion, and voice.

[0056] When a child touches the preset touch area on the robot's head, the touch sensing module quickly collects the touch signal. After de-jittering and anti-interference processing, the signal is transmitted to the main control module 100. The main control module 100 immediately triggers the binocular blinking feedback mechanism. The blinking effect is precisely linked to the child's stroking operation. When the stroking frequency is fast, the opening and closing speed of the top layer of the seventh eyelid and the bottom layer of the fifth eyelid is accelerated, and the blinking amplitude is reduced, presenting a light blinking feedback. When the stroking frequency is slow and the duration is long, the opening and closing speed of the eyelids is slowed down, and the blinking amplitude is increased, presenting a gentle blinking feedback. If multiple consecutive touches are detected, a continuous blinking effect will be triggered to simulate the pleasant reaction of a human being stroked, further enhancing the affinity and naturalness of human-computer interaction, making the robot's eye response more in line with human emotional expression habits.

[0057] In a preferred embodiment of the robot eye display method of the present invention, the following steps are included before step S1: Step S 01 Start the robot and display the startup status screen in the unified rendering frame; Step S 021. Detect the working status of each hardware module; if a hardware abnormality is detected, the corresponding fault prompt information is displayed synchronously in the unified rendering frame; if no abnormality is detected, proceed to step S1.

[0058] Preferably, after the robot is powered on and started, the binocular display system is activated simultaneously. The LVGL top layer prioritizes the display of the system startup logo, startup progress bar, and hardware initialization status prompts. The display is clear and intuitive, making it easy for operators to understand the startup process in real time.

[0059] During startup, the main control module 100 monitors the working status of each hardware module (including the touch sensing module, camera module, head motor module, binocular display module, etc.) in real time. If any hardware module is detected to be abnormal (such as touch module communication failure, camera not starting properly, motor jamming, etc.), the corresponding fault icon and clear fault prompt text will be displayed synchronously on the binocular interface, along with concise and easy-to-understand solution guidance (such as "Touch module abnormal, please check the wiring" and "Camera not started, please restart the device"). Problems can be quickly troubleshooted without professional personnel, improving the convenience of equipment maintenance.

[0060] See Figures 1-2 The present invention provides a robot eye display system for implementing the robot eye display method described above. The robot eye display system includes a main control module 100, a shared rendering module 200, a binocular synchronization control module 300, and a display output module 400.

[0061] The main control module 100 is used to receive external interactive signals.

[0062] The shared rendering module 200 is electrically connected to the main control module 100. The shared rendering module 200 is used to generate a unified rendering frame containing the left eye image and the right eye image in a single rendering.

[0063] The binocular synchronization control module 300 is electrically connected to the main control module 100 and the shared rendering module 200. The binocular synchronization control module 300 is used to receive the unified rendering frame and generate line synchronization and field synchronization signals of the same origin. The display output module 400 includes a left-eye display module 410 and a right-eye display module 420. The left-eye display module 410 and the right-eye display module 420 are electrically connected to the binocular synchronization control module 300. The display output module 400 drives the left-eye display module 410 and the right-eye display module 420 to synchronously display the left-eye image and the right-eye image according to the same horizontal and vertical synchronization signals.

[0064] As a preferred embodiment of the robot eye display system of the present invention, the main control module 100 adopts the Horizon X5 embedded main control chip.

[0065] The Horizon X5 embedded main control chip was selected to meet the requirements for rendering data caching and command processing. USB and Ethernet interfaces are reserved for receiving external interactive commands and status feedback.

[0066] The main control module 100 serves as the core scheduling center of the entire robot eye display system. It adopts the Horizon X5 embedded main control chip, an 8-core A55, with a built-in 10 TOPS BPU. It is responsible for receiving external interaction commands (such as user operations and robot status commands), scheduling the work of each module, and synchronously controlling rendering, display, and expression logic. It features low power consumption and high computing power and is compatible with embedded robot platforms.

[0067] Preferably, the Horizon X5 embedded main control chip uses an HDMI signal cable and an HDMI splitter, and has a built-in synchronization clock module to generate identical horizontal synchronization (HSYNC) and vertical synchronization (VSYNC) signals, ensuring that the left and right eye display modules 420 receive rendering frames and refresh the screen synchronously, eliminating frame delay and screen tearing.

[0068] As a preferred embodiment of the robot eye display system of the present invention, the main control module 100 integrates a unified expression engine unit, which is configured with multiple preset basic expressions and eye movements, and is used to generate expression instructions to control at least one graphics layer parameter in the shared rendering module 200 according to external interaction signals.

[0069] Preferably, the unified expression engine unit is integrated into the main control module 100, which is responsible for the unified scheduling of eye posture and emotional expression. It presets a variety of basic expressions (happy, aggrieved, focused, tired) and eye movements (blinking, eyeball rotation, pupil dilation and contraction), receives instructions from the main control module 100, and synchronously controls the posture parameters of the left and right eyes to achieve coordinated eye movement and expression presentation.

[0070] In a preferred embodiment of the robot eye display system of the present invention, the shared rendering module 200 is integrated into the main control module 100. The shared rendering module 200 is based on the LVGL graphics library or the OpenGL ES rendering engine and constructs the unified rendering frame by superimposing multiple graphics layers.

[0071] The shared rendering module 200 is based on the LVGL graphics library or the OpenGL ES rendering engine. It adopts a single rendering and dual-eye sharing mode, without the need to set up an independent rendering channel.

[0072] Preferably, the shared rendering module 200 can generate a unified image for both eyes in a single rendering without the need for an additional independent rendering chip, relying on the hardware acceleration function of the main control module 100 and the LVGL graphics library; the video memory uses the built-in RAM of the main control module 100, eliminating the need for additional video memory expansion and reducing hardware cost and size.

[0073] As a preferred embodiment of the robot eye display system of the present invention, the binocular synchronization control module 300 includes an HDMI splitter and a synchronization clock module.

[0074] The HDMI splitter is used to receive HDMI signals from the main control module 100 and convert them into two MIPI signals for output to the display output module 400.

[0075] The synchronization clock module is used to generate identical horizontal and vertical synchronization signals to ensure that the two MIPI signals are synchronized.

[0076] Preferably, the binocular synchronization control module 300 includes an HDMI splitter (one-to-two screen splitter), and the driver chip is LT6911C, which converts the HDMI signal into dual MIPI signals. It is connected to the Horizon X5 embedded main control chip of the main control module 100 via an HDMI high-definition signal cable, and outputs two MIPI signals to the driver interfaces of the left and right eye display modules 420 respectively. At the same time, it integrates a signal buffer circuit to avoid synchronization signal attenuation and ensure the synchronous response of the left and right eyes.

[0077] In a preferred embodiment of the robot eye display system of the present invention, the left eye display module 410 and the right eye display module 420 adopt a dot matrix color screen, and the robot eye display system includes only a main control module 100 and a shared rendering module 200.

[0078] like Figure 2 As shown, the left eye display module 410 and the right eye display module 420 can be connected to the main control module 100 and the shared rendering module 200 via the bridge board 500.

[0079] Preferably, the display output module 400 uses two independent display carriers (compatible with OLED, LCD, LED dot matrix screen or projection module), namely the left eye display module 410 and the right eye display module 420, which serve as the robot's left and right eyes respectively. They are connected to the binocular synchronization control module 300 through the HDIM interface to receive synchronous refresh signals and present a unified eye image. At the same time, it supports synchronous calibration of brightness and contrast to ensure consistent display effects for both eyes. It can achieve a high refresh rate of 120Hz without any stuttering, ensuring smooth display.

[0080] Preferably, both eyes use 2.8-inch dot matrix color screens (480×480 resolution), which support MIPI interface communication, have fast response speed and low power consumption, and are suitable for the miniaturization requirements of robot eyes; the dot matrix color screens are connected to the binocular synchronization control module 300 to receive synchronous refresh signals and realize synchronous image updates; the position and angle of the left and right eyes are calibrated during screen installation to ensure a unified visual eye effect.

[0081] The robot's eye display hardware of this invention adopts a minimalist design, eliminating the need for two independent rendering and driving modules, thus significantly simplifying the structure and reducing costs. The overall system solution adopts a modular architecture combining a single rendering core and dual display outputs. The core consists of a main control module 100, a shared rendering module 200, a binocular synchronization control module 300, and a display output module 400, among other modules. These modules work collaboratively to achieve synchronized display of both eyes and natural facial expressions.

[0082] As a preferred embodiment of the robot eye display system of the present invention, the robot eye display system also includes a power supply module. The power supply module adopts JW5361, which takes 12V DC voltage as input and outputs a stable 5V voltage to power each module. It integrates reverse connection protection and overcurrent protection circuits to avoid damage to the modules due to abnormal power supply. At the same time, it is designed with a low power supply mode, supports synchronous sleep and screen-off control, and reduces battery consumption.

[0083] The robot eye display system of this invention adopts a structure combining single main control, single rendering, and dual output. Compared with the existing solution, it reduces one rendering and driving module, reduces the number of components by 30%, reduces hardware cost by 40%, and shrinks size by 50%. Moreover, it is simple to debug, requiring only one calibration of display parameters, which is convenient for mass production.

[0084] The robot eye display method and system of the present invention, compared with the existing robot binocular display schemes, combines human eye multi-layer simulation design and LVGL 7-layer layer stacking technology, which has significant technical advantages and application value, solves the core defects of the existing technology, and is quantifiable and verifiable.

[0085] In summary, the robot eye display method and system of the present invention have the following advantages: I. High degree of human eye simulation, natural and dynamic display, enhancing the affinity of human-computer interaction. The robot eye display method and system of this invention adopts an LVGL 7-layer layered design that simulates the structure of the human eye. The 7-layer structure accurately corresponds to the sclera, eyeball, pupil, highlights, and eyelids (bottom / middle / top layers) from bottom to top, completely restoring the visual layers and details of the human eye. Each layer independently controls transparency and motion parameters, which can accurately simulate real human eye movements such as pupil dilation, natural blinking, and eyeball rotation. At the same time, through the refined presentation of eyelid folds, sclera texture, and pupil highlights, it completely solves the problems of stiff display and lack of detail in existing solutions, making the robot's eyes more biomimetic, effectively reducing the unfamiliarity of human-computer interaction, and improving the user experience.

[0086] 2. Precise binocular synchronization, no image fragmentation, smooth and stable display.

[0087] The robot eye display method and system of this invention adopts a design that combines shared rendering and homogeneous synchronous control. The shared rendering module generates a unified image for the left and right eyes in a single rendering (each containing 7 layers of human eye simulation structure). The binocular synchronous control module generates homogeneous VSYNC and HSYNC signals to trigger simultaneous refresh of the left and right eyes, ensuring that the refresh time difference between the left and right eye images is <10μs, with no frame delay, no screen tearing, and no blinking asynchrony. At the same time, through an abnormal calibration mechanism, refresh deviation is dynamically compensated, completely solving the core defects of existing technologies such as binocular asynchrony and screen fragmentation, and improving display smoothness by more than 80%.

[0088] Third, significantly reduced computing power and video memory usage, better adapted to embedded platforms, and longer battery life.

[0089] This invention abandons the existing independent rendering mode and adopts a combination of single shared canvas and single rendering. Combined with the efficient rendering logic of LVGL 7-layer layering, it eliminates the need to repeatedly render the left and right eye images, reducing memory usage by 50% compared to existing solutions and CPU / GPU computing power consumption by more than 40%. At the same time, it designs low-power logic such as on-demand rendering and synchronous sleep mode, reducing system power consumption by more than 60% and increasing battery life by more than 50% in sleep mode. It is perfectly adapted to embedded robot platforms such as STM32 and ESP32, avoiding problems such as lag and insufficient battery life.

[0090] IV. Simple hardware structure, low cost, and easy to mass-produce.

[0091] This invention adopts a minimalist hardware architecture combining a single main controller, single rendering, and dual outputs. It eliminates the need for two independent rendering and driving units, reducing the number of components by 30%, hardware costs by 40%, and module size by 50%. At the same time, it eliminates the need for separate calibration of left and right eye display parameters, simplifying the debugging process. It is compatible with various display carriers such as OLED, LCD, and LED dot matrix screens, and has strong versatility. It effectively solves the shortcomings of existing solutions, such as complex hardware, high cost, cumbersome debugging, and difficulty in mass production, significantly improving production efficiency and reducing mass production costs.

[0092] V. Flexible facial expression control to adapt to diverse interaction needs

[0093] This invention achieves coordinated control of binocular posture and expression through a unified expression engine. It has multiple built-in basic expression templates and supports custom expression parameters and individual eye-differentiation fine-tuning. Based on the LVGL 7-layer layered design, the parameters of each layer can be flexibly adjusted to achieve a refined presentation of complex emotional expressions. It can ensure binocular coordination and consistency while meeting diverse interaction needs such as slight squinting and highlight shift in one eye. It solves the problems of limited expression and poor flexibility in existing solutions and is suitable for various robot interaction scenarios such as home, commercial, and educational applications.

[0094] VI. Strong anti-interference capability and stable and reliable operation.

[0095] The robot eye display system of this invention integrates reverse connection protection and overcurrent protection circuits at the hardware level to avoid power abnormalities. The signal line uses a high-definition shielded HDMI cable harness to reduce the impact of electromagnetic interference on the display system. At the software level, mechanisms such as cyclic caching and abnormal calibration are used to ensure stable rendering and refresh, reducing problems such as frame drops and abnormal display. The overall system operates stably and reliably, with a failure rate that is more than 70% lower than that of existing solutions, and is suitable for complex application environments such as industrial and home use.

[0096] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of the present invention is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but all such changes and modifications fall within the scope of protection of the present invention.

Claims

1. A method for displaying robot eyes, characterized in that, The robot eye display method includes the following steps: S1. The main control module receives external interactive signals; the shared rendering module generates a unified rendering frame containing the left-eye image and the right-eye image using a single rendering; the unified rendering frame is constructed based on the superposition of multiple graphics layers to simulate the structure of the human eye. S2. The binocular synchronization control module receives the unified rendering frame and generates line synchronization and field synchronization signals of the same origin. S3. The display output module drives the left eye display module and the right eye display module to synchronously display the left eye image and the right eye image according to the same source of line synchronization and field synchronization signals.

2. The robot eye display method as described in claim 1, characterized in that, In step S1, the multi-layered pattern includes the sclera layer, eyeball layer, pupil layer, highlight layer, eyelid bottom layer, eyelid middle layer and eyelid top layer, which are stacked sequentially from bottom to top.

3. The robot eye display method as described in claim 1, characterized in that, In step S1, the shared rendering module independently controls the display state and / or motion parameters and / or transparency of at least one of the graphics layers to generate dynamic eye expressions.

4. The robot eye display method as described in claim 2, characterized in that, In step S1, when the main control module does not receive an external interaction signal, the robot enters a silent interaction mode; when the main control module receives an external interaction signal, the robot switches to an active interaction mode.

5. The robot eye display method as described in claim 4, characterized in that, In silent interactive mode, generating the unified rendering frame in step S1 includes the following steps: S 11 Based on preset robot personality parameters, the basic expression is randomly determined; S 12 Based on the basic expression, control the dynamic rendering of the eyeball layer, pupil layer, and highlight layer; randomly trigger and control the top and bottom layers of the eyelid to perform blinking actions.

6. The robot eye display method as described in claim 4, characterized in that, In the active interaction mode, generating the unified rendering frame in step S1 includes the following steps: S 11 When an interactive object is detected, the eye layer and pupil layer are rendered so that the pupil orientation follows the position of the interactive object. S 12 Based on the facial expressions of the interacting object or the frequency and duration of touch operations, the parameters of pupil size and blinking are dynamically adjusted.

7. The robot eye display method as described in claim 2, characterized in that, The following steps are included before step S1: Step S 01 Start the robot and display the startup status screen in the unified rendering frame; Step S 02 1. Detect the working status of each hardware module; if a hardware abnormality is detected, the corresponding fault prompt information is displayed synchronously in the unified rendering frame. If no abnormalities are detected, proceed to step S1.

8. A robot eye display system, characterized in that, For implementing the robot eye display method as described in any one of claims 1-7, the robot eye display system includes a main control module, a shared rendering module, a binocular synchronization control module, and a display output module; The main control module is used to receive external interactive signals; The shared rendering module is electrically connected to the main control module. The shared rendering module is used to generate a unified rendering frame containing the left eye image and the right eye image using a single rendering. The binocular synchronization control module is electrically connected to the main control module and the shared rendering module. The binocular synchronization control module is used to receive the unified rendering frame and generate line synchronization and field synchronization signals of the same origin. The display output module includes a left-eye display module and a right-eye display module. The left-eye display module and the right-eye display module are electrically connected to the binocular synchronization control module. The display output module drives the left-eye display module and the right-eye display module to synchronously display the left-eye image and the right-eye image according to the same horizontal synchronization and vertical synchronization signals.

9. The robot eye display system as described in claim 8, characterized in that, The main control module uses the Horizon X5 embedded main control chip.

10. The robot eye display system as described in claim 8, characterized in that, The main control module integrates a unified expression engine unit, which is configured with multiple preset basic expressions and eye movements, and is used to generate expression commands that control at least one graphics layer parameter in the shared rendering module based on external interaction signals.

11. The robot eye display system as described in claim 8, characterized in that, The shared rendering module is integrated into the main control module. The shared rendering module is based on the LVGL graphics library or the OpenGL ES rendering engine and constructs the unified rendering frame by overlaying multiple graphics layers.

12. The robot eye display system as described in claim 8, characterized in that, The binocular synchronization control module includes an HDMI splitter and a synchronization clock module; The HDMI splitter is used to receive HDMI signals from the main control module and convert them into two MIPI signals for output to the display output module; The synchronization clock module is used to generate identical horizontal and vertical synchronization signals to ensure that the two MIPI signals are synchronized.

13. The robot eye display system as described in claim 8, characterized in that, The left eye display module and the right eye display module adopt a dot matrix color screen, and the robot eye display system includes only one main control module and one shared rendering module.

Images