Robot and expression display method

Abstract

This application provides a robot and an expression display method; the robot includes an expression display device and a processor; the expression display device includes: an array light panel connected to a driving circuit; a diffuser disposed on the light-emitting side of the array light panel, with a light-transmitting window and a light-shielding layer between the light-transmitting windows, for allowing light to exit from the light-transmitting window and suppressing light crosstalk through the light-shielding layer; a front panel disposed on the light-emitting side of the diffuser, for displaying the expression image; a driving circuit connected to the processor, for outputting a driving signal to the array light panel according to the received target dot matrix data; the processor is used to: acquire target expression information; map the target expression information into a first vector in a continuous semantic space; determine at least one target expression atom based on the first vector and a plurality of preset discrete expression atoms in a continuous semantic space second vector; fuse the dot matrix data corresponding to each target expression atom to obtain target dot matrix data; and send the target dot matrix data to the driving circuit.

Description

Technical Field

[0001] This application relates to robotics technology, and more particularly to a robot and a method for displaying facial expressions. Background Technology

[0002] In scenarios such as companionship, education, and service, robots create a human-like interactive experience through facial expressions. Currently, the main solutions for realizing robot facial expressions include mechanically driven simulated faces, projection / display screens, and dot-matrix facial expressions. However, mechanically driven simulated faces are structurally complex and costly; projection / display screens have a strong sense of component presence and are costly and power-consuming; while dot-matrix facial expression solutions are easy to integrate, have low power consumption, and are low cost, they are difficult to express complex or subtle emotions. Summary of the Invention

[0003] This application provides a robot and a method for displaying facial expressions.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a robot, which includes an expression display device and a processor; the expression display device includes: The array light panel is connected to the driver circuit. A diffuser is disposed on the light-emitting side of the array lamp panel. The diffuser has a light-transmitting window that corresponds to each light-emitting unit on the array lamp panel, and a light-shielding layer is disposed in the area between each light-transmitting window. The diffuser is used to allow the light emitted by the light-emitting unit to be emitted from the corresponding light-transmitting window, and the light-shielding layer suppresses cross-lighting of light emitted from adjacent light-transmitting windows. A front panel is disposed on the light-emitting side of the diffuser; the front panel is used to display an expression image composed of light emitted from each of the light-transmitting windows; A driving circuit is connected to the processor; the driving circuit is used to receive target dot matrix data sent by the processor and output a driving signal to the array light board according to the target dot matrix data. The processor is configured to: acquire target facial expression information; map the target facial expression information into a first vector in a continuous semantic space; determine at least one target facial expression atom from the plurality of discrete facial expression atoms based on the first vector and a plurality of preset second vectors in the continuous semantic space; wherein the distance between the second vector of each target facial expression atom and the first vector in the continuous semantic space is less than a preset threshold, and each target facial expression atom has a preset corresponding dot matrix data; fuse the dot matrix data corresponding to each target facial expression atom to obtain target dot matrix data; and send the target dot matrix data to the driving circuit.

[0005] This application provides an expression display method applied to a robot equipped with an expression display device. The expression display device includes an array light panel, a diffuser, a front panel, and a driving circuit. The array light panel is connected to the driving circuit. The diffuser is disposed on the light-emitting side of the array light panel, and has light-transmitting windows corresponding to each light-emitting unit on the array light panel. A light-shielding layer is disposed in the area between each light-transmitting window. The diffuser allows light emitted by each light-emitting unit to exit through the corresponding light-transmitting window, and the light-shielding layer suppresses cross-lighting from adjacent light-transmitting windows. The front panel is disposed on the light-emitting side of the diffuser and is used to display an expression image composed of light emitted from each light-transmitting window. The driving circuit receives target dot matrix data and outputs a driving signal to the array light panel according to the target dot matrix data. The method includes: Obtain the target's facial expression information; The target facial expression information is mapped to a first vector in a continuous semantic space; Based on the first vector and the second vectors of a plurality of discrete facial expression atoms in the continuous semantic space, at least one target facial expression atom is determined from the plurality of discrete facial expression atoms; wherein, the distance between the second vector of each target facial expression atom in the continuous semantic space and the first vector is less than a preset threshold, and each target facial expression atom has a corresponding dot matrix data. The dot matrix data corresponding to each of the target facial expression atoms are fused to obtain the target dot matrix data; The target dot matrix data is sent to the driving circuit. Attached Figure Description

[0006] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application. It is obvious that the drawings described below are some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0007] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0008] Figure 1 This is a schematic diagram of the structure of a robot provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of an array lamp panel provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a diffuser provided in an embodiment of this application; Figure 4 This is one of the flowcharts illustrating an expression display method provided in an embodiment of this application; Figure 5 This is a second schematic flowchart of an expression display method provided in an embodiment of this application; Figure 6 This is the third flowchart illustrating an expression display method provided in this application embodiment; Figure 7 This is a schematic diagram of the parameter configuration of an expression atom provided in an embodiment of this application; Figure 8 This is a schematic diagram of the architecture of a robot facial expression interaction system provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of an expression display device provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of an expression display device provided in an embodiment of this application.

[0009] Figure label: 10: Expression display device; 101: Array light panel; 102: Diffuser; 1021: Light transmission window; 103: Front panel; 104: Drive circuit; 20: Processor. Detailed Implementation

[0010] To make the technical solutions and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0011] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0012] In the following description, references to "some embodiments," "this embodiment," "this application embodiment," and examples, etc., describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments and may be combined with each other without conflict.

[0013] The terms "first," "second," and "third" appearing in the embodiments of this application do not have a specific meaning (e.g., they are not in any particular order, nor do they indicate a special limitation on the number of devices in the embodiments of this application). They are used to clearly describe the embodiments of this application and do not constitute any limitation on the embodiments of this application. The term "multiple" appearing in the embodiments of this application refers to two or more integers.

[0014] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies or terms of the embodiments of this application are described below. The following related technologies or related terms are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0015] Robot facial expression output typically serves two purposes: firstly, it provides operational status prompts such as battery level, charging status, network status, fault / alarm status, and task progress; secondly, it presents emotional / intent expressions to the user.

[0016] While existing facial expression implementations using projection / display screens offer strong image performance, the overall presentation often has a noticeable device-like feel, and the appearance differs between screen-off and screen-on states, easily creating a disjointed "screen-attached" look and failing to achieve a natural, integrated facial effect. In contrast, dot matrix facial expression solutions offer advantages such as low cost, low power consumption, and ease of integration. However, conventional dot matrix solutions often rely on preset patterns or scattered rules to generate expressions, lacking an engineering-reusable expression asset organization method and a unified semantic driving mechanism. This makes it difficult for expressions and status cues to form a stable expressive system, resulting in stiff expression presentations and high maintenance costs across different dot matrix specifications, platforms, and version iterations.

[0017] In related technologies, facial expression display and emotional interaction mainly include the following types of solutions: (1) Mechanically driven simulated face. For example, robots such as Kobian and Han use motors to drive simulated human facial bones and "muscles" to achieve actions such as blinking, raising eyebrows, and opening the mouth, resulting in realistic expressions. However, this solution has a complex structure, high cost and weight, is difficult to maintain, occupies a large space, and is difficult to modularize and mass-produce, making it unsuitable for small or lightweight mobile robots.

[0018] (2) Projection / Display + Mask / Transparent Cover Solution. For example, the Furhat robot uses a hybrid face design with a rear projection display and a semi-transparent mask. This solution has relatively lightweight hardware, good programmability, and easy expression updates. However, it lacks three-dimensional texture, is difficult to integrate with the overall shape of the robot, is sensitive to light and viewing angle, and has high requirements for display resolution, brightness, and protection, resulting in higher cost and power consumption, which limits its application in outdoor or industrial environments.

[0019] (3) Simple LED / LED strip signal scheme. Some robots use LED strips, color and brightness changes to express state or emotion, such as using lights to draw diagrams in collaborative robots. This scheme is low in cost, simple in structure and easy to integrate, but it is difficult to express complex or subtle emotions, has a low degree of anthropomorphism and cannot present dynamic changes with "facial expression".

[0020] At the software and interaction levels, related technologies utilize Large Language Models (LLMs) to assess the emotions of conversation history and generate facial expressions accordingly, making it easier for users to perceive the robot's emotions when facial expressions are displayed. However, most of these technologies remain at the software level and lack systematic integration with specific hardware forms, emotion models, and multimodal behaviors.

[0021] The inventors of this application discovered in their research and analysis that the related technologies have significant shortcomings in the following aspects: limited richness and naturalness of emotional expression, especially insufficient subtlety in expressing mixed and transitional emotions; lack of modularity and size adaptability in hardware solutions, making it difficult to adapt to different models and appearance designs; difficulty in balancing low cost, lightweight design, and high expressiveness, affecting commercial mass production; complex structures or strong environmental dependence of mechanical simulation face and rear projection screen solutions, resulting in insufficient maintainability and reliability; and the fact that existing LLM+ expression generation solutions are mostly not deeply coupled with specific hardware expression devices and emotion computing models, failing to form a unified, computable, and scalable emotion interaction system.

[0022] To address at least one of the aforementioned problems in related technologies, embodiments of this application provide a robot and an expression display method. This method can suppress inter-pixel crosstalk and improve expression display contrast on a low-cost dot matrix display structure by using an array of light panels in conjunction with a diffuser having a light-transmitting window and a light-shielding layer. Simultaneously, it maps expression semantics into vectors in a continuous semantic space, and then retrieves and fuses discrete expression atoms based on the distance of vectors in the continuous semantic space to express complex or subtle emotions, thereby achieving continuous generation and asset reuse of dot matrix expressions.

[0023] Figure 1 This is a schematic diagram of the structure of a robot provided in an embodiment of this application; as shown... Figure 1 As shown, the robot includes an expression display device 10 and a processor 20; the expression display device 10 includes: The array lamp board 101 is connected to the driving circuit 104; A diffuser 102 is disposed on the light-emitting side of the array lamp panel 101. The diffuser 102 has light-transmitting windows that correspond one-to-one with each light-emitting unit on the array lamp panel 101, and a light-shielding layer is disposed in the area between each light-transmitting window. The diffuser 102 is used to allow the light emitted by the light-emitting unit to be emitted from the corresponding light-transmitting window, and the light-shielding layer suppresses cross-lighting of light emitted from adjacent light-transmitting windows. A front panel 103 is disposed on the light-emitting side of the diffuser 102; the front panel 103 is used to display an expression image composed of light emitted from each of the light-transmitting windows; A driving circuit 104 is connected to the processor 20; the driving circuit 104 is used to receive target dot matrix data sent by the processor 20 and output a driving signal to the array lamp board 101 according to the target dot matrix data. The processor 20 is configured to: acquire target facial expression information; map the target facial expression information into a first vector in a continuous semantic space; determine at least one target facial expression atom from the plurality of discrete facial expression atoms based on the first vector and a plurality of preset second vectors of discrete facial expression atoms in the continuous semantic space; wherein the distance between the second vector of each target facial expression atom and the first vector in the continuous semantic space is less than a preset threshold, and each target facial expression atom has a preset corresponding dot matrix data; fuse the dot matrix data corresponding to each target facial expression atom to obtain target dot matrix data; and send the target dot matrix data to the driving circuit 104.

[0024] It should be noted that the robot in the embodiments of this application can be a mobile or stationary robot that has a need for facial expression display, such as a companion robot, educational robot, service robot or industrial robot.

[0025] In some embodiments, the expression display device 10 is disposed on the facial area of ​​the robot for displaying expression patterns and / or animations in a dot matrix manner.

[0026] In some embodiments, the array lamp panel 101 includes a plurality of independently driveable light-emitting units, which are regularly arranged on the array lamp panel 101.

[0027] In some embodiments, the array lamp board 101 is an aluminum-based printed circuit board. A large area of ​​copper foil is provided on the back of the aluminum-based printed circuit board, and heat is conducted through the aluminum substrate to improve heat dissipation performance, thereby meeting the heat dissipation requirements of high-brightness continuous display.

[0028] For example, Figure 2 This is a schematic diagram of the structure of an array lamp panel provided in an embodiment of this application; as shown... Figure 2As shown, the array light panel 101 has dimensions of 165.3mm × 50mm; it has 24 pixel columns along the long side and 8 pixel rows along the short side, for a total of 192 light-emitting units. Each light-emitting unit includes an LED chip with independently controllable R, G, and B colors.

[0029] In some embodiments, each row or adjacent multiple rows of light-emitting units on the array lamp board 101 are connected to the driving circuit 104 in series and / or parallel. The driving circuit 104 is connected to the processor 20 through a serial peripheral interface (SPI) or an inter-integrated circuit (I²C) bus.

[0030] It should be noted that the light-emitting side of the array light panel 101 refers to the side of the array light panel 101 from which light is emitted, that is, the side facing the user and used to display facial expressions. The opposite side is the backlight side (or non-light-emitting side), which is usually used for wiring or heat dissipation. In this embodiment, the light emitted by the light-emitting unit (LED chip) on the array light panel 101 is emitted from the light-emitting side, passes through the diffuser 102 and the front panel 103 in sequence, and is finally seen by the user.

[0031] In some embodiments, the diffuser 102 is made of transparent polycarbonate (PC) or polymethyl methacrylate (PMMA).

[0032] In some embodiments, the light-transmitting window provided on the diffuser 102 is a square light-transmitting window, which matches the shape of the dot matrix pixels formed by each light-emitting unit on the array lamp panel 101.

[0033] In some embodiments, the light-shielding layer disposed in the area between each light-transmitting window on the diffuser 102 is a black matte coating formed by spraying, which is used to absorb light leakage between adjacent light-transmitting windows and suppress light crosstalk.

[0034] For example, Figure 3 This is a schematic diagram of the structure of a diffuser provided in an embodiment of this application; as shown... Figure 3 As shown, the diffuser 102 has regularly arranged light-transmitting windows 1021, and the shape of the light-transmitting windows 1021 is square. Each light-transmitting window 1021 corresponds to a light-emitting unit on the array lamp board. The area between the light-transmitting windows 1021 is sprayed with a black matte coating to absorb light leakage and improve pixel contrast. The thickness and diffusion characteristics of the diffuser 102 are optimized so that the light emitted by multiple light-emitting units is uniformly mixed in the corresponding light-transmitting windows 1021 before being emitted. The light-shielding layer suppresses cross-lighting from adjacent light-transmitting windows 1021, thereby forming a clear mosaic pixel effect.

[0035] It should be noted that the light-emitting side of the diffuser 102 refers to the side from which light is emitted, i.e., the side facing the front panel 103 and away from the array lamp panel 101. After the light emitted by the light-emitting unit exits from the light-emitting side of the array lamp panel 101, it enters the corresponding light-transmitting window from the light-incident side of the diffuser 102. After being uniformly mixed within the light-transmitting window, it exits from the light-emitting side of the diffuser 102 and is then transmitted through the front panel 103 for the user to view. In other words, the front panel 103 is used to display the facial expression image composed of the light emitted from each light-transmitting window.

[0036] In some embodiments, the driving circuit 104 is connected to the processor 20. The driving circuit 104 is used to receive target dot matrix data sent by the processor 20 and output driving signals to the array lamp board 101 according to the target dot matrix data. The driving circuit 104 may include a constant current driving chip, which is connected to the processor 20 via an SPI or I²C bus. Each row or each adjacent multiple rows of light-emitting units on the array lamp board 101 are connected to the constant current driving chip in series and / or parallel.

[0037] In some embodiments, the processor 20 may be a microcontroller unit (MCU) or a system on chip (SoC).

[0038] In some embodiments, the processor 20 is connected to the driving circuit 104 via a frame buffer output interface to send the generated dot matrix frame buffer data (i.e., target dot matrix data) to the driving circuit 104 so that the driving circuit 104 drives the array light panel 101 to emit light, which, after passing through the diffuser 102, displays the corresponding facial expression on the front panel 103.

[0039] In some embodiments, the processor 20 can generate target dot matrix data including color and brightness information, and achieve pixel-by-pixel independent control of each light-emitting unit on the array light panel 101 via the driving circuit 104; the processor 20 can also generate dot matrix data frame by frame according to a preset frame rate and send it to the driving circuit 104. The driving circuit 104 drives each light-emitting unit on the array light panel 101 to emit light according to the received dot matrix data, so as to display the corresponding facial expression or facial animation.

[0040] In some embodiments, the processor 20 runs an expression display driver program to execute the expression display method. The processor 20 is used to generate LED driving timing (timing signals that control the lighting of light-emitting units row by row or column by column) and pulse width modulation (PWM) dimming signals according to the target expression information output by the upper application layer, and to control the brightness and color of each light-emitting unit on the array lamp board 101 through the driving circuit 104.

[0041] It should be noted that the continuous semantic space is a continuous multidimensional space used to describe the semantics of facial expressions, and its coordinate axes represent different semantic dimensions. In the embodiments of this application, the continuous semantic space is preferably the PAD emotion space, that is, a three-dimensional space with pleasure, arousal and dominance as coordinate axes, and the value range of each dimension is [-1, 1].

[0042] It should be noted that discrete expression atoms are multiple pre-defined, independent basic expression atoms. Each expression atom includes dot matrix pattern data (i.e., the on / off state and color information of each pixel when displaying an expression on the array light panel 101), a corresponding second vector in the continuous semantic space (the coordinate position of the expression in the continuous semantic space (such as the PAD emotion space), a type label (such as emotion category (happy, sad, angry), status category (low battery, charging, standby), intention category (greeting, warning)), and performance parameters (such as color, brightness, rhythm, etc.). The multiple discrete expression atoms are discretely distributed in the continuous semantic space, that is, these expression atoms are isolated, finite points in the semantic space, and cannot cover the entire semantic space.

[0043] It should be noted that dot matrix data is used to describe the color and brightness information of each pixel in a frame of the displayed image on the array light board.

[0044] It should be noted that the target facial expression information is the semantic information of the expression used to drive the generation of facial expressions. The target facial expression information may include at least one of the following categories: interaction information between the user and the robot (such as the user's voice, text, etc.), normal operating status information of the robot (such as battery level, task progress, network status, etc.), abnormal fault information of the robot (such as motor failure, communication interruption, etc.), and safety alarm information of the robot (such as obstacle avoidance triggering, approach to dangerous areas, etc.).

[0045] For example, when a user says "I won the lottery," the processor 20 obtains the emotional information of "happy" through voice acquisition and semantic understanding as the target facial expression information. Similarly, when the robot's battery level is below 5%, the processor 20 obtains the status information of "low battery" as the target facial expression information.

[0046] In some embodiments, the processor 20 can simultaneously acquire user interaction information and system status information, and then use both the acquired user interaction information and system status information as target expression information.

[0047] In this embodiment, the processor 20 can map the target facial expression information into a first vector in a continuous semantic space. For example, when the target facial expression information is a single label (such as "happy"), the processor 20 retrieves the corresponding vector according to a preset label-vector mapping table, such as (+0.8, +0.3, +0.4). As another example, when the target facial expression information includes system status events, the processor 20 retrieves the corresponding first vector according to a preset system status-vector mapping table, for example, "low battery" is mapped to (-0.5, 0.2, -0.6).

[0048] In some embodiments, when the target expression information includes multiple emotion information and their probabilities, the processor 20 maps each emotion information into a vector in a continuous semantic space, and then performs a weighted average with the probability corresponding to each emotion information as the weight to obtain a first vector. For example, the vector corresponding to "happy" is (+0.8, +0.3, +0.4), with a probability of 0.6; the vector corresponding to "sad" is (-0.9, +0.2, -0.8), with a probability of 0.3; the vector corresponding to "surprised" is (-0.2, +0.7, -0.3), with a probability of 0.1; then the first vector is: P=0.8×0.6+(-0.9)×0.3+(-0.2)×0.1=0.19, A=0.3×0.6+0.2×0.3+0.7×0.1=0.31, D=0.4×0.6+(-0.8)×0.3+(-0.3)×0.1=-0.03, that is, the first vector is (0.19, 0.31, -0.03).

[0049] In some embodiments, the Euclidean distance between the first vector and the second vector of each preset discrete facial expression atom can be calculated. Then, all facial expression atoms whose distance is less than a preset threshold are selected as target facial expression atoms. Each target facial expression atom has corresponding dot matrix data preset. For example, if the first vector is (0.19, 0.31, -0.03) and the preset threshold is 0.8, then all facial expression atoms whose second vector is less than 0.8 from the first vector are selected as target facial expression atoms.

[0050] It should be noted that the specific value of the preset threshold can be adjusted according to the required expression precision. The smaller the threshold, the more accurate the selected expression atoms; the larger the threshold, the wider the coverage.

[0051] In some embodiments, the K (K≥1) discrete facial expression atoms closest to the first vector can be selected as the target facial expression atoms.

[0052] In some embodiments, all facial expression atoms whose distance is less than a preset threshold can be identified first. If no facial expression atom falls within the threshold range, the closest facial expression atom is selected as the target facial expression atom.

[0053] In some embodiments, the processor 20 can synthesize the dot matrix data of multiple target expression atoms into a new dot matrix data, namely target dot matrix data, to generate an expression image corresponding to the first vector.

[0054] In some embodiments, when there is only one target expression atom, the processor 20 can directly use the dot matrix data corresponding to the target expression atom as the target dot matrix data.

[0055] In some embodiments, when there are multiple target expression atoms, the processor 20 can determine the fusion weight corresponding to each target expression atom based on the distance between the first vector and the second vector of each target expression atom. The closer the distance, the greater the weight. Then, pixel-level weighted mixing is performed on the dot matrix data corresponding to each target expression atom, that is, the values ​​of each dot matrix data are weighted and summed at the corresponding pixel positions to obtain the target dot matrix data.

[0056] In some embodiments, the processor 20 may fuse the dot matrix data corresponding to each target facial expression atom using an equal-weighted method.

[0057] In some embodiments, the processor 20 can send target dot matrix data to the driving circuit 104 via SPI or I²C bus. After receiving the target dot matrix data, the driving circuit 104 outputs driving signals (such as row / column scanning timing and PWM dimming signals) to the array lamp board 101 according to the target dot matrix data, driving each light-emitting unit on the array lamp board 101 to emit light. The light passes through the diffuser 102 and displays an expression image composed of light on the front panel 103.

[0058] In this embodiment, the processor 20 maps the target facial expression information into a first vector in a continuous semantic space. Since the coordinates in the continuous semantic space can take continuous values, any corresponding facial expression can be generated by changing the value of the first vector, enabling the robot to express complex or subtle emotions, breaking through the limitation of traditional solutions that can only express a limited number of preset facial expressions. Then, the target facial expression atom is retrieved based on the distance between the first vector and the second vector of multiple discrete facial expression atoms in the continuous semantic space. The dot matrix data corresponding to the target facial expression atom is then fused to obtain the target dot matrix data, making the facial expression generation process independent of the specific dot matrix size and resolution. When adapting to different models, only the underlying hardware parameters need to be adjusted, without redesigning the facial expression materials, realizing the cross-platform reuse of facial expression assets and effectively reducing development and maintenance costs. Furthermore, in response to the defect of insufficient display contrast caused by light crosstalk between pixels in dot matrix displays, this application embodiment uses an array lamp board 101 in conjunction with a diffuser 102 having a light-transmitting window and a light-shielding layer. This allows the light from each light-emitting unit to be emitted from the corresponding light-transmitting window and the light-shielding layer to suppress light crosstalk between adjacent light-transmitting windows, which significantly improves the contrast and pixel clarity of the dot matrix display, enabling robots with low-cost dot matrix structures to present clear and delicate facial expressions.

[0059] In some embodiments, the front panel 103 is a transparent panel; the front panel 103 includes a light-shielding layer, which is disposed on the side of the front panel 103 facing the diffuser 102, and has gaps at positions directly opposite each of the light-transmitting windows along the light emission direction; The light-shielding layer is used to make the front panel 103 match the color of the housing of the expression display device 10, and the gaps in the light-shielding layer are used to allow light emitted from the light-transmitting window to pass through in order to display the expression image.

[0060] In some embodiments, the front panel 103 is made of transparent PMMA material.

[0061] In this embodiment, the back side of the front panel 103 (the side facing the diffuser 102) is coated with a light-shielding layer that matches the color of the housing of the expression display device 10, and gaps are left at the positions directly opposite each light-transmitting window to form a light-transmitting area. In this way, when the lights are off, the front panel 103 and the housing of the expression display device 10 are integrated, achieving "invisibility when the screen is off"; when the lights are on, light shines out from the light-transmitting area (gap), displaying a clear dot matrix pattern, achieving "visibility when the screen is on".

[0062] In some embodiments, the front panel 103 is integrated with the housing of the expression display device 10 and is fixed to the robot's head frame by screws, clips, or adhesive. In the dark state, the user sees only a complete panel, while when the light is on, the light-transmitting area displays a dot matrix pattern, enhancing the anthropomorphic "facial" effect.

[0063] It is understood that, in this embodiment, by providing a light-shielding layer of the same color as the housing of the expression display device 10 on the side of the front panel 103 facing the diffuser 102, the front panel 103 and the housing of the expression display device 10 become integrated when the light-emitting unit is not emitting light, achieving "screen-off invisibility." Simultaneously, by leaving gaps in the light-shielding layer at positions directly opposite the light-transmitting windows, light can pass through these gaps to display the expression image when the light-emitting unit is emitting light, achieving "screen-on display." Therefore, there is no obvious mechanical feel when the screen is off, and the image is clearly visible when the screen is on, improving the robot's overall appearance and expression display effect.

[0064] This application also provides an expression display method, which can be applied to a robot equipped with an expression display device. The specific structure of the expression display device can be found in the descriptions of the array light panel 101, diffuser 102, front panel 103, and driving circuit 104 in the foregoing embodiments, and will not be repeated here. The expression display method provided in this application can be executed by the processor 20 included in the robot in the foregoing embodiments.

[0065] Figure 4 This is one of the flowcharts illustrating an expression display method provided in this application embodiment, such as... Figure 4 As shown, the method includes steps 401 to 405: Step 401: Obtain the target facial expression information.

[0066] It should be noted that the target facial expression information is the semantic information of the expression used to drive the generation of the expression, including the interaction information between the user and the robot (such as emotion / intent labels) and / or the robot's system status information (such as operating status, abnormal failures, safety alarms, etc.).

[0067] In some embodiments, the user's voice input is acquired through a voice acquisition module, the voice is recognized to obtain text information, and then the corresponding emotion / intent label is obtained through semantic understanding as the target expression information. For example, when a user says "I won the lottery," the emotion label "happy" is obtained through semantic understanding.

[0068] In some embodiments, system status events are obtained from the robot's status monitoring module as target facial expression information. For example, when the battery level is below 5%, the status information "low battery" is used as the target facial expression information.

[0069] In some embodiments, user interaction information and system status information can be acquired simultaneously, and then both the acquired user interaction information and system status information can be used as target facial expression information.

[0070] Step 402: Map the target facial expression information into a first vector in a continuous semantic space.

[0071] It should be noted that the continuous semantic space is a continuous multidimensional space used to describe the semantics of facial expressions, and its coordinate axes represent different semantic dimensions. In the embodiments of this application, the continuous semantic space is preferably the PAD emotion space, that is, a three-dimensional space with pleasure, arousal and dominance as coordinate axes, and the value range of each dimension is [-1, 1].

[0072] In this embodiment, the target facial expression information can be mapped to a first vector in a continuous semantic space. For example, when the target facial expression information is a single label (such as "happy"), the corresponding vector is obtained by querying a preset label-vector mapping table, such as (+0.8, +0.3, +0.4). As another example, when the target facial expression information includes system status events, the corresponding first vector is obtained by querying a preset system status-vector mapping table; for example, "low battery" is mapped to (-0.5, 0.2, -0.6).

[0073] In some embodiments, when the target expression information includes multiple emotion information and their probabilities, the processor 20 maps each emotion information into a vector in a continuous semantic space, and then performs a weighted average with the probability corresponding to each emotion information as the weight to obtain a first vector. For example, the vector corresponding to "happy" is (+0.8, +0.3, +0.4), with a probability of 0.6; the vector corresponding to "sad" is (-0.9, +0.2, -0.8), with a probability of 0.3; the vector corresponding to "surprised" is (-0.2, +0.7, -0.3), with a probability of 0.1; then the first vector is: P=0.8×0.6+(-0.9)×0.3+(-0.2)×0.1=0.19, A=0.3×0.6+0.2×0.3+0.7×0.1=0.31, D=0.4×0.6+(-0.8)×0.3+(-0.3)×0.1=-0.03, that is, the first vector is (0.19, 0.31, -0.03).

[0074] Step 403: Based on the first vector and the second vectors of a plurality of preset discrete facial expression atoms in the continuous semantic space, determine at least one target facial expression atom from the plurality of discrete facial expression atoms; wherein, the distance between the second vector of each target facial expression atom in the continuous semantic space and the first vector is less than a preset threshold, and each target facial expression atom has a preset corresponding dot matrix data.

[0075] It should be noted that discrete expression atoms are multiple pre-defined, independent basic expression atoms. Each expression atom includes dot matrix pattern data (i.e., the on / off state and color information of each pixel when displaying an expression on the array light panel 101), a corresponding second vector in the continuous semantic space (the coordinate position of the expression in the continuous semantic space (such as the PAD emotion space), a type label (such as emotion category (happy, sad, angry), status category (low battery, charging, standby), intention category (greeting, warning)), and performance parameters (such as color, brightness, rhythm, etc.). The multiple discrete expression atoms are discretely distributed in the continuous semantic space, that is, these expression atoms are isolated, finite points in the semantic space, and cannot cover the entire semantic space.

[0076] It should be noted that dot matrix data is used to describe the color and brightness information of each pixel in a frame of the displayed image on the array light board.

[0077] It should be noted that, in this embodiment, the distance between the first vector and the second vectors of each discrete facial expression atom is calculated. The magnitude of the distance determines the proximity of the first vector's position in the continuous semantic space to each discrete facial expression atom; the closer the distance, the closer the semantics. The specific value of the preset threshold can be adjusted according to the required facial expression precision. The smaller the threshold, the more precise the selected facial expression atom; the larger the threshold, the wider the coverage.

[0078] In some embodiments, the Euclidean distance between the first vector and the second vector of each preset discrete facial expression atom can be calculated. Then, all facial expression atoms whose distance is less than a preset threshold are selected as target facial expression atoms. Each target facial expression atom has corresponding dot matrix data preset. For example, if the first vector is (0.19, 0.31, -0.03) and the preset threshold is 0.8, then all facial expression atoms whose second vector is less than 0.8 from the first vector are selected as target facial expression atoms.

[0079] In some embodiments, the K (K≥1) discrete facial expression atoms closest to the first vector can be selected as the target facial expression atoms.

[0080] In some embodiments, all facial expression atoms whose distance is less than a preset threshold can be identified first. If no facial expression atom falls within the threshold range, the closest facial expression atom is selected as the target facial expression atom.

[0081] Step 404: Fuse the dot matrix data corresponding to each of the target facial expression atoms to obtain the target dot matrix data.

[0082] In this embodiment of the application, the dot matrix data of multiple target facial expression atoms can be combined into a new dot matrix data, namely target dot matrix data, to generate an facial expression image corresponding to the first vector.

[0083] In some embodiments, when there is only one target expression atom, the dot matrix data corresponding to the target expression atom can be directly used as the target dot matrix data.

[0084] In some embodiments, when there are multiple target expression atoms, the fusion weight corresponding to each target expression atom can be determined according to the distance between the first vector and the second vector of each target expression atom. The closer the distance, the greater the weight. Then, pixel-level weighted mixing is performed on the dot matrix data corresponding to each target expression atom, that is, the values ​​of each dot matrix data are weighted and summed at the corresponding pixel positions to obtain the target dot matrix data.

[0085] In some embodiments, the dot matrix data corresponding to each target facial expression atom can be fused using an equal-weighting method.

[0086] Step 405: Send the target dot matrix data to the driving circuit.

[0087] In some embodiments, the target dot matrix data can be sent to the driving circuit 104 via SPI or I²C bus. After receiving the target dot matrix data, the driving circuit 104 outputs driving signals (such as row / column scanning timing and PWM dimming signals) to the array lamp board 101 according to the target dot matrix data, driving each light-emitting unit on the array lamp board 101 to emit light. The light passes through the diffuser 102 and displays an expression image composed of light on the front panel 103.

[0088] In this embodiment, by mapping the target facial expression information to a first vector in a continuous semantic space, since the coordinates in the continuous semantic space can take continuous values, any corresponding facial expression can be generated by changing the value of the first vector, enabling the robot to express complex or subtle emotions, breaking through the limitation of traditional solutions that can only express a limited number of preset facial expressions; then, the target facial expression atom is retrieved based on the distance between the first vector and multiple discrete facial expression atoms in the second vector in the continuous semantic space, and then the dot matrix data corresponding to the target facial expression atom is fused to obtain the target dot matrix data, so that the facial expression generation process is independent of the specific dot matrix size and resolution. When adapting to different models, only the underlying hardware parameters need to be adjusted, without redesigning the facial expression material, realizing the cross-platform reuse of facial expression assets and effectively reducing development and maintenance costs. Furthermore, in response to the deficiency of insufficient display contrast caused by light crosstalk between pixels in dot matrix displays, the embodiments of this application use an array of light panels in conjunction with a diffuser sheet having a light-transmitting window and a light-shielding layer. This allows the light from each light-emitting unit to be emitted from its corresponding light-transmitting window and the light-shielding layer to suppress light crosstalk between adjacent light-transmitting windows, significantly improving the contrast and pixel clarity of the dot matrix display. This enables robots with low-cost dot matrix structures to present clear and detailed facial expressions.

[0089] In some embodiments, such as Figure 5As shown, step 404 may further include steps 4041 to 4043: Step 4041: Determine the distance between the first vector and the second vector corresponding to each of the target facial expression atoms.

[0090] In this embodiment, the distance between the first vector and each of the second vectors is calculated. This distance can be any one of Euclidean distance, Manhattan distance, or cosine distance. For example, if the first vector is (0.19, 0.31, -0.03), and the target expression atom includes three atoms A, B, and C, with their second vectors being (0.8, 0.3, 0.4), (0.2, -0.8, 0.1), and (-0.7, 0.8, 0.6), then the Euclidean distances between the three atoms A, B, and C and the first vector are calculated respectively.

[0091] Step 4042: Based on the distance, determine the first weight of the dot matrix data corresponding to each of the target facial expression atoms; the first weight is negatively correlated with the distance.

[0092] It should be noted that the first weight represents the proportion of the bitmap data corresponding to each target expression atom in the fusion process. This first weight is determined based on the distance between the first vector and the second vector of each target expression atom. The closer the distance, the larger the first weight, indicating that the bitmap data corresponding to that target expression atom contributes more to the final fusion result.

[0093] In some embodiments, an inverse distance weighting method is used to determine the first weight of the dot matrix data corresponding to each target expression atom. Specifically, the first weight of each dot matrix data is proportional to the inverse of the distance, that is, the smaller the distance, the greater the weight, and the larger the distance, the smaller the weight. For example, when the distance corresponding to target expression atom A is 0.2, the distance corresponding to target expression atom B is 0.5, and the distance corresponding to target expression atom C is 1.0, the dot matrix data corresponding to target expression atom A obtains the maximum weight, and the dot matrix data corresponding to target expression atom C obtains the minimum weight.

[0094] In some embodiments, a nearest neighbor weighted method can be used, that is, the K nearest discrete facial expression atoms are selected as target facial expression atoms, and then a first weight is assigned to the dot matrix data corresponding to the K target facial expression atoms based on the distance.

[0095] Step 4043: Based on the first weight, fuse the dot matrix data corresponding to each of the target facial expression atoms to obtain the target dot matrix data.

[0096] In this embodiment of the application, the pixel-level weighted mixing of the dot matrix data corresponding to each target expression atom can be performed based on the first weight of the dot matrix data corresponding to each target expression atom determined in step 4042. That is, at each corresponding pixel position of the dot matrix data, the pixel values ​​of each dot matrix data are weighted and summed to obtain the target pixel value at that position, thereby generating the target dot matrix data.

[0097] In some embodiments, when there is only one target expression atom, the first weight of the dot matrix data corresponding to the target expression atom is 1, and the dot matrix data corresponding to the target expression atom is directly used as the target dot matrix data.

[0098] It is understood that, in this embodiment of the application, the distance between the first vector and the second vector corresponding to each target expression atom is determined, and then the first weight of the dot matrix data corresponding to each target expression atom is determined based on this distance (the closer the distance, the larger the first weight), so that discrete expression atoms that are semantically closer to the target expression information occupy a higher proportion in the fusion. Therefore, even if the target expression information does not correspond to any preset discrete expression atom in the continuous semantic space, a new expression matching the target expression information can be synthesized by fusing discrete expression atoms that are semantically closer, thus achieving the expression of an infinite number of subtle emotions with a finite number of discrete expression atoms, improving the continuity and accuracy of expression.

[0099] In some embodiments, such as Figure 6 As shown, step 405 may further include steps 4051 to 4054: Step 4051: Obtain the target dot matrix data from the previous step.

[0100] It should be noted that the target dot matrix data in the previous instance refers to the target dot matrix data generated and sent when the expression display method of this application embodiment was executed in the previous instance. This dot matrix data corresponds to the expression screen currently displayed by the robot.

[0101] In some embodiments, the processor 20 may read target dot matrix data generated and stored during the previous execution of the expression generation method from memory. For example, after each generation of target dot matrix data, the processor 20 temporarily stores it in memory, and when transition dot matrix data needs to be generated, it reads the data from memory as the target dot matrix data from the previous execution.

[0102] In some embodiments, if the robot has not yet displayed any expression (e.g., upon initial startup), the processor obtains preset default dot matrix data as the target dot matrix data for the previous operation.

[0103] Step 4052: Generate transitional matrix data based on the previous target matrix data and the current target matrix data.

[0104] It should be noted that transition bitmap data refers to the bitmap data corresponding to the intermediate state between the previous target bitmap data and the current target bitmap data, which is used to achieve a smooth transition from the current expression screen to the target expression screen.

[0105] In some embodiments, the processor 20 generates at least one transitional dot matrix data by interpolation based on the previous target dot matrix data and the current target dot matrix data.

[0106] In some embodiments, the processor 20 can generate a corresponding number of transition bitmap data according to a preset number of transition frames (such as 5 frames or 10 frames). The more transition frames, the smoother the facial expression transition.

[0107] Step 4053: Combine the transition dot matrix data and the target dot matrix data of this time in sequence to form a dot matrix data sequence.

[0108] It should be noted that a dot matrix data sequence refers to a data set formed by arranging multiple frames of dot matrix data in chronological order.

[0109] In some embodiments, the processor 20 places the transition dot matrix data generated in step 4052 before the target dot matrix data of the current time, and combines them into a dot matrix data sequence in chronological order, that is, the transition dot matrix data comes first and the target dot matrix data of the current time comes later.

[0110] Step 4054: Send the dot matrix data sequence to the driving circuit.

[0111] In some embodiments, the processor 20 sends the dot matrix data sequence frame by frame to the driving circuit 104 via an SPI or I²C bus. After receiving the data, the driving circuit 104 drives each light-emitting unit on the array light board 101 to emit light according to each frame of dot matrix data, and sequentially displays the transition expression screen and the target expression screen, forming an animation effect that smoothly transitions from the current expression screen to the target expression screen.

[0112] It is understood that, in this embodiment of the application, the target dot matrix data from the previous time is obtained, and transition dot matrix data is generated based on the target dot matrix data from the previous time and the target dot matrix data from the current time. Then, the transition dot matrix data and the target dot matrix data from the current time are combined in sequence to form a dot matrix data sequence and sent to the driving circuit. This enables the driving circuit to drive the array light board to display the transition expression screen and the target expression screen in sequence, thereby achieving a smooth transition of the expression screen, avoiding abrupt changes when switching expressions, and improving the smoothness and naturalness of the expression display.

[0113] In some embodiments, generating transitional dot matrix data based on the previous target dot matrix data and the current target dot matrix data includes: Obtain the third vector corresponding to the previous target dot matrix data in the continuous semantic space; Interpolate the third vector and the first vector on the time axis to generate at least one intermediate vector; Generate transition matrix data corresponding to each of the intermediate vectors.

[0114] It should be noted that the third vector refers to the vector corresponding to the previous target dot matrix data in the continuous semantic space, that is, the vector corresponding to the currently displayed expression. Temporal interpolation refers to generating several intermediate vectors between two vectors in chronological order. The interpolation method includes, but is not limited to, linear interpolation, and the number of interpolations can be determined based on a preset frame rate or transition time. The intermediate vector is the vector located between the third vector and the first vector, used to generate the dot matrix data corresponding to the transition expression. This application does not limit the specific method of interpolating the third vector and the first vector, as long as one or more intermediate vectors can be generated between the two vectors.

[0115] In some embodiments, when executing the current expression generation method, the processor 20 reads the target dot matrix data from the memory and determines its corresponding third vector in the continuous semantic space based on the mapping relationship used when generating the data. For example, if the target dot matrix data generated previously corresponds to the "surprised" atom in the expression atom library, and the second vector of this atom is (-0.2, 0.7, -0.3), then the third vector is (-0.2, 0.7, -0.3).

[0116] In some embodiments, if the robot has not yet displayed any expressions (e.g., upon initial startup), the processor 20 obtains a preset default vector as a third vector.

[0117] In some embodiments, the processor 20 performs linear interpolation between the third vector and the first vector to generate several intermediate vectors. For example, the third vector is the vector corresponding to the current expression (-0.2, 0.7, -0.3), and the first vector is the vector corresponding to the target expression (0.8, 0.3, 0.4). The processor 20 performs equally spaced linear interpolation between the two to generate 5 intermediate vectors. The first intermediate vector is closer to the third vector, and the last intermediate vector is closer to the first vector, thus forming a gradual path from the current expression to the target expression.

[0118] In some embodiments, the processor 20 determines the number of intermediate vectors based on a preset frame rate (e.g., 30 frames per second) and a transition time (e.g., 0.5 seconds). For example, if the transition time is 0.5 seconds and the frame rate is 30 frames per second, then 15 intermediate vectors are generated.

[0119] In some embodiments, for each intermediate vector, the processor 20 determines the target expression atom from multiple discrete expression atoms based on the intermediate vector, following the steps 403 and 404 described above, and fuses the dot matrix data corresponding to each target expression atom to generate transition dot matrix data corresponding to the intermediate vector. Thus, each intermediate vector corresponds to one frame of transition expression image, and all transition dot matrix data are arranged in chronological order to form an intermediate image sequence that smoothly transitions from the current expression to the target expression.

[0120] It is understood that this embodiment of the application obtains the third vector corresponding to the previous target dot matrix data in the continuous semantic space, and then interpolates the third vector and the first vector on the time axis to generate at least one intermediate vector, so that the change path of the intermediate vector in the semantic space is continuous; then, by generating transition dot matrix data corresponding to each intermediate vector, a smooth transition of the expression image at the semantic level is achieved. As a result, a natural frame-by-frame transition can be achieved from the current expression to the target expression, avoiding the abruptness caused by direct switching and improving the smoothness and coherence of the expression display.

[0121] In some embodiments, the target facial expression information includes at least one of the following types of information: User emotion information, user intent information, and the robot's operating status information.

[0122] It should be noted that user emotion information refers to the user's emotional state labels obtained by analyzing user input (i.e., based on the interaction information between the user and the robot, such as voice and text) through semantic understanding, such as "happy," "sad," "angry," and "surprised." User intent information refers to the user's interaction purpose labels obtained by analyzing user input through semantic understanding, such as "greeting," "issuing instructions," "asking questions," and "saying goodbye." User intent information and user emotion information can coexist or exist independently, and this application embodiment does not limit this. Operational status information refers to various status data generated by the robot during operation, such as battery information, task execution progress, and network connection status, and this application embodiment does not limit the specific type of operational status information.

[0123] In some embodiments, the processor 20 acquires the user's voice input through the voice acquisition module on the robot, recognizes the voice to obtain text information, and then performs sentiment analysis on the text through a semantic understanding model, outputting the corresponding sentiment label as the user's emotional information. For example, when the user says "I won the lottery," the processor 20 outputs the "happy" sentiment label; when the user says "I'm so sad today," the processor 20 outputs the "sad" sentiment label.

[0124] In some embodiments, the semantic understanding model can also output multiple emotion labels and their corresponding probability values, such as "happy 0.6, sad 0.3, surprised 0.1".

[0125] In some embodiments, the processor 20 performs intent recognition on the text input by the user using a semantic understanding model and outputs the corresponding intent label as user intent information. For example, when the user says "hello", the processor 20 outputs the "greet" intent label; when the user says "come here", the processor 20 outputs the "instruction" intent label; and when the user says "goodbye", the processor 20 outputs the "farewell" intent label.

[0126] In some embodiments, the processor 20 obtains operational status information from the robot's status monitoring module. For example, when the robot's battery level is below a preset threshold (e.g., 5%), the status monitoring module generates "low battery" status information; when the robot is performing a task, the status monitoring module generates "task in progress" status information; when the robot's network connection is lost, the status monitoring module generates "network disconnected" status information. After obtaining the above status information, the processor 20 uses it as part of the target facial expression information to drive facial expression generation.

[0127] In some embodiments, the processor 20 simultaneously acquires user emotion / intention information and operational status information, and then fuses and processes the multiple types of information to generate an expression that comprehensively reflects the user's interaction state and the robot's own state. For example, when the user expresses "happiness" and the robot's battery is low, the processor 20 can incorporate the low battery status feature into the happy expression to generate a composite expression with a battery level indicator.

[0128] In some embodiments, the target facial expression information includes multiple types of information, and mapping the target facial expression information into a first vector in a continuous semantic space includes: Each type of information in the target facial expression is mapped to a fourth vector in the continuous semantic space; The first vector is determined based on the fourth vector corresponding to each type of information in the target facial expression information.

[0129] It should be noted that "multi-class information" refers to the target facial expression information including at least two different types of information, such as simultaneously including user emotion information and operational status information. The fourth vector refers to the vector obtained by mapping each type of information in the target facial expression information into a continuous semantic space.

[0130] In some embodiments, the target facial expression information includes two types of information: user emotion information and operating status information. The processor 20 maps the user emotion information to a fourth vector in a continuous semantic space, and simultaneously maps the operating status information to another fourth vector in the continuous semantic space. For example, if the user emotion information is "happy," the mapped fourth vector V1 is (0.8, 0.3, 0.4); if the operating status information is "low battery," the mapped fourth vector V2 is (-0.5, 0.2, -0.6). Thus, the two types of information correspond to two different fourth vectors.

[0131] In some embodiments, the target facial expression information may include more types of information, such as user emotion information, user intent information, and running status information, which are then mapped to three fourth vectors respectively.

[0132] In this embodiment, a first vector is determined based on multiple fourth vectors. For example, different weights are assigned to different fourth vectors, and the multiple fourth vectors are weighted according to these weights to obtain the first vector.

[0133] It is understood that this application's embodiments map various types of information in the target facial expression to a fourth vector in a continuous semantic space, and then determine a first vector based on the fourth vector corresponding to each type of information. This allows semantic information from different information sources (such as user emotions, user intentions, robot operating states, etc.) to be fused in a unified continuous semantic space, thereby driving facial expression generation. As a result, the robot can comprehensively reflect the user's interaction state and its own operating state, simultaneously expressing multi-dimensional information such as emotions and states in the same facial expression image. This avoids the one-sidedness of expression caused by a single information source driving facial expressions, and improves the richness and information carrying capacity of facial expression.

[0134] In some embodiments, determining the first vector based on the fourth vector corresponding to each type of information in the target facial expression information includes: Determine the display priority of each type of information in the target facial expression information; Based on the display priority, determine the second weight corresponding to each of the fourth vectors; The first vector is determined based on the fourth vector and the second weights corresponding to the fourth vector.

[0135] It should be noted that display priority refers to the degree of priority of various types of information in driving expression generation. The higher the display priority, the greater the impact of that type of information on expression generation, and the greater its corresponding weight during fusion. The second weight refers to the weight coefficient of each fourth vector in determining the first vector. The second weight is positively correlated with the display priority of various types of information; that is, the higher the priority, the greater the second weight.

[0136] In some embodiments, the display priority of various types of information can be preset according to the urgency or importance of the information. For example, the priority of operation status information is higher than that of user emotion information / user intent information. Among the operation status information, security alarm information has the highest display priority, followed by abnormal fault information, and then normal operation status information. The above priority settings are only examples, and the embodiments of this application do not limit them. They can be adjusted according to specific application scenarios.

[0137] In some embodiments, the processor 20 assigns a second weight to each fourth vector based on the sorting result of display priority. For example, the target facial expression information includes two types of information: user emotion information and running status information, corresponding to fourth vectors V1 and V2, respectively. Since the display priority of running status information is higher than that of user emotion information, the second weight corresponding to V1 is 0.7, the second weight corresponding to V2 is 0.3, and the sum of the two weights is 1.

[0138] In some embodiments, the processor 20 may also determine the second weight in other ways, such as assigning a weight of 1 to the information with the highest priority and a weight of 0 to the other information, that is, using only the fourth vector corresponding to the information with the highest priority as the first vector. This application does not limit the specific method for determining the second weight.

[0139] In some embodiments, the processor 20 performs a weighted average of the fourth vectors to obtain a first vector. For example, if the fourth vector V1 is (0.8, 0.3, 0.4) and the corresponding second weight is 0.7; and the fourth vector V2 is (-0.5, 0.2, -0.6) and the corresponding second weight is 0.3, then the first vector = 0.7 × V1 + 0.3 × V2 = (0.41, 0.27, 0.10).

[0140] It is understood that, by determining the display priority of each type of information and assigning a second weight to each fourth vector based on the display priority (the higher the priority, the larger the second weight), high-priority information dominates the fusion process, while the contribution of low-priority information decreases accordingly. Therefore, when multiple pieces of information coexist, the robot can prioritize responding to information of higher importance, ensuring that key states are expressed in a timely manner, while not completely ignoring other information, thus improving the rationality of facial expressions and scene adaptability.

[0141] In some embodiments, the target facial expression information includes multiple types of information, and mapping the target facial expression information into a first vector in a continuous semantic space includes: Determine the target information from the various types of information in the target facial expression information; The target information is mapped to a first vector in the continuous semantic space.

[0142] In this embodiment, when the target facial expression information includes multiple types of information, the processor 20 may not use a weighted fusion method, but instead selects one type of information as the target information from the multiple types of information through a filtering method, and then maps the target information to a first vector. Filtering refers to the process of selecting a certain type of information as the target information from multiple types of information according to certain rules. These rules can be based on display priority (such as selecting the information with the highest priority), or they can be based on other rules, such as preset rules or model judgment. The target information refers to the information filtered from the multiple types of information used to map the first vector; the target information is a single piece of information, not the result of fusing multiple types of information.

[0143] In some embodiments, the processor 20 selects the highest priority information from multiple types of information as the target information based on the display priority of each type of information. For example, if the target facial expression information includes both user intent information and running status information, and the display priority of the running status information is higher than that of the user intent information, then the processor 20 selects the running status information as the target information.

[0144] In some embodiments, the processor 20 may determine the target information according to preset scenario rules, such as prioritizing security alarm information in the running status information in an emergency scenario, and prioritizing user emotion / intent information in a dialogue scenario.

[0145] This application does not limit the specific rules for screening, as long as one type of information can be determined from multiple types of information as the target information.

[0146] In some embodiments, the processor 20 maps the determined target information into a first vector in a continuous semantic space in the same manner as in step 402 described above. For example, if the target information is "security alarm", it is mapped to the first vector corresponding to "security alarm"; if the target information is "happy", it is mapped to the first vector corresponding to "happy". Since the target information is a single piece of information, the first vector obtained after mapping is also a single vector. Compared with the vector obtained by weighted fusion, this method emphasizes the priority expression of high-priority or high-importance information.

[0147] It is understood that in the embodiments of this application, when multiple types of information coexist, the robot can first select the most important type of information to drive the expression generation, thereby ensuring that the information that needs the most attention can be expressed first.

[0148] It is understood that, in terms of hardware, this application embodiment uses an array of light panels in conjunction with a diffuser sheet having a light-transmitting window and a light-shielding layer to suppress cross-lighting from adjacent light-transmitting windows, thereby improving the contrast and clarity of the dot matrix display. In terms of software, a PAD continuous semantic space is established as an intermediate layer, binding multiple preset discrete expression atoms with their corresponding dot matrix data and second vectors in the continuous semantic space. When the first vector obtained by mapping the target expression information is within the influence range of multiple discrete expression atoms, a weighted fusion is performed based on the distance between the first vector and each second vector, and a transition frame is generated through time axis interpolation. The output frame buffer data drives the dot matrix display.

[0149] The following examples illustrate possible implementation schemes for one or more of the above embodiments.

[0150] To address the shortcomings of existing robot facial expression displays, such as a strong sense of mechanical appearance, fragmented design, discrete dot matrix expression assets, abrupt expression switching, and difficulty in engineering reuse, this solution provides an invisible integrated display device for robot facial dot matrix (i.e., the expression display device of the aforementioned embodiment) and its expression frame generation and output method (i.e., the expression display method of the aforementioned embodiment). The display device includes an array light panel, a diffuser, and a front panel. The diffuser is provided with light-transmitting windows that correspond one-to-one with the light-emitting units on the array light panel, and a light-shielding layer is provided between the light-transmitting windows to suppress light crosstalk; a light-shielding layer is provided on the back of the front panel, and a gap is left at the position directly opposite the light-transmitting windows to form a light-transmitting area, so that the display device is consistent with the robot's appearance when the lights are off, and the dot matrix pattern is clear and the color is uniform when the lights are on, and can adapt to the engineering requirements of high-brightness continuous display. The method uses the PAD three-dimensional continuous semantic space as an intermediate representation of facial expression semantics. It generates transition frames through facial expression atomic library, neighborhood retrieval and weight synthesis, and time smoothing, and outputs them to the driving circuit in a frame buffer manner to realize continuous transition of dot matrix facial expressions and cross-model reuse. The semantic understanding and intent / emotion parsing can be implemented by rule engine, classification model, retrieval enhancement generation model or large language model.

[0151] In this solution, the robot's facial dot matrix invisible integrated display device adopts a three-layer stacked structure of aluminum-based RGB array light board, pixelated diffuser and transparent front panel: 1) The array light board is an M×N independently controllable RGB dot matrix (i.e., light-emitting unit). The aluminum-based PCB improves heat dissipation and supports long-term high-brightness display. The array light board is connected to the processor in the robot through the driver circuit, and can control color, brightness and animation frames by pixel. Different models can be adapted by setting the dot matrix size and resolution parametrically. 2) The diffuser has square light-transmitting windows corresponding to the RGB dot matrix. The non-light-transmitting areas (the areas between the light-transmitting windows) are sprayed with black to suppress light crosstalk and form a clear mosaic pixel effect. 3) The front panel is made of transparent PMMA material and painted on the back. Only a gap is left in the position of the square light-transmitting window to form a light-transmitting area, so as to achieve a face display effect that is invisible when the screen is off and visible when the screen is on, which is integrated with the appearance of the robot.

[0152] Through the above three-layer superimposed structure, the robot's facial dot matrix invisible integrated display device is simple and reliable in structure, has good heat dissipation, uniform display, and the number of light-emitting units (pixels) and the size of the front panel can be adjusted according to different robot platforms or models.

[0153] In addition, the expression frame generation and output method based on continuous semantic space provided by this solution enables the robot's operating state and the user's emotions / intent expressions to be modeled and mapped into dot matrix data in a unified emotional semantic space, forming a scalable, interpretable, and cross-model reusable dot matrix expression asset and output link, avoiding the rigidity and difficulty in maintenance caused by conventional discrete rules and scattered expression materials.

[0154] This solution uses the PAD emotion model as a unified emotional semantic space, describing emotional states through three-dimensional vectors: pleasure (P), arousal (A), and control (D). Several discrete facial expression atoms (such as smile, frown, surprise, fear, etc.) are pre-designed, and each atom is bound to the following information: 1) Pixel patterns and animation data on the array light panel, where animation data refers to a dynamic facial expression sequence formed by multiple frames of dot matrix pixel patterns (i.e., dot matrix data); 2) Corresponding PAD coordinates and emotion / state / intention labels; 3) Display parameters such as color and rhythm.

[0155] This solution divides the PAD semantic space into several emotion regions using heuristic design rules, supporting interpolation and combination between discrete expression atoms to achieve a computable mapping from emotion to pixel expression. It also ensures the reuse of expression assets across array light panels of different resolutions and different robot platforms. Based on the PAD emotion model, this solution constructs an expression atom library and implements bidirectional mapping from expression to emotion and from emotion to expression.

[0156] For the discrete expression atom definition in the expression atom library, several discrete expression atoms E1 to En are defined in memory, each expression atom including: a) Pattern data: The on / off state and color information of the corresponding array light panel dot matrix (pixels), which can be multi-frame animation; b) PAD coordinates (vectors): For example, "smiley face" E1 corresponds to (+0.8, +0.3, +0.4), "closed eyes" E2 corresponds to (+0.2, -0.8, +0.1), "fearful" E3 corresponds to (-0.8, +0.9, -0.7), etc. c) Type tags: Emotion (joy, anger, sorrow, fear, etc.), Status (start-up, standby, battery information), Intent (turning warning, greeting, etc.).

[0157] For example, Figure 7 This is a schematic diagram illustrating the parameter configuration of an expression atom according to an embodiment of this application; as shown... Figure 7 As shown, each facial expression atom (such as E1, E2, and E3) has pre-defined brightness and color parameters. For example, E1 includes three display states: 100% brightness - white, 30% brightness - white, and pink; E2 includes one display state: 30% brightness - white; and E3 includes two display states: 100% brightness - white and 30% brightness - white. Furthermore, facial expression atom E3 is also configured with frame timing parameters. For example, after the expression pauses for 3000ms, it transitions from one display state to the next after 60ms, remains in that state for 30ms, and then resumes after another 60ms. Thus, each facial expression atom not only has a fixed vector position in the continuous semantic space but also carries multi-dimensional performance parameters such as brightness, color, and animation rhythm, providing a data foundation for the rich expression capabilities of facial expression atoms.

[0158] In this scheme, the PAD space and heuristic rules are as follows: The P-axis represents pleasure, the A-axis represents arousal, and the D-axis represents control, both ranging from -1 to +1. Design a set of heuristic rules to divide the PAD space into several emotional zones, for example: When P > 0.6 and A ∈ [-0.2, 0.6], it is preferentially mapped to "smiley face" E1; When P < -0.6 and A < -0.2, the mapping is "crying face" E8; When P < -0.6, A > 0.4, and D > 0, the mapping is "staring" E6; At the same time, the animation rhythm is controlled according to the A value; for example, the larger the A value, the higher the frequency of blinking or breathing light; and the brightness and other visual features are adjusted according to the D value.

[0159] Furthermore, in this scheme, when the target PAD coordinates are within the influence range of multiple facial expression atoms, the facial expression synthesis module interpolates or splices the patterns of multiple facial expression atoms according to weights to achieve facial expression blending. To avoid abrupt changes, the PAD vector can be smoothly interpolated according to the timeline to generate transition frames, achieving animation effects such as a gradual transition from "surprise" to "satisfaction".

[0160] It is understandable that, through the above design in this scheme, facial expressions can be described parametrically, and any emotion point can generate an approximate visual representation based on discrete facial expression atoms, thereby supporting the upper-level semantic parsing module (optionally implemented by a large language model) to drive facial expressions numerically.

[0161] For example, Figure 8 This is a schematic diagram of the architecture of a robot facial expression interaction system provided in an embodiment of this application; as shown... Figure 8 As shown, the system includes: an emotion input acquisition module 801, a semantic understanding and intent / emotion parsing module 802, a PAD emotion calculation and mapping module 803, an expression generation and output module 804, a speech synthesis module 805, a motion control module 806, an expression display device 807, and a context awareness and feedback adjustment module 808. The implementation process of the system includes: 1) The emotion input acquisition module 801 collects user voice / text, robot status, and environmental information; 2) The semantic understanding and intent / emotion parsing module 802 understands and parses the information collected by the emotion input collection module 801, and outputs dialogue feedback content and / or emotion tags and intent tags; 3) The PAD emotion calculation and mapping module 803 maps emotion / intention tags to PAD emotion vectors, and determines the target PAD vector by combining the scene and historical state through heuristic rules or parameterization strategies. 4) The expression generation and output module 804 selects or synthesizes corresponding expression atoms and animation parameters in the PAD space to generate array light panel driving data, and simultaneously drives the matching voice tone and actions. Specifically, the expression generation and output module 804 sends the generated array light panel driving data to the expression display device 807 to drive the expression display device 807 to display the corresponding expression image; simultaneously, the expression generation and output module 804 sends the emotion parameters corresponding to the current expression to the speech synthesis module 805, so that the speech synthesis module 805 synthesizes a speech signal with corresponding tone, speed, and timbre based on the emotion parameters; furthermore, the expression generation and output module 804 sends the interaction intent corresponding to the current expression to the action control module 806, so that the action control module 806 drives the robot's body to perform corresponding actions based on the interaction intent, realizing multimodal collaborative output of visual expressions, voice, and actions.

[0162] 5) The context awareness and feedback adjustment module 808 adjusts the PAD vector and expression output in real time based on user feedback and environmental changes, and ensures that alarm / abnormal expressions are displayed before ordinary social expressions through priority and throttling mechanisms.

[0163] It is understandable that this system, through the collaborative work of its various modules, enables the robot's facial expressions to be used both to display its operating status and to convey the user's emotions and interactive experience.

[0164] It should be noted that the emotion input acquisition module 801 can acquire multi-source information required to drive emotion calculation, including: user speech and its recognized text, text commands from applications or remote terminals, robot posture, battery level, task progress, network and fault information, as well as environmental and user state information such as ambient light, obstacles, dangerous areas, user distance, and gaze direction. This module performs time alignment and structured processing on the above data to form a unified input of user-related information and system state information.

[0165] The semantic understanding and intent / emotion parsing module 802 can perform semantic understanding and intent / emotion parsing on the structured information acquired by the emotion input acquisition module 801, and fuse information such as user voice / text input, dialogue history, robot current state, and environment / task context to output interaction intent and emotion estimation results. This module can be implemented by any one or a combination of rule engine, classification model, retrieval enhancement generation model or large language model.

[0166] In some embodiments, the semantic understanding and intent / emotion parsing module 802 can further generate or select the feedback text that the robot needs to output, and simultaneously output the emotion / intent label or probability distribution corresponding to the feedback text. After obtaining the emotion / intent label or probability distribution, the PAD emotion calculation and mapping module 803 performs emotion quantization processing on it to obtain the target PAD vector or its candidate set, thereby providing input basis for subsequent expression atom retrieval, neighborhood weight synthesis and transition frame generation.

[0167] The PAD emotion calculation and mapping module 803 integrates information from two sources within a unified emotion space: one is the interactive emotion and interactive intent obtained by the semantic understanding and intent / emotion parsing module 802, and the other is system state events from state monitoring (such as low battery, charging, task execution, obstacle avoidance trigger, safety alarm, fault, etc.). The PAD emotion calculation and mapping module 803 smooths and attenuates the initial PAD vector, sets the PAD baseline and weights based on dialogue history and scene type, and maps various state events to specific regions in the PAD space, marking them as system state-type expressions. Subsequently, it calls the expression atom library and heuristic rules to select or synthesize corresponding expression atoms and animation parameters for the current target PAD vector, achieving unified modeling of "emotional interaction-type expressions" and "system state-type expressions."

[0168] The expression generation and output module 804 generates frame buffer data (i.e., dot matrix data) based on the target expression atoms and animation parameters. Each frame includes the color and brightness information of each pixel in the array light panel and is sent to the driving circuit at a predetermined refresh rate. The expression display device 807 then presents the running status and emotional expression in the form of dot matrix animation. Simultaneously, the expression generation and output module 804 transmits the feedback text and current emotion parameters output by the semantic understanding and intent / emotion parsing module 802 to the speech synthesis module 805. The speech synthesis module 805 then generates speech output with corresponding tone, speed, and timbre. The motion control module 806 receives the interaction intent and maps it to the robot's motion commands, such as nodding, shaking, turning, approaching, or moving backward, ensuring that visual expressions, speech, and actions are consistent in time and emotion.

[0169] The context perception and feedback adjustment module 808 comprehensively perceives the effects of facial expressions, voice and action output, as well as the environment and subsequent user behavior. When it detects user confusion, lack of response, or new state events (such as the occurrence or resolution of a fault, the appearance or disappearance of a security risk), it adjusts the current PAD vector and expression type and feeds the correction results back to the semantic understanding and intent / emotion analysis module 802 and the PAD emotion calculation and mapping module 803, thereby forming a closed-loop adjustment.

[0170] This solution also includes setting emoji priorities and a throttling mechanism. Generally, emojis for abnormal faults and security alarms have the highest priority, followed by emojis for critical system statuses, then social and emotional interaction emojis, and finally the default standby emoji. When a high-priority status is triggered, it can preempt the current social / emotional emoji to ensure the status information is prominent; when there are no high-priority events, more social / emotional emojis are displayed to improve the user experience. A throttling strategy is used for frequently triggered low-priority emojis to limit repeated triggering within a short period, avoiding frequent switching that could affect status readability.

[0171] This solution implements a robot facial expression interaction system that uses semantic understanding and intent / emotion parsing as the entry point, a PAD continuous semantic space as the intermediate layer, and an expression atom library and frame buffer output as the final link. Combined with structured facial dot matrix display hardware (array light board, diffuser, front panel) and a parameterizable emotion semantic layer, it achieves highly expressive and scalable robot facial dot matrix expression display and emotional interaction capabilities while maintaining easy integration. This system enables the robot to simultaneously complete operational status prompts and emotional interaction expressions on the same facial display platform. Through feedback and priority control, it obtains natural, reliable, and scalable emotional interaction effects, forming a unified output for robot operational status display and emotional interaction on the same platform.

[0172] It should be noted that the robot facial expression interaction system provided in this solution can be adapted to different robot models (such as quadrupedal wheeled robots, humanoid service robots, etc.). Specifically, for array light panels with different resolutions, the same facial expression atom can be projected onto different resolutions simply by defining pixel mapping rules in the configuration file. The PAD model and facial expression atom library are independent of specific robot models; only the heuristic rule parameters need to be adjusted for different appearances and application scenarios (e.g., increasing the weight of warning expressions for safety robots and increasing the weight of friendly expressions for service robots). Through a unified interface protocol (such as Message Queuing Telemetry Transport (MQTT), Robot Operating System Topic (ROS Topic), or a custom bus protocol), the facial expression interaction system is decoupled from the speech synthesis module and motion control module of each robot model, achieving one-time design and multi-terminal reuse.

[0173] Compared with traditional solutions, the solution proposed in this application has at least the following advantages: (1) By combining the structure of “one-to-one correspondence between the light-emitting units on the array light board and the light-transmitting windows on the diffuser, setting a light-shielding layer between the light-transmitting windows and setting a light-shielding layer on the back of the front panel”, in the off state, the light-shielding layer on the back of the front panel blocks the internal structure, so that the front panel and the robot shell are the same color, and the facial appearance is consistent; in the on state, the light of each light-emitting unit shines out from the corresponding light-transmitting window, the light-shielding layer between the light-transmitting windows suppresses the cross-light between adjacent pixels, improves the pixel contrast, and at the same time the diffuser makes the light mix evenly in the light-transmitting window, thereby improving the uniformity and clarity of the dot matrix display, obtaining a clear and stable dot matrix facial presentation effect, and the structure is simple and easy to mass-produce. (2) By binding the PAD continuous semantic space with the fields of the expression atom library (pattern / animation, PAD coordinates, type label, color / rhythm / brightness parameters), a computable mapping mechanism of "semantics → expression → dot matrix data" is formed, making the expression assets reusable, scalable, and easy to migrate between different dot matrix specifications and different robot platforms; (3) By generating transition frames through neighborhood weight interpolation / splicing and time smoothing, the dot matrix expressions can be synthesized and gradually output according to the target semantic vector, thereby improving the continuity and naturalness of the expression, which is different from the conventional scheme that can only select preset patterns discretely.

[0174] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps; or steps from different embodiments may be combined into a new technical solution.

[0175] Based on the same inventive concept as the foregoing embodiments, this application provides an expression display device.

[0176] Figure 9 This is a schematic diagram of the structure of an expression display device provided in an embodiment of this application; as shown... Figure 9 As shown, the facial expression display device 900 includes: an acquisition module 910, a mapping module 920, a determination module 930, a fusion module 940, and a sending module 950; wherein: Module 910 is used to acquire target facial expression information; Mapping module 920 is used to map the target facial expression information into a first vector in a continuous semantic space; The determining module 930 is used to determine at least one target expression atom from the plurality of discrete expression atoms in the continuous semantic space based on the first vector and the second vectors of the plurality of preset discrete expression atoms in the continuous semantic space; wherein the distance between the second vector of each target expression atom in the continuous semantic space and the first vector is less than a preset threshold, and each target expression atom has a corresponding dot matrix data preset. The fusion module 940 is used to fuse the dot matrix data corresponding to each of the target facial expression atoms to obtain target dot matrix data; The transmitting module 950 is used to transmit the target dot matrix data to the driving circuit.

[0177] In some embodiments, the fusion module 940 is further configured to: determine the distance between the first vector and the second vector corresponding to each of the target expression atoms; determine a first weight of the dot matrix data corresponding to each of the target expression atoms based on the distance; the first weight is negatively correlated with the distance; and fuse the dot matrix data corresponding to each of the target expression atoms based on the first weight to obtain the target dot matrix data.

[0178] In some embodiments, the sending module 950 is further configured to: acquire the previous target dot matrix data; generate transition dot matrix data based on the previous target dot matrix data and the current target dot matrix data; sequentially combine the transition dot matrix data and the current target dot matrix data into a dot matrix data sequence; and send the dot matrix data sequence to the driving circuit.

[0179] In some embodiments, the sending module 950 is further configured to: obtain a third vector corresponding to the previous target dot matrix data in the continuous semantic space; interpolate the third vector and the first vector on the time axis to generate at least one intermediate vector; and generate transition dot matrix data corresponding to each of the intermediate vectors respectively.

[0180] In some embodiments, the target facial expression information includes at least one of the following types of information: user emotion information, user intent information, and the robot's operating status information.

[0181] In some embodiments, the target facial expression information includes multiple types of information, and the mapping module 920 is further configured to: map each type of information in the target facial expression information to a fourth vector in the continuous semantic space; and determine the first vector based on the fourth vector corresponding to each type of information in the target facial expression information.

[0182] In some embodiments, the mapping module 920 is further configured to: determine the display priority corresponding to each type of information in the target expression information; determine the second weight corresponding to each of the fourth vectors based on the display priority; and determine the first vector based on the fourth vector and the second weight corresponding to the fourth vector.

[0183] In some embodiments, the mapping module 920 is further configured to: determine target information from various types of information in the target expression information; and map the target information into a first vector in the continuous semantic space.

[0184] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0185] Based on the same inventive concept as the foregoing embodiments, this application provides an expression display device.

[0186] Figure 10 This is a schematic diagram of the structure of an expression display device provided in an embodiment of this application. Figure 10 The facial expression display device 1000 shown includes a processor 1001, which can call and run computer programs from memory to implement the methods in the embodiments of this application.

[0187] Optionally, such as Figure 10As shown, the facial expression display device 1000 may further include a memory 1002. The processor 1001 can retrieve and run computer programs from the memory 1002 to implement the methods described in this embodiment.

[0188] The memory 1002 can be a separate device independent of the processor 1001, or it can be integrated into the processor 1001.

[0189] Optionally, such as Figure 10 As shown, the facial expression display device 1000 may also include a transceiver 1003. The processor 1001 can control the transceiver 1003 to communicate with other devices. Specifically, it can send information or data to other devices or receive information or data sent by other devices.

[0190] The transceiver 1003 may include a transmitter and a receiver. The transceiver 1003 may further include an antenna, and the number of antennas may be one or more.

[0191] This application also provides a computer-readable storage medium for storing computer programs.

[0192] Optionally, the computer-readable storage medium can be applied to the facial expression display device in the embodiments of this application, and the computer program causes the processor or facial expression display device to execute the various methods of the embodiments of this application, which will not be described in detail here for the sake of brevity.

[0193] This application also provides a computer program product, including computer program instructions.

[0194] Optionally, the computer program product can be applied to the facial expression display device in the embodiments of this application, and the computer program instructions cause the processor or facial expression display device to execute the various methods of the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0195] This application also provides a computer program.

[0196] Optionally, the computer program can be applied to the facial expression display device in the embodiments of this application. When the computer program runs on the processor or the facial expression display device, it causes the processor or the facial expression display device to execute the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0197] It should be noted that the descriptions of the devices, storage media, computer program products, and computer program embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the device, storage media, computer program products, and computer program embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0198] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0199] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0200] It should be understood that the above-described memory is exemplary and not a limiting description. For example, the memory in the embodiments of this application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM), etc. That is to say, the memory in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0201] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0202] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0203] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are illustrative; for instance, the division of units is a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0204] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0205] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0206] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a robot to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0207] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A robot, the robot comprising an expression display device and a processor; the expression display device comprising: The array light panel is connected to the driver circuit. A diffuser is disposed on the light-emitting side of the array lamp panel. The diffuser has a light-transmitting window that corresponds to each light-emitting unit on the array lamp panel, and a light-shielding layer is disposed in the area between each light-transmitting window. The diffuser is used to allow the light emitted by the light-emitting unit to be emitted from the corresponding light-transmitting window, and the light-shielding layer suppresses cross-lighting of light emitted from adjacent light-transmitting windows. A front panel is disposed on the light-emitting side of the diffuser; the front panel is used to display an expression image composed of light emitted from each of the light-transmitting windows; The driving circuit is connected to the processor; The driving circuit is used to receive target dot matrix data sent by the processor and output driving signals to the array light board according to the target dot matrix data; The processor is configured to: acquire target facial expression information; and map the target facial expression information into a first vector in a continuous semantic space; Based on the first vector and the second vectors of a plurality of preset discrete facial expression atoms in the continuous semantic space, at least one target facial expression atom is determined from the plurality of discrete facial expression atoms; wherein, the distance between the second vector of each target facial expression atom in the continuous semantic space and the first vector is less than a preset threshold, and each target facial expression atom has a preset corresponding dot matrix data; the dot matrix data corresponding to each target facial expression atom is fused to obtain target dot matrix data; The target dot matrix data is sent to the driving circuit.

2. The robot according to claim 1, wherein the front panel is a transparent panel; the front panel includes a light-shielding layer, the light-shielding layer is disposed on the side of the front panel facing the diffuser, and a gap is left at the position directly opposite each of the light-transmitting windows along the light emission direction; The light-shielding layer is used to make the front panel match the color of the housing of the expression display device, and the gaps in the light-shielding layer are used to allow light emitted from the light-transmitting window to pass through in order to display the expression image.

3. An expression display method, applied to a robot equipped with an expression display device, the expression display device comprising an array light panel, a diffuser, a front panel, and a driving circuit; the array light panel is connected to the driving circuit; the diffuser is disposed on the light-emitting side of the array light panel, the diffuser having light-transmitting windows corresponding one-to-one with each light-emitting unit on the array light panel, and a light-shielding layer being disposed in the area between each of the light-transmitting windows; the diffuser is used to allow light emitted by each of the light-emitting units to exit through the corresponding light-transmitting windows, and the light-shielding layer suppresses cross-lighting from adjacent light-transmitting windows; the front panel is disposed on the light-emitting side of the diffuser and is used to display an expression image composed of light emitted from each of the light-transmitting windows; The driving circuit is used to receive target dot matrix data and output a driving signal to the array light panel according to the target dot matrix data; the method includes: Obtain the target's facial expression information; The target facial expression information is mapped to a first vector in a continuous semantic space; Based on the first vector and the second vectors of a plurality of discrete facial expression atoms in the continuous semantic space, at least one target facial expression atom is determined from the plurality of discrete facial expression atoms; wherein, the distance between the second vector of each target facial expression atom in the continuous semantic space and the first vector is less than a preset threshold, and each target facial expression atom has a corresponding dot matrix data. The dot matrix data corresponding to each of the target facial expression atoms are fused to obtain the target dot matrix data; The target dot matrix data is sent to the driving circuit.

4. The method according to claim 3, wherein fusing the dot matrix data corresponding to each of the target facial expression atoms to obtain target dot matrix data includes: Determine the distance between the first vector and the second vector corresponding to each of the target facial expression atoms; Based on the distance, the first weight of the dot matrix data corresponding to each of the target facial expression atoms is determined respectively; The first weight is negatively correlated with the distance; Based on the first weight, the dot matrix data corresponding to each of the target facial expression atoms are fused to obtain the target dot matrix data.

5. The method according to claim 3, wherein sending the target dot matrix data to the driving circuit comprises: Retrieve the target dot matrix data from the previous step; Based on the previous target dot matrix data and the current target dot matrix data, generate transition dot matrix data; The transition dot matrix data is sequentially combined with the target dot matrix data to form a dot matrix data sequence; The dot matrix data sequence is sent to the driving circuit.

6. The method according to claim 5, wherein generating transitional dot matrix data based on the previous target dot matrix data and the current target dot matrix data includes: Obtain the third vector corresponding to the previous target dot matrix data in the continuous semantic space; Interpolate the third vector and the first vector on the time axis to generate at least one intermediate vector; Generate transition matrix data corresponding to each of the intermediate vectors.

7. The method according to claim 3, wherein the target facial expression information includes at least one of the following types of information: User emotion information, user intent information, and the robot's operating status information.

8. The method according to claim 7, wherein the target facial expression information includes multiple types of information, and mapping the target facial expression information into a first vector in a continuous semantic space includes: Each type of information in the target facial expression is mapped to a fourth vector in the continuous semantic space; The first vector is determined based on the fourth vector corresponding to each type of information in the target facial expression information.

9. The method according to claim 8, wherein determining the first vector based on the fourth vector corresponding to each type of information in the target facial expression information comprises: Determine the display priority of each type of information in the target facial expression information; Based on the display priority, determine the second weight corresponding to each of the fourth vectors; The first vector is determined based on the fourth vector and the second weights corresponding to the fourth vector.

10. The method according to claim 7, wherein the target facial expression information includes multiple types of information, and mapping the target facial expression information into a first vector in a continuous semantic space includes: Determine the target information from the various types of information in the target facial expression information; The target information is mapped to a first vector in the continuous semantic space.

Images