Digital human display device and display method

Abstract

The invention relates to the technical field of display, in particular to a digital human display device, a digital human display method, an acquisition unit, an image processing unit, a deviation processing unit, an angle processing unit and an angle adjusting unit. When the 3D digital person is displayed on the display equipment in the prior art, the angle processing unit and the angle adjusting unit can adjust the angle of the digital person in real time according to different standing positions of the person, so that the digital person faces the direction of the person all the time, and the digital person presentation effect is more intelligent and more vivid.

Description

Technical Field

[0001] The present invention relates to the field of display technology, and in particular to a digital human display device and a display method. Background Art

[0002] With the development of display technology, the presentation scenarios of digital humans are becoming more and more extensive. Digital humans refer to virtual characters with digital appearances. Unlike robots with physical bodies, digital humans rely on display devices to exist. When users interact with digital humans, the display device obtains relevant information from the user and uses information science methods to virtually simulate the 3D digital human images of the human body at different levels of form and function, so that users can interact with the digital humans based on the relevant information.

[0003] When a 3D digital human is currently displayed on a display device (such as a TV), the display angle or specific action is generally set according to given parameters. However, digital human technology focuses more on the linkage of facial expressions, lip shapes and broadcast text. When the position of a person changes, there is a delay in adjusting the angle of the digital human, and the angle of the digital human cannot be changed in real time, resulting in an unrealistic presentation of the image. Summary of the invention

[0004] The purpose of the present invention is to provide a digital human display device and a display method to solve the problems in the prior art that there is a delay in adjusting the angle of the digital human in the display device, the angle of the digital human cannot be changed in real time, and the presentation effect of the presented picture is not realistic.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions: A digital human display device comprises: an acquisition unit, used for acquiring a human face image; An image processing unit, used to preset a plurality of facial image marking points according to a facial image, preset a plurality of facial image marking points according to a facial image, preset reference coordinate information according to a facial image, activate a digital human image, generate digital human image marking points corresponding to facial image marking points, and obtain coordinate information of a current facial image marking point and coordinate information of a current digital human image marking point; A deflection processing unit, used to process the coordinate information of the marker points of the current face image according to the preset reference coordinate information, and obtain the deflection coordinate information of the current face image; An angle processing unit, used to obtain the digital human image adjustment angle according to the preset reference coordinate information, the current face image deviation coordinate information and the current digital human image marker point coordinate information; The angle adjustment unit is used to adjust the angle according to the digital human image and adjust the display angle of the digital human image.

[0006] The present application also provides a digital human display method, comprising: obtaining a face image, and presetting a plurality of face image marking points according to the face image; According to the preset reference coordinate information of the face image, the digital human image is activated, and digital human image marking points corresponding to the face image marking points are generated on the digital human image; Obtain the coordinate information of the marker points of the current face image and the coordinate information of the marker points of the current digital human image; According to the preset reference coordinate information, the coordinate information of the marker points of the current face image is processed to obtain the bias coordinate information of the current face image; According to the preset reference coordinate information, the current face image deviation coordinate information and the current digital human image marker point coordinate information, the digital human image adjustment angle is obtained; Adjust the display angle of the digital human image according to the angle adjustment of the digital human image.

[0007] A further technical solution is that the step of generating digital human image marker points corresponding to the facial image marker points on the digital human image includes: Generating digital human image marker points on the digital human image according to preset facial image marker points; Compare the facial image markers with the digital human image markers, find and delete the singular points, and make the processed facial image markers correspond to the digital human image markers one by one.

[0008] A further technical solution is that the step of obtaining the coordinate information of the marker points of the current face image and the coordinate information of the marker points of the current digital human image includes: obtaining the coordinate information of the marker points of each face image and the coordinate information of the marker points of each corresponding digital human image; Performing distance comparison processing on the coordinate information of each facial image marker point and the coordinate information of each corresponding digital human image marker point, and according to the processing result, taking the facial image marker point closest to the digital human image as the current facial image marker point; According to the current facial image markers, the corresponding current digital human image markers are obtained; The coordinate information of the current face image marker points and the coordinate information of the current digital human image marker points are obtained according to the current face image marker points and the current digital human image marker points.

[0009] A further technical solution is to process the coordinate information of the marker points of the current face image according to the preset reference coordinate information to obtain the bias coordinate information of the current face image, including: Get the screen resolution; According to the screen resolution and the coordinate information of the marker points in the current face image, the bias coordinate information of the current face image is obtained.

[0010] A further technical solution is to obtain the adjustment angle of the digital human image according to the preset reference coordinate information, the current face image deviation coordinate information and the current digital human image marker coordinate information, including: Get the screen resolution; According to the current face image deviation coordinate information and the current digital human image marker point coordinate information, the current digital human image marker point deviation coordinate is obtained; According to the screen resolution and the coordinates to which the current digital human image marker points should be deviated, the digital human image adjustment angle is obtained.

[0011] A further technical solution is that the step of obtaining a face image includes: obtaining current video stream data, processing the current video stream data, and obtaining a face image.

[0012] A further technical solution is that after the step of obtaining the current video stream data, it includes: processing the current video stream data, and based on the processing result, hiding the digital human image when no face image is detected in the current video stream data.

[0013] A further technical solution is to process the current video stream data to obtain gesture information; query and obtain response information based on the gesture information, and feed back the response information through the digital human image.

[0014] Compared with the prior art, the present invention has the following beneficial effects: In the above-mentioned digital human display method, by obtaining a face image, a plurality of face image marker points are preset according to the face image; according to the preset reference coordinate information of the face image, the digital human image is activated, and a digital human image marker point corresponding to the face image marker point is generated on the digital human image; the current face image marker point coordinate information and the current digital human image marker point coordinate information are obtained; according to the preset reference coordinate information, the current face image marker point coordinate information is processed to obtain the current face image deviation coordinate information; according to the preset reference coordinate information, the current face image deviation coordinate information and the current digital human image marker point coordinate information, the digital human image adjustment angle is obtained; according to the digital human image adjustment angle, the display angle of the digital human image is adjusted to achieve real-time adjustment of the display angle of the digital human image according to the face angle, so that the TV digital human is more intelligent and the presentation effect of the picture is more realistic. This application activates the digital human through the face image, and when displaying the 3D digital human on the display device in the prior art, the angle of the digital human can be adjusted in real time according to the different standing positions of the person, so that the digital human always faces the direction of the person, and the digital human presentation effect is more intelligent and realistic. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a structural block diagram of the digital human display device of the present invention.

[0016] Figure 2 This is a flow chart of the digital human display method of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0018] In one embodiment, Figure 1 As shown, a digital human display device is provided, comprising: an acquisition unit, for acquiring a human face image; An image processing unit, used to preset a plurality of facial image marking points according to a facial image, preset a plurality of facial image marking points according to a facial image, preset reference coordinate information according to a facial image, activate a digital human image, generate digital human image marking points corresponding to facial image marking points, and obtain coordinate information of a current facial image marking point and coordinate information of a current digital human image marking point; A deflection processing unit, used to process the coordinate information of the marker points of the current face image according to the preset reference coordinate information, and obtain the deflection coordinate information of the current face image; An angle processing unit, used to obtain the digital human image adjustment angle according to the preset reference coordinate information, the current face image deviation coordinate information and the current digital human image marker point coordinate information; An angle adjustment unit, used to adjust the angle according to the digital human image and adjust the display angle of the digital human image; Based on the aforementioned digital human display device, in this embodiment, refer to Figure 2 It is shown that a digital human display method is provided, comprising the following steps: obtaining a facial image, and presetting a plurality of facial image marking points according to the facial image, wherein the facial image includes facial information, and the facial image marking points can be set as marking points for identifying specific positions in the facial information.

[0019] According to the preset reference coordinate information of the face image, the digital human image is activated, and digital human image marking points corresponding to the face image marking points are generated on the digital human image.

[0020] The digital human image refers to an image that is displayed on a display screen in the prior art by using information science methods to perform virtual simulation of the human body's morphology and functions at different levels based on the execution of the digital human program. The digital human image may be a 3D digital human image, and the digital human image markers may be digital human image markers that correspond to the markers of the face image in the 3D digital human image.

[0021] Obtain the coordinate information of the marker points of the current face image and the coordinate information of the marker points of the current digital human image; The image processing unit can perform recognition processing on the acquired face image, and then obtain face information. The image processing unit can perform analysis processing on the face information, and then obtain the corresponding current face image marker point coordinate information and current digital human image marker point coordinate information. The image processing unit can also activate the digital human image according to the acquired face image, so that the digital human image is displayed and processed through the display device in the prior art.

[0022] In addition, the image processing unit can perform recognition processing on the face image, identify the face features in the face image, and output the corresponding face information. The image processing unit can establish a coordinate system based on the face image, so that each face image marker point on the face image corresponds to a coordinate point, and then obtain the current coordinate information of the corresponding face image marker point in the face image, that is, the face image marker point coordinate information, and at the same time obtain the current coordinate information of the corresponding digital human image marker point in the digital human image, that is, the digital human image marker point coordinate information.

[0023] According to the preset reference coordinate information, the coordinate information of the marker points of the current face image is processed to obtain the bias coordinate information of the current face image; The facial image deviation coordinate information refers to the deviation coordinates of the face relative to the coordinate system established based on the facial image. The preset reference coordinate information can be the coordinate information of the corresponding facial image marker point, that is, the facial image deviation coordinate information is the deviation coordinate information of the facial image relative to the facial image marker point.

[0024] Exemplarily, the deviation processing unit may process the coordinate information of the marker points of the current face image according to preset reference coordinate information to obtain the deviation coordinate information of the current face image.

[0025] According to the preset reference coordinate information, the current face image deviation coordinate information and the current digital human image marker point coordinate information, the digital human image adjustment angle is obtained; Exemplarily, the angle processing unit may compare the current digital human image marker point coordinate information and the current facial image deviation coordinate information according to preset reference coordinate information and the current facial image deviation coordinate information to obtain the digital human image adjustment angle.

[0026] Adjust the display angle of the digital human image according to the angle adjustment of the digital human image.

[0027] The angle adjustment unit can adjust the display angle of the digital human image in real time according to the size of the face deflection angle. For example, if the face in the current face image is deflected to the left relative to the front of the screen of the display device, the direction of the digital human image is adjusted, that is, the digital human in the digital human image is turned to face the user's face, so as to update the angle of the digital human image in real time according to the different standing positions of the user, so that the digital human converted from the digital human image always faces the direction of the user's face.

[0028] In the above embodiment, by obtaining a face image, a plurality of face image markers are preset according to the face image; according to the preset reference coordinate information of the face image, the digital human image is activated, and a digital human image marker corresponding to the face image marker is generated on the digital human image; the coordinate information of the current face image marker and the coordinate information of the current digital human image marker are obtained; according to the preset reference coordinate information, the coordinate information of the current face image marker is processed to obtain the current face image deviation coordinate information; according to the preset reference coordinate information, the current face image deviation coordinate information and the current digital human image marker coordinate information, the digital human image adjustment angle is obtained; according to the digital human image adjustment angle, the display angle of the digital human image is adjusted to achieve real-time adjustment of the display angle of the digital human image according to the face angle, so that the TV digital human is more intelligent and the presentation effect of the picture is more realistic. The present application activates the digital human through the face image, and when displaying the 3D digital human on the display device in the prior art, the angle of the digital human can be adjusted in real time according to the different standing positions of the person, so that the digital human always faces the direction of the person, and the digital human presentation effect is more intelligent and realistic.

[0029] In one example, the step of generating digital human image marker points corresponding to facial image marker points on the digital human image includes: Generating digital human image marker points on the digital human image according to preset facial image marker points; Compare the facial image markers with the digital human image markers, find and delete the singular points, and make the processed facial image markers correspond to the digital human image markers one by one.

[0030] Among them, due to the diversity of facial images, it is inevitable that the facial image marker points will fail to be recognized in the process of facial information recognition obtained based on the facial image, resulting in the digital human image not generating the corresponding digital human image marker points. Therefore, the image processing unit compares the facial image marker points and the digital human image marker points, finds the singular points and deletes them, so that the facial image marker points and the generated digital human image marker points are one-to-one corresponding.

[0031] The step of obtaining the coordinate information of the marker points of the current face image and the coordinate information of the marker points of the current digital human image includes: obtaining the coordinate information of the marker points of each face image and the coordinate information of the marker points of each corresponding digital human image; The image processing unit performs face recognition processing on the acquired face image to obtain the face information of each face in the face image. The image processing unit processes the face information based on the coordinate system model to obtain the coordinate information of each face image marker point and the corresponding coordinate information of each digital human image marker point.

[0032] Performing distance comparison processing on the coordinate information of each facial image marker point and the coordinate information of each corresponding digital human image marker point, and according to the processing result, taking the facial image marker point closest to the digital human image as the current facial image marker point; According to the current facial image markers, the corresponding current digital human image markers are obtained; The coordinate information of the current face image marker points and the coordinate information of the current digital human image marker points are obtained according to the current face image marker points and the current digital human image marker points.

[0033] The step of processing the coordinate information of the marker points of the current face image according to the preset reference coordinate information to obtain the bias coordinate information of the current face image includes: Get the screen resolution; The screen resolution refers to the resolution of the screen of the display device, and the bias processing unit can obtain the screen resolution by querying the resolution currently used by the screen of the display device. For example, the screen resolution is (width, height).

[0034] According to the screen resolution and the coordinate information of the marker points in the current face image, the bias coordinate information of the current face image is obtained.

[0035] Among them, the deviation processing unit can process the acquired current face image marker point coordinate information, confirm the distance between the current face image marker point coordinate information and the corresponding preset face image marker point, and then obtain the corresponding current face image deviation coordinate information.

[0036] The step of obtaining the adjustment angle of the digital human image according to the preset reference coordinate information, the current face image deviation coordinate information and the current digital human image marker coordinate information includes: Get the screen resolution; The screen resolution refers to the resolution of the screen of the display device, and the bias processing unit can obtain the screen resolution by querying the resolution currently used by the screen of the display device. For example, the screen resolution is (width, height).

[0037] According to the current face image deviation coordinate information and the current digital human image marker point coordinate information, the current digital human image marker point deviation coordinate is obtained; Among them, the angle processing unit can compare and process the current face image deviation coordinate information and the current digital human image marker point coordinate information according to the screen resolution (width, height), and then obtain the current digital human image marker point deviation coordinates.

[0038] According to the screen resolution and the coordinates to which the current digital human image marker points should be deviated, the digital human image adjustment angle is obtained.

[0039] Among them, the angle processing unit can compare and process the coordinates of the current digital human image mark point that should be deviated and the coordinate information of the current digital human image mark point according to the screen resolution (width, height), obtain the coordinates of the current digital human image mark point that should be deviated, and then convert it into the digital human image adjustment angle according to the digital human image distance between the coordinates of the current digital human image mark point that should be deviated and the coordinate information of the current digital human image mark point.

[0040] The coordinate information of each facial image marker point is compared with the coordinate information of each digital human image marker point, and according to the processing result, the facial image marker point closest to the digital human image is used as the current facial image marker point, and the current facial image deflection coordinate information is obtained according to the screen resolution and the current facial image marker point coordinate information, and the current digital human image marker point deflection coordinates are obtained according to the current facial image deflection coordinate information and the current digital human image marker point coordinate information, and the digital human image adjustment angle is obtained according to the screen resolution and the current digital human image marker point deflection coordinates, and the digital human image display angle is adjusted according to the digital human image adjustment angle, so as to adjust the display angle of the digital human image in real time according to the facial angle, making the TV digital human more intelligent and the presentation effect of the picture more realistic. The digital human is activated by the facial image, and when the 3D digital human is displayed on the TV, the angle can be updated in real time according to the different standing positions of the human, so that the digital human always faces the direction of the human, and the digital human presentation effect is more intelligent and realistic.

[0041] In the above embodiment, the digital human image is activated by the face image, and when the 3D digital human is displayed on the TV, the deflection angle between the face and the front of the screen is calculated, and the angle is updated in real time according to the different standing positions of the person, so that the digital human always faces the direction of the person, making the digital human presentation more intelligent and realistic.

[0042] In one example, the step of acquiring a facial image includes: acquiring current video stream data, processing the current video stream data, and obtaining a facial image.

[0043] The current video stream data can be detected and obtained by the acquisition unit. The acquisition unit can monitor the video data of the current environment in real time, thereby acquiring the current video stream data, and processing the acquired current video stream data to obtain the face image.

[0044] In order to improve the intelligence and authenticity of the digital human and further enhance the 3D digital human, in one example, after the step of obtaining the current video stream data, the following steps are included: The current video stream data is processed, and according to the processing result, when no face image is detected in the current video stream data, the digital human image is hidden.

[0045] The acquisition unit can acquire the current video stream data transmitted by the camera module and process the current video stream data, for example, intercepting an image from the current video stream data based on a preset period, and performing face recognition processing on the intercepted image to determine whether the intercepted image contains face information. When the acquisition unit does not detect a face image in the current video stream data, the digital human image is hidden.

[0046] In the above embodiment, the digital human is activated by a facial image, and when the 3D digital human is displayed on the TV, the angle can be updated in real time according to the different standing positions of the person, so that the digital human always faces the direction of the person. When the facial image is not detected, the digital human image is hidden, making the digital human presentation more intelligent and realistic.

[0047] In one example, the digital human display method further includes: Get the current video stream data, process the current video stream data, and get the gesture information.

[0048] Gesture information refers to the user's gesture information. In one example, the acquisition unit is provided with a sign language recognition module, through which the user's sign language can be recognized from the current video stream data detected by the camera module, for example, the gesture information is: "What is the weather like tomorrow?".

[0049] It should be noted that the sign language recognition module can be composed of an image acquisition module, an image recognition module, a semantic conversion module and a semantic output module, which converts sign language into electrical signals and then into text.

[0050] According to the gesture information, the query obtains the response information, and the response information is fed back through the digital human image.

[0051] The acquisition unit can obtain response information corresponding to the gesture information by querying the database or the cloud server according to the gesture information, and feed back the response information to the user through the digital human image, so as to better display the digital human image.

[0052] Although the present invention is described herein with reference to a number of illustrative embodiments of the present invention, it will be appreciated that those skilled in the art may devise many other modifications and implementations that fall within the scope and spirit of the principles disclosed herein. More specifically, within the scope of the present disclosure, drawings, and claims, a variety of variations and modifications may be made to the components and / or layout of the subject combination layout. In addition to the variations and modifications made to the components and / or layout, other uses will also be apparent to those skilled in the art.

Claims

1. A digital human display device, characterized in that: include: An acquisition unit, used for acquiring a face image; An image processing unit, configured to preset a plurality of facial image marker points according to the facial image, preset a plurality of facial image marker points according to the facial image, preset reference coordinate information according to the facial image, activate the digital human image, generate digital human image marker points corresponding to the facial image marker points, and obtain coordinate information of the current facial image marker points and coordinate information of the current digital human image marker points; A deflection processing unit, used to process the coordinate information of the marker points of the current face image according to the preset reference coordinate information, and obtain the deflection coordinate information of the current face image; An angle processing unit, used to obtain the digital human image adjustment angle according to the preset reference coordinate information, the current face image deviation coordinate information and the current digital human image marker point coordinate information; The angle adjustment unit is used to adjust the display angle of the digital human image according to the angle adjustment of the digital human image.

2. A digital human display method, characterized in that: include: Acquire a facial image, and preset a plurality of facial image marking points according to the facial image; According to the preset reference coordinate information of the face image, the digital human image is activated, and digital human image marking points corresponding to the face image marking points are generated on the digital human image; Obtain the coordinate information of the marker points of the current face image and the coordinate information of the marker points of the current digital human image; According to the preset reference coordinate information, the coordinate information of the marker points of the current face image is processed to obtain the bias coordinate information of the current face image; According to the preset reference coordinate information, the current face image deviation coordinate information and the current digital human image marker point coordinate information, the digital human image adjustment angle is obtained; According to the adjustment angle of the digital human image, the display angle of the digital human image is adjusted.

3. A digital human display method according to claim 2, characterized in that: The step of generating digital human image marker points corresponding to the facial image marker points on the digital human image comprises: Generating digital human image marker points on the digital human image according to preset facial image marker points; Compare the facial image markers with the digital human image markers, find and delete singular points, and make the processed facial image markers correspond to the digital human image markers one by one.

4. A digital human display method according to claim 3, characterized in that: The step of obtaining the coordinate information of the marker points of the current face image and the coordinate information of the marker points of the current digital human image includes: obtaining the coordinate information of the marker points of each face image and the coordinate information of the marker points of each corresponding digital human image; Performing distance comparison processing on the coordinate information of each facial image marker point and the coordinate information of each corresponding digital human image marker point, and according to the processing result, taking the facial image marker point closest to the digital human image as the current facial image marker point; According to the current facial image markers, the corresponding current digital human image markers are obtained; The coordinate information of the current face image marker points and the coordinate information of the current digital human image marker points are obtained according to the current face image marker points and the current digital human image marker points.

5. A digital human display method according to claim 4, characterized in that: The step of processing the coordinate information of the marker points of the current face image according to the preset reference coordinate information to obtain the bias coordinate information of the current face image includes: Get the screen resolution; The current face image deviation coordinate information is obtained according to the screen resolution and the current face image marker point coordinate information.

6. A digital human display method according to claim 5, characterized in that: The step of obtaining the adjustment angle of the digital human image according to the preset reference coordinate information, the current face image deviation coordinate information and the current digital human image marker point coordinate information comprises: Get the screen resolution; According to the current face image deviation coordinate information and the current digital human image marker point coordinate information, the current digital human image marker point deviation coordinate is obtained; The adjustment angle of the digital human image is obtained according to the screen resolution and the coordinates to which the current digital human image mark point should be deviated.

7. A digital human display method according to claim 2, characterized in that: The step of acquiring the face image includes: acquiring current video stream data, processing the current video stream data, and obtaining the face image.

8. A digital human display method according to claim 7, characterized in that: After the step of acquiring the current video stream data, the method includes: processing the current video stream data, and hiding the digital human image when no face image is detected in the current video stream data according to the processing result.

9. A digital human display method according to claim 7, characterized in that: The current video stream data is processed to obtain gesture information; response information is obtained through querying according to the gesture information, and the response information is fed back through the digital human image.

Images