Virtual image display method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a virtual image display method and device, equipment and a storage medium, is applied to electronic equipment comprising a display screen, and comprises the following steps: determining that a moving object exists in front of the electronic equipment, and displaying a virtual image corresponding to the moving object on the display screen according to a preset driving parameter; determining a target driving parameter of the virtual image based on the motion parameter of the motion object; the motion parameters comprise a first parameter, and the first parameter represents the direction angle of the motion object relative to the display screen; the target driving parameter comprises a first driving parameter, and the first driving parameter represents a rotation coefficient of the virtual image displayed on the display screen; and displaying the virtual image on the display screen according to the target driving parameter. The rotation angle of the virtual image can be determined according to the rotation coefficient, so that the rotation angle of the virtual image is consistent with the direction angle of the moving object relative to the display screen, and the visual angles of the virtual image and the moving object are effectively linked.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, and in particular to a virtual image display method, device, equipment and storage medium. Background Art

[0002] As an interactive portal that supports multi-modal output, virtual images are currently widely used in the fields of hosting, anchoring, customer service, etc. Recently, with the central control screen and other screen devices as the portal to the whole house intelligence, the use of virtual images on screen devices has gradually attracted the attention of smart home appliances and whole house intelligence manufacturers. However, in the related technology, there is a defect that the virtual image and the user's linkage perspective are not synchronized. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a virtual image display method, device, equipment and storage medium, aiming to effectively link the virtual image with the perspective of a moving object.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] An embodiment of the present application provides a method for displaying a virtual image, which is applied to an electronic device including a display screen, and the method includes:

[0006] Determining that there is a moving object in front of the electronic device, displaying a virtual image corresponding to the moving object on the display screen according to preset driving parameters;

[0007] Determining target drive parameters of the virtual image based on motion parameters of the motion object; the motion parameters include a first parameter, the first parameter represents the direction angle of the motion object relative to the display screen; the target drive parameters include a first drive parameter, the first drive parameter represents the rotation coefficient of the virtual image displayed on the display screen;

[0008] The virtual image is displayed on the display screen according to the target driving parameters.

[0009] In the above solution, the step of determining that there is a moving object in front of the electronic device and then displaying a virtual image corresponding to the moving object on the display screen according to preset driving parameters includes:

[0010] It is determined that the display screen is in a lighted state and that there is a moving object in front of the electronic device, and then a virtual image corresponding to the moving object is displayed on the display screen according to the preset driving parameters.

[0011] In the above solution, determining the target driving parameters of the virtual image based on the motion parameters of the motion object includes:

[0012] Determine the first parameter according to the position parameter of the moving object; the position parameter represents the pixel coordinates of the moving object in the virtual coordinate system;

[0013] The first driving parameter is determined based on the first parameter.

[0014] In the above scheme, the position parameter includes a first pixel and a second pixel; the first pixel represents the pixel coordinate of the moving object on the first coordinate axis in the virtual coordinate system; the second pixel represents the pixel coordinate of the moving object on the second coordinate axis in the virtual coordinate system; and determining the first parameter according to the position parameter of the moving object includes:

[0015] The first parameter is determined based on the first pixel and the second pixel.

[0016] In the above solution, determining the first parameter based on the first pixel and the second pixel includes:

[0017] Determine a third pixel of at least two center position parameters of the moving objects based on at least two of the first pixels, and determine a fourth pixel of at least two center position parameters of the moving objects based on at least two of the second pixels; the third pixel represents the pixel coordinate of the first coordinate axis of the center position parameter in the virtual coordinate system; the fourth pixel represents the pixel coordinate of the second coordinate axis of the center position parameter in the virtual coordinate system;

[0018] The first parameter is determined according to the third pixel and the fourth pixel.

[0019] In the above scheme, the motion parameter also includes a second parameter; wherein the second parameter represents the distance between the moving object and the display screen; the target drive parameter also includes a second drive parameter; the second drive parameter represents the scaling factor of the moving object displayed on the display screen; the step of determining the target drive parameter of the moving object based on the motion parameter of the virtual image includes:

[0020] Determining a first zoom threshold based on a first height of the display screen and a second height of the virtual image; the second height represents a height value of the virtual image in the display screen in the preset driving parameters;

[0021] The second driving parameter is determined based on the first scaling threshold, the first threshold and the second parameter; wherein the further away the second parameter is from the first threshold, the more obvious the change of the second driving parameter is; and the closer the second parameter is to the first threshold, the slower the change of the second driving parameter is.

[0022] In the above solution, the method further includes:

[0023] If it is determined that the second parameter is greater than or equal to the first threshold, the first scaling threshold is determined as the second driving parameter.

[0024] In the above solution, the method further includes:

[0025] Determining a second zoom threshold based on a third height of the virtual image and the second height; the third height represents a lower limit of the height of the virtual image in the display screen;

[0026] It is determined that the second parameter is less than or equal to a second threshold, and the second scaling threshold is determined as the second driving parameter; the first threshold is greater than the second threshold; and the first scaling threshold is greater than the second scaling threshold.

[0027] An embodiment of the present application provides a virtual image display device, which is applied to an electronic device including a display screen, and the device includes:

[0028] A first display module, configured to determine that there is a moving object in front of the electronic device, and then display a virtual image corresponding to the moving object on the display screen according to preset driving parameters;

[0029] A first determination module is used to determine the target driving parameters of the virtual image based on the motion parameters of the motion object; the motion parameters include a first parameter, the first parameter represents the direction angle of the motion object relative to the display screen; the target driving parameters include a first driving parameter, the first driving parameter represents the rotation coefficient of the virtual image displayed on the display screen;

[0030] The second display module is used to display the virtual image on the display screen according to the target driving parameters.

[0031] The embodiment of the present application provides a virtual image display device, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein:

[0032] The processor is used to execute the steps of the above-mentioned virtual image display method when running a computer program.

[0033] An embodiment of the present application provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned virtual image display method are implemented.

[0034] The embodiments of the present application provide a method, device, equipment and storage medium for displaying a virtual image, which are applied to an electronic device including a display screen, including: if it is determined that there is a moving object in front of the electronic device, then the virtual image corresponding to the moving object is displayed on the display screen according to preset driving parameters; based on the motion parameters of the moving object, the target driving parameters of the virtual image are determined; the motion parameters include a first parameter, the first parameter represents the direction angle of the moving object relative to the display screen; the target driving parameters include a first driving parameter, the first driving parameter represents the rotation coefficient of the virtual image displayed on the display screen; the virtual image is displayed on the display screen according to the target driving parameters. By adopting the technical solution of the embodiments of the present application, the rotation coefficient of the virtual image displayed on the display screen can be determined according to the motion parameters of the moving object, and then the rotation angle of the virtual image can be determined according to the rotation coefficient, so that the rotation angle of the virtual image is consistent with the direction angle of the moving object relative to the display screen, and the perspective of the virtual image and the moving object are effectively linked. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a schematic diagram of the implementation process of the virtual image display method of the embodiment of the present application;

[0036] Figure 2 A schematic diagram of the structure of a control system corresponding to the virtual image display method of an embodiment of the present application;

[0037] Figure 3 A schematic diagram of a virtual image coordinate system of a virtual image display method in an application example of the present application;

[0038] Figure 4 This is a schematic diagram of a flow chart of a method for realizing a virtual image display in an application example of the present application;

[0039] Figure 5 This is a schematic diagram of the structure of the virtual image display device according to an embodiment of the present application;

[0040] Figure 6 A schematic diagram of a hardware entity structure of a virtual image display device according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0043] The embodiments of the present application provide a virtual image display method, apparatus, device and storage medium, which are applied to an electronic device including a display screen, and aim to effectively realize the linkage between the virtual image and the perspective of a moving object.

[0044] In the related art, the perspective linkage scheme of the virtual image is mainly as follows:

[0045] Solution 1: Touch control, sending control instructions through buttons, etc., to drive the virtual image forward, backward, turn left, turn right, etc.

[0046] Option 2: Real-person driven, using motion capture equipment to capture the movements of the operator's hands, feet, head, etc., and then pass the data to the virtual image to drive the virtual image forward, backward, turn left, turn right, etc.

[0047] Solution 3: Distance-driven, divide the perceived user distance into segments and then perform level scaling.

[0048] However, the shortcoming of Solution 1 is that it requires the user to send control commands, cannot sense the user's relative position, and the perspective cannot be linked to the user; the shortcoming of Solution 2 is that it is driven by a real person, the perspective and action are consistent with the operator behind the scenes, but the perspective cannot be linked to the user; the shortcoming of Solution 3 is that it is distance-driven, and the scaling smoothness is poor.

[0049] The present application provides a method for displaying a virtual image, which is applied to an electronic device including a display screen, such as Figure 1 As shown, the method includes:

[0050] Step 101: Determine that there is a moving object in front of the electronic device, and then display a virtual image corresponding to the moving object on the display screen according to preset driving parameters.

[0051] Exemplarily, the electronic device may be a household device provided with a display screen, which is not limited here. As an example, the electronic device may be an air conditioning device including a display screen.

[0052] Exemplarily, the moving object may be a user of the electronic device, and determining whether there is a moving object in front of the electronic device may be determining whether there is a human body in front of the electronic device; a sensor for sensing the position of the human body may be used to determine whether there is a moving object in front of the electronic device, wherein the sensor for sensing the position of the human body includes but is not limited to infrared, millimeter-wave radar, camera, etc. Preferably, the sensor for sensing the position of the human body may be a millimeter-wave radar.

[0053] Exemplarily, the preset driving parameters may be default driving parameters of the skeleton model. The driving parameters include at least a scaling factor s, a rotation factor α, and a virtual image world coordinate system, wherein the scaling factor s in the default driving parameters is 1, the rotation factor α in the default driving parameters is 0, and the origin of the virtual image world coordinate system in the default driving parameters is the center point of the electronic device including the display screen.

[0054] Exemplarily, the process of determining the virtual image corresponding to the moving object can be determined according to actual conditions, which is not limited here. The virtual image can be mounted on an electronic device including a display screen. The virtual image can be a skeleton model, and the virtual image is displayed on the display screen through model driving. The virtual image can be an algorithm-driven virtual person, specifically, the virtual person can be any one of 3D real person, 3D simulated person, 2.5D real person, 3D cartoon, etc.

[0055] In some embodiments, when it is determined that there is a moving object in front of the electronic device, a virtual image corresponding to the moving object is displayed on the display screen according to preset driving parameters, including:

[0056] When it is determined that the display screen is in a lighted state and that there is a moving object in front of the electronic device, a virtual image corresponding to the moving object is displayed on the display screen according to preset driving parameters.

[0057] Exemplarily, if it is determined that the display screen is in an off state and there is a moving object in front of the electronic device, the state of the display screen is switched to a lit state; when the display screen is in the lit state, a virtual image corresponding to the moving object is displayed on the display screen according to preset driving parameters.

[0058] Step 102: Determine target drive parameters of the virtual image based on motion parameters of the moving object; the motion parameters include a first parameter, the first parameter represents the direction angle of the moving object relative to the display screen; the target drive parameters include a first drive parameter, the first drive parameter represents the rotation coefficient of the virtual image displayed on the display screen.

[0059] Exemplarily, the motion parameter of the moving object may include at least one of the direction angle β of the moving object relative to the display screen, the distance L between the moving object and the display screen, and the number of the moving object. The process of obtaining the motion parameter of the moving object can be determined according to actual conditions and is not limited here. As an example, the direction angle in the motion parameter can be obtained according to a sensor that senses the position of a human body; the distance in the motion parameter can be obtained according to a sensor that senses the distance of a human body; and the number of human bodies in the motion parameter can be obtained according to a sensor that senses the human body.

[0060] Exemplarily, the target driving parameter may include a rotation coefficient α for displaying the virtual image on the display screen and a scaling coefficient s for displaying the virtual image on the display screen.

[0061] In an application example, determining target driving parameters of a virtual image based on motion parameters of a motion object includes:

[0062] Determine the first parameter according to the position parameter of the moving object; the position parameter represents the pixel coordinates of the moving object in the virtual coordinate system;

[0063] A first driving parameter is determined based on the first parameter.

[0064] Exemplarily, determining the first driving parameter based on the first parameter may be determining the first parameter as the first driving parameter, specifically, determining the direction angle of the moving object relative to the display screen as the rotation coefficient of the virtual image displayed on the display screen.

[0065] Exemplarily, the virtual coordinate system may be a virtual image world coordinate system, which includes at least an origin, a first coordinate axis, and a second coordinate axis. For example, the center point of the display screen may be used as the origin, the horizontal direction of the display screen may be used as the first coordinate axis (i.e., the x-axis), and the direction perpendicular to the plane of the display screen may be used as the second coordinate axis (i.e., the z-axis). In some embodiments, the virtual coordinate system may also include a third coordinate axis. For example, the vertical direction of the display screen may be used as the third coordinate axis (i.e., the y-axis).

[0066] Exemplarily, the motion parameter also includes a third parameter, and the third parameter characterizes the number of moving objects; the first parameter is determined according to the position parameter of the moving object, and if the third parameter is determined to be equal to a third threshold, the first parameter is determined according to the position parameter of the moving object; if the third parameter is determined to be greater than the third threshold, the first parameter is determined according to at least two position parameters of at least two moving objects. The third threshold may be 1. For example, when the number of moving objects is 1, a sine operation is performed according to the pixel coordinates of the moving object to obtain the direction angle of the moving object relative to the display screen.

[0067] In an application example, the position parameter includes a first pixel and a second pixel; the first pixel represents the pixel coordinate of the moving object on a first coordinate axis in a virtual coordinate system; the second pixel represents the pixel coordinate of the moving object on a second coordinate axis in the virtual coordinate system; determining the first parameter according to the position parameter of the moving object includes:

[0068] A first parameter is determined based on the first pixel and the second pixel.

[0069] Exemplarily, a sine operation may be performed based on the first pixel and the second pixel to obtain a direction angle of the moving object relative to the display screen; and the direction angle of the moving object relative to the display screen is determined as the first parameter.

[0070] For another example, when the number of moving objects is greater than 1, the center position parameters of at least two moving objects are determined based on at least two position parameters of the at least two moving objects; a sine operation is performed based on the pixel coordinates of the center position parameters to obtain the direction angle of the moving object relative to the display screen.

[0071] In an application example, determining a first parameter based on a first pixel and a second pixel includes:

[0072] Determine a third pixel of the center position parameter of at least two moving objects based on at least two first pixels, and determine a fourth pixel of the center position parameter of at least two moving objects based on at least two second pixels; the third pixel represents the pixel coordinate of the first coordinate axis of the center position parameter in the virtual coordinate system; the fourth pixel represents the pixel coordinate of the second coordinate axis of the center position parameter in the virtual coordinate system;

[0073] A first parameter is determined according to the third pixel and the fourth pixel.

[0074] Exemplarily, based on at least two first pixels, the calculation process of determining the third pixel of the center position parameter of at least two moving objects is as shown in formula (1):

[0075]

[0076] Wherein, x″ is the third pixel of the center of the moving object, xi is the first pixel in the position parameter of the i-th moving object, and N is the total number of pixels of the first pixel in the position parameter of the moving object.

[0077] Exemplarily, based on at least two second pixels, the calculation process of determining the fourth pixel of the center position parameter of at least two moving objects is as shown in formula (2):

[0078]

[0079] Wherein, z″ is the fourth pixel of the center of the moving object, zi is the second pixel in the position parameter of the i-th moving object, and N is the total number of the second pixel in the position parameter of the moving object.

[0080] Exemplarily, the calculation process of determining the first parameter according to the third pixel and the fourth pixel is shown in formula (3):

[0081] β=tan -1 x” / z” (3)

[0082] Among them, β is the first parameter.

[0083] In an application example, the motion parameter further includes a second parameter; wherein the second parameter represents the distance between the moving object and the display screen; the target drive parameter further includes a second drive parameter; the second drive parameter represents a scaling factor for displaying the moving object on the display screen; based on the motion parameter of the moving object, determining the target drive parameter of the virtual image includes:

[0084] Determining a first scaling threshold based on a first height of the display screen and a second height of the virtual image; the second height represents a height value of the virtual image in the display screen in the preset driving parameters;

[0085] Based on the first scaling threshold, the first threshold and the second parameter, a second driving parameter is determined; wherein the further the second parameter is from the first threshold, the more obvious the change of the second driving parameter is; and the closer the second parameter is to the first threshold, the slower the change of the second driving parameter is.

[0086] Exemplarily, the first height may be a height value for displaying an image on a display screen; and the second height may be a height value for displaying a virtual image on a display screen based on preset driving parameters.

[0087] Exemplarily, the first zoom threshold is determined based on the first height of the display screen and the second height of the virtual image. A first ratio of the first height to the second height is determined, and the first ratio is determined as the first zoom threshold. As an example, the first zoom threshold may be 1.25.

[0088] In other embodiments, the virtual image further includes a fourth height, which represents the upper limit of the height of the virtual image in the display screen, and the fourth height may be the maximum height value that can be used to display the virtual image in the display screen. As an example, the fourth height may be determined based on the clarity of the virtual image. It is understood that the larger the height value of the fourth height is, the lower the clarity of the virtual image is; the smaller the height value of the fourth height is, the higher the clarity of the virtual image is. As another example, the fourth height may be determined based on the first height of the display screen. It is understood that the fourth height may be less than or equal to the first height, that is, the virtual image may fill the entire display screen or be displayed at a height less than the entire display screen.

[0089] Exemplarily, the first threshold may be the distance between the moving object corresponding to the fourth height of the virtual image and the display screen. As an example, the first threshold may be 2.5 m. Based on the first scaling threshold, the first threshold and the second parameter, the calculation process of determining the second driving parameter is shown in formula (4):

[0090]

[0091] Among them, s is the second driving parameter, L is the second parameter, S maxis the first scaling threshold.

[0092] It can be understood that the second driving parameter of the virtual image (i.e., the scaling factor) and the distance between the moving object and the display screen are in a downward-opening parabola relationship. The scaling is slow when the distance is far, and the scaling is obvious when the distance is close. This can be more in line with the actual user experience, and also avoids the problem in the related technology that if multiple distances between the moving object and the display screen are in the same interval, the display ratio of the virtual image is the same, that is, the correspondence between the distance and the display ratio is discontinuous, resulting in a poor user experience.

[0093] In an application example, the method further includes:

[0094] If it is determined that the second parameter is greater than or equal to the first threshold, the first scaling threshold is determined as the second driving parameter.

[0095] It can be understood that if the second parameter is greater than or equal to the first threshold, the first zoom threshold will not continue to increase, and the height of the virtual image will not continue to increase; at this time, the first zoom threshold is determined as the second driving parameter, and the first height of the display screen is determined as the height of the virtual image.

[0096] In an application example, the method further includes:

[0097] Determining a second zoom threshold based on a third height and a second height of the virtual image; the third height represents a lower limit of the height of the virtual image in the display screen;

[0098] It is determined that the second parameter is less than or equal to the second threshold, and the second scaling threshold is determined as the second driving parameter; the first threshold is greater than the second threshold; and the first scaling threshold is greater than the second scaling threshold.

[0099] Exemplarily, the third height may be the minimum height value that can be seen by the user when the virtual image is displayed on the display screen; the second zoom threshold is determined based on the third height and the second height of the virtual image, which may be a second ratio of the third height to the second height, and the second ratio is determined as the second zoom threshold. As an example, the second zoom threshold may be 0.5.

[0100] Exemplarily, the second threshold may be the distance between the moving object corresponding to the third height of the virtual image and the display screen. As an example, the second threshold may be 0.5 m.

[0101] Step 103: Displaying the virtual image corresponding to the moving object on the display screen according to the target driving parameters.

[0102] It can be understood that the virtual image corresponding to the moving object can be displayed on the display screen according to the rotation coefficient represented by the first driving parameter and the scaling coefficient represented by the second driving parameter, so that the viewing angle of the virtual image can be linked with the direction and distance of the user relative to the screen device, maintain social distance, always face the user, and have a good sense of anthropomorphism and a good interactive experience.

[0103] In other embodiments, the motion instructions or operation instructions of the moving object can be obtained through human perception or voice recognition; the motion instructions or operation instructions are used to instruct the virtual image to perform corresponding actions or operations.

[0104] In order to understand the embodiments of the present invention, a perspective linkage method of a virtual image on a screen device is used as an example for explanation. The perspective linkage method is applied to a control system, such as Figure 2 As shown, the control system includes a voice service module, a device with a screen, and a virtual image displayed on the device with a screen. The device with a screen may be an air-conditioning device including a display screen.

[0105] Among them, the voice service module includes but is not limited to voice recognition, semantic understanding, voice synthesis, etc., which are used to recognize user voices, understand user intentions and respond; the voices emitted by people can also be synthesized into audio streams to instruct the virtual image to display according to the user's voice instructions. On the screen device, it is equipped with a sound pickup device, human body perception, etc., wherein the sound pickup device can be a microphone array, including but not limited to 2 microphones, 4 microphones, and 6 microphones. The sound pickup device can pass the picked up sound to the voice service module, and the voice service module performs voice recognition on the picked up sound; human body perception is a sensor for perceiving the human body posture, including but not limited to infrared, millimeter wave radar, camera, etc., preferably millimeter wave radar. The virtual image is carried on the screen device, and the virtual image is an algorithm-driven virtual person. The virtual person can be a 3D real person, a 3D simulated person, a 2.5D real person, a 3D cartoon, etc. The virtual image includes but is not limited to a skeleton model, a model driver, etc., wherein the model driver receives the direction and distance of the human body perceived by the human body perception sensor, and outputs the rotation, scaling and other driving parameters of the skeleton model.

[0106] like Figure 3 As shown, the world coordinate system used by the virtual image is preferentially a Cartesian coordinate system, with the vertical direction being the Y axis. Preferably, the default skeletal model scaling factor s is 1, and the rotation factor α is 0. The rotation factor α of the skeletal model is the yaw angle in the Euler angle of the world coordinate system, and the origin of the virtual image world coordinate system is the center point of the screen device. Preferably, the maximum rotation factor is ±45°.

[0107] The avatar can rotate around the X, Y, and Z axes. Preferably, the avatar rotates only around the Y axis. As an example, based on the rotation coefficient α and the scaling coefficient s, the pixel coordinate (x, y, z) is rotated around the Y axis and scaled to the pixel coordinate (x ′ ,y ′ ,z ′ ) is calculated as shown in formula (5):

[0108]

[0109] like Figure 4 As shown, the method flow of this embodiment is as follows:

[0110] Step 401: The sensor senses the presence of a human body.

[0111] Exemplarily, whether a human body exists can be sensed based on the detection signal of the sensor, that is, it can be determined based on the detection signal whether a person passes in front of the device with a screen.

[0112] Step 402: Determine whether the screen is on, if not, execute step 403; if yes, execute step 404.

[0113] Step 403: Turn on the screen, and then execute step 404.

[0114] Step 404: Display the virtual image.

[0115] Here, the driving parameters of the skeleton model may be default values, that is, the virtual image is displayed based on the default values.

[0116] Step 405: Determine whether the direction is sensed.

[0117] Exemplarily, it is determined whether a signal representing a human body direction parameter is received, where the human body direction parameter may be a direction angle of the human body relative to the display screen; if so, step 406 is executed; if not, step 407 is executed.

[0118] Step 406: Determine whether the distance is sensed.

[0119] Exemplarily, it is determined whether a signal representing a human body distance parameter is received, where the human body distance parameter may be the distance between a human body and a display screen; if so, step 408 is executed; if not, step 409 is executed.

[0120] Step 407: Using the driving parameters of the default skeleton model, the virtual image is displayed on the display screen.

[0121] Step 408: Calculate the driving parameters (rotation coefficient, scaling coefficient) of the skeleton model.

[0122] The virtual image receives human body parameters sensed by the sensor, such as direction parameter β, distance parameter L, number of people parameter, etc.

[0123] The calculation of the rotation coefficient α is as follows:

[0124] For a single-person scene, α=β.

[0125] For multi-person scenes, the user center point is calculated based on the scattered point mean strategy. The human body pixel in the sensor field of view is assumed to be (x, z). The calculation process of the user center point (x″, z″) refers to the aforementioned formula (1) and formula (2).

[0126] The user's direction parameter is calculated based on the user's center point. The calculation process refers to the aforementioned formula (3).

[0127] The rotation coefficient is determined based on the orientation parameter, α=β.

[0128] The scaling factor s is calculated as follows:

[0129] Preferably, when the user is 0.5m away from the screen device, the virtual image is at the minimum size, and when the user is more than 2.5m away from the screen device, the virtual image is at the maximum size. max is the screen height / normal display height of the virtual image, preferably S max is 1.25.

[0130] The calculation process of the scaling factor s refers to the aforementioned formula (4).

[0131] Among them, s ≥ 0.5, that is, the minimum scaling factor is 0.5. max The zoom factor is 1.25. For example, when the user is 1.5m away from the device with a screen, the zoom factor is 1.06. When the user is 2m away from the device with a screen, the zoom factor is 1.2.

[0132] Step 409: Drive the skeleton model to display the virtual image.

[0133] The embodiment of the present application estimates the center point in single-user and multi-user scenarios through the scattered point mean strategy to determine the relative angle of the user; it also realizes the mapping of distance and zoom factor through a nonlinear lower-opening parabola, and the zoom effect is more in line with the real user experience; it also estimates the direction angle and position of the virtual image based on the user position and distance, and performs perspective linkage, so that the user maintains a social distance with the virtual image, which is in line with the social etiquette strategy.

[0134] The present application provides a virtual image display device, which is applied to an electronic device including a display screen, such as Figure 5 As shown, the device 500 includes:

[0135] The first display module 501 is used to determine that there is a moving object in front of the electronic device, and then display a virtual image corresponding to the moving object on the display screen according to preset driving parameters;

[0136] A first determination module 502 is used to determine target drive parameters of the virtual image based on motion parameters of the motion object; the motion parameters include a first parameter, the first parameter represents the direction angle of the motion object relative to the display screen; the target drive parameters include a first drive parameter, the first drive parameter represents the rotation coefficient of the virtual image displayed on the display screen;

[0137] The second display module 503 is used to display the virtual image on the display screen according to the target driving parameters.

[0138] In some embodiments, the first display module 501 is further used to determine that the display screen is in a lit state and that there is a moving object in front of the electronic device, and then display a virtual image corresponding to the moving object on the display screen according to preset driving parameters.

[0139] In some embodiments, the first determination module 502 is further used to determine the first parameter according to the position parameter of the moving object; the position parameter represents the pixel coordinates of the moving object in the virtual coordinate system; and determine the first driving parameter based on the first parameter.

[0140] In some embodiments, the position parameters include a first pixel and a second pixel; the first pixel represents the pixel coordinates of the moving object on a first coordinate axis in a virtual coordinate system; the second pixel represents the pixel coordinates of the moving object on a second coordinate axis in the virtual coordinate system; the first determination module 502 is also used to determine the first parameter based on the first pixel and the second pixel.

[0141] In some embodiments, the first determination module 502 is also used to determine a third pixel of the center position parameters of at least two of the moving objects based on at least two of the first pixels, and to determine a fourth pixel of the center position parameters of at least two of the moving objects based on at least two of the second pixels; the third pixel represents the pixel coordinate of the first coordinate axis of the center position parameter in the virtual coordinate system; the fourth pixel represents the pixel coordinate of the second coordinate axis of the center position parameter in the virtual coordinate system; and the first parameter is determined based on the third pixel and the fourth pixel.

[0142] In some embodiments, the motion parameter also includes a second parameter; wherein the second parameter represents the distance between the moving object and the display screen; the target drive parameter also includes a second drive parameter; the second drive parameter represents the zoom factor of the moving object displayed on the display screen; the first determination module 502 is also used to determine the first zoom threshold based on the first height of the display screen and the second height of the virtual image; the second height represents the height value of the virtual image in the display screen in the preset drive parameter; the second drive parameter is determined based on the first zoom threshold, the first threshold and the second parameter; wherein the farther the second parameter is from the first threshold, the more obvious the degree of change of the second drive parameter; the closer the second parameter is to the first threshold, the slower the degree of change of the second drive parameter.

[0143] In some embodiments, the apparatus 500 further includes:

[0144] The second determination module is used to determine that if the second parameter is greater than or equal to the first threshold, then determine the first scaling threshold as the second driving parameter.

[0145] In some embodiments, the apparatus 500 further includes:

[0146] A third determination module is used to determine a second scaling threshold based on a third height and a second height of the virtual image; the third height represents a lower limit of the height of the virtual image in the display screen;

[0147] The fourth determination module is used to determine that if the second parameter is less than or equal to the second threshold, the second scaling threshold is determined as the second driving parameter; the first threshold is greater than the second threshold; and the first scaling threshold is greater than the second scaling threshold.

[0148] It should be noted that: when the virtual image display device provided in the above embodiment is controlled, only the division of the above program modules is used as an example. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the virtual image display device provided in the above embodiment and the aforementioned virtual image display method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0149] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides a virtual image display device. Figure 6 Only an exemplary structure of the virtual image display device is shown, not all structures, and can be implemented as needed. Figure 6 Partial or complete structure shown.

[0150] like Figure 6As shown, the virtual image display device 600 provided in the embodiment of the present application includes: at least one processor 601, a memory 602 and a user interface 603. The various components in the virtual image display device 600 are coupled together through a bus system 604. It can be understood that the bus system 604 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 604 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Figure 6 Various buses are labeled as bus system 604 .

[0151] The user interface 603 may include a display, a keyboard, a mouse, a trackball, a click wheel, keys, buttons, a touch pad or a touch screen.

[0152] The memory 602 in the embodiment of the present application is used to store various types of data to support the operation of the control device. Examples of such data include: any computer program used to operate on the control device.

[0153] The virtual image display method disclosed in the embodiment of the present application can be applied to the processor 601, or implemented by the processor 601. The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the virtual image display method can be completed by the hardware integrated logic circuit or software instructions in the processor 601. The above-mentioned processor 601 can be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 601 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 602, and the processor 601 reads the information in the memory 602, and completes the steps of the virtual image display method provided in the embodiment of the present application in combination with its hardware.

[0154] In an exemplary embodiment, the virtual image display device can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0155] It can be understood that the memory 602 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0156] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, which can be a computer-readable storage medium, for example, a memory 602 storing a computer program, and the computer program can be executed by a processor 601 of the virtual image display device to complete the steps described in the method of the present application embodiment. The computer-readable storage medium can be a memory such as a ROM, a PROM, an EPROM, an EEPROM, a Flash Memory, a magnetic surface memory, an optical disk, or a CD-ROM.

[0157] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0158] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0159] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for displaying a virtual image, characterized in that: Applied to an electronic device including a display screen, the method comprises: Determining that there is a moving object in front of the electronic device, displaying a virtual image corresponding to the moving object on the display screen according to preset driving parameters; Determining target drive parameters of the virtual image based on motion parameters of the motion object; the motion parameters include a first parameter, the first parameter represents the direction angle of the motion object relative to the display screen; the target drive parameters include a first drive parameter, the first drive parameter represents the rotation coefficient of the virtual image displayed on the display screen; The virtual image is displayed on the display screen according to the target driving parameters.

2. The method according to claim 1, characterized in that The step of determining the target driving parameters of the virtual image based on the motion parameters of the motion object includes: Determine the first parameter according to the position parameter of the moving object; the position parameter represents the pixel coordinates of the moving object in the virtual coordinate system; The first driving parameter is determined based on the first parameter.

3. The method according to claim 2, characterized in that The position parameter includes a first pixel and a second pixel; the first pixel represents the pixel coordinate of the moving object on the first coordinate axis in the virtual coordinate system; the second pixel represents the pixel coordinate of the moving object on the second coordinate axis in the virtual coordinate system; The determining the first parameter according to the position parameter of the moving object comprises: The first parameter is determined based on the first pixel and the second pixel.

4. The method according to claim 3, characterized in that: The determining the first parameter based on the first pixel and the second pixel comprises: Determine a third pixel of at least two center position parameters of the moving objects based on at least two of the first pixels, and determine a fourth pixel of at least two center position parameters of the moving objects based on at least two of the second pixels; the third pixel represents the pixel coordinate of the first coordinate axis of the center position parameter in the virtual coordinate system; the fourth pixel represents the pixel coordinate of the second coordinate axis of the center position parameter in the virtual coordinate system; The first parameter is determined according to the third pixel and the fourth pixel.

5. The method according to claim 1, characterized in that The motion parameter also includes a second parameter; wherein the second parameter represents the distance between the moving object and the display screen; the target drive parameter also includes a second drive parameter; the second drive parameter represents the scaling factor of the moving object displayed on the display screen; the determining the target drive parameter of the virtual image based on the motion parameter of the moving object includes: Determining a first zoom threshold based on a first height of the display screen and a second height of the virtual image; the second height represents a height value of the virtual image in the display screen in the preset driving parameters; The second driving parameter is determined based on the first scaling threshold, the first threshold and the second parameter; wherein the further away the second parameter is from the first threshold, the more obvious the change of the second driving parameter is; and the closer the second parameter is to the first threshold, the slower the change of the second driving parameter is.

6. The method according to claim 5, characterized in that The method further comprises: If it is determined that the second parameter is greater than or equal to the first threshold, the first scaling threshold is determined as the second driving parameter.

7. The method according to claim 6, characterized in that The method further comprises: Determining a second zoom threshold based on a third height of the virtual image and the second height; the third height represents a lower limit of the height of the virtual image in the display screen; It is determined that the second parameter is less than or equal to a second threshold, and the second scaling threshold is determined as the second driving parameter; the first threshold is greater than the second threshold; and the first scaling threshold is greater than the second scaling threshold.

8. A virtual image display device, characterized in that: Applicable to electronic devices including display screens, including: A first display module, configured to determine that there is a moving object in front of the electronic device, and then display a virtual image corresponding to the moving object on the display screen according to preset driving parameters; A first determination module is used to determine the target driving parameters of the virtual image based on the motion parameters of the motion object; the motion parameters include a first parameter, the first parameter represents the direction angle of the motion object relative to the display screen; the target driving parameters include a first driving parameter, the first driving parameter represents the rotation coefficient of the virtual image displayed on the display screen; The second display module is used to display the virtual image on the display screen according to the target driving parameters.

9. A virtual image display device, characterized in that: include: A processor and a memory for storing a computer program that can be executed on the processor, wherein: The processor is used to execute the steps of the method according to any one of claims 1 to 7 when running a computer program.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images