Image data display method and device, electronic equipment and readable storage medium

Abstract

The invention relates to an image data display method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring eyeball movement information and head movement information corresponding to image data to be displayed; determining a fixation point offset based on the eyeball movement information and the head movement information; determining a compensation offset corresponding to the to-be-displayed image data based on the fixation point offset and the image reduction scale; determining a target display position of the to-be-displayed image data based on the compensation offset; and displaying the to-be-displayed image data based on the target display position. By adopting the method, the effect of preventing motion sickness can be achieved.

Description

Technical Field

[0001] The present application relates to the field of image display technology, and in particular to an image data display method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] When people use electronic devices while riding in public transportation, they may experience motion sickness. Motion sickness refers to the discomfort caused by a mismatch between perceived body movement and actual movement or a conflict between sensory input. Common symptoms of motion sickness include dizziness, nausea, vomiting, sweating, fatigue, etc.

[0003] In traditional technology, when people use electronic devices while riding in public transportation, the electronic devices control the image data to be displayed in the form of motion effects based on the acceleration obtained in real time, so as to reduce motion sickness caused by sensory conflict. However, obvious or frequent motion effects will aggravate motion sickness, and thus fail to achieve the effect of preventing motion sickness. Summary of the invention

[0004] The embodiments of the present application provide an image data display method, device, electronic device, and computer-readable storage medium to improve the effect of preventing motion sickness.

[0005] In a first aspect, the present application provides an image data display method, comprising:

[0006] Obtaining eye movement information and head movement information corresponding to the image data to be displayed;

[0007] Determining a gaze point offset based on the eye movement information and the head movement information;

[0008] Determining a compensation offset corresponding to the image data to be displayed based on the gaze point offset and the image reduction ratio;

[0009] Determining a target display position of the image data to be displayed based on the compensation offset;

[0010] Based on the target display position, the image data to be displayed is displayed.

[0011] In a second aspect, the present application further provides an image data display device, comprising:

[0012] An acquisition module, used for acquiring eye movement information and head movement information corresponding to the image data to be displayed;

[0013] A gaze point offset determination module, used to determine a gaze point offset based on the eye movement information and the head movement information;

[0014] A compensation offset determination module, used to determine the compensation offset corresponding to the image data to be displayed based on the gaze point offset and the image reduction ratio;

[0015] A display position determination module, used to determine a target display position of the image data to be displayed based on the compensation offset;

[0016] A display module is used to display the image data to be displayed based on the target display position.

[0017] In a third aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0018] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0019] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0020] The above-mentioned image data display method, device, electronic device, computer-readable storage medium and computer program product, when a user uses the electronic device while riding in a vehicle, the electronic device collects the user's eye movement information and head movement information, determines the gaze point offset according to the eye movement information and the head movement information, the gaze point offset represents the offset of the gaze point caused by the user's eye movement and head movement, and the gaze point offset in the opposite direction represents the offset of the gaze point adjusted by the user's neural network, the electronic device determines the compensation offset corresponding to the image data to be displayed according to the gaze point offset and the image reduction ratio, the compensation offset represents the offset that needs to be adjusted for the image data to be displayed, determines the target display position of the image data to be displayed based on the compensation offset, and displays the image data to be displayed based on the target display position. It can be understood that the offset of the image data to be displayed on the display screen using the compensation offset is equivalent to the offset of the eye gaze point adjustment, even if the nervous system adjusts the position of the eye gaze point, the user's eye is still in a focused state, avoiding nystagmus, thereby achieving the effect of preventing motion sickness. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A schematic diagram of a flow chart of an image data display method in one embodiment;

[0023] Figure 2 A schematic flow chart of a step of determining an eye movement offset in one embodiment;

[0024] Figure 3 A schematic flow chart of a step of determining a head movement offset in one embodiment;

[0025] Figure 4 It is a flowchart of the steps of turning on the anti-motion sickness mode in one embodiment;

[0026] Figure 5 A schematic diagram of a flow chart of a step of updating a display position determination in one embodiment;

[0027] Figure 6 A schematic diagram of a display screen in an embodiment;

[0028] Figure 7 A schematic flow chart of a step of determining a motion displacement and a motion speed in one embodiment;

[0029] Figure 8 is a schematic diagram of the connection between the vestibule and the eye in one embodiment;

[0030] Fig. 9 A schematic diagram of a flow chart of an image data display method in another embodiment;

[0031] Fig.10 is a structural block diagram of an image data display device in one embodiment;

[0032] Fig.11 FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0034] In an exemplary embodiment, Figure 1As shown, a method for displaying image data is provided, and the method is applied to electronic devices as an example for explanation. The electronic devices may be, but are not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, smart cars, etc. The portable wearable devices may be smart watches and smart bracelets, etc. It can be understood that the method can also be applied to a system including an electronic device and a server, and is implemented through the interaction between the electronic device and the server. In this embodiment, the method is applied to an electronic device as an example for explanation, including steps 102 to 110, wherein:

[0035] Step 102: Obtain eye movement information and head movement information corresponding to the image data to be displayed.

[0036] The image data to be displayed refers to a data set representing the image to be displayed, and the image data to be displayed includes the position information and color information of each pixel in the image to be displayed. It can be understood that the image currently displayed on the display screen of the electronic device is the current image, and the image displayed in the next frame on the display screen is the image to be displayed. The data set representing the image to be displayed is the image data to be displayed. The eye movement information and head movement information corresponding to the image data to be displayed refer to the eye movement information and head movement information used to determine the target display position of the image data to be displayed.

[0037] Eye movement information refers to data information that characterizes eye movement, and the eye movement information includes but is not limited to the gaze point position and eye rotation angle. The gaze point position refers to the position of the eye's gaze point on the display screen of the electronic device when the eye is looking at the electronic device, and the gaze point position is usually expressed in two-dimensional coordinates. The eye rotation angle refers to the rotation angle of the eye around its own axis in the head. The eye rotation angle is usually divided into two main directions, the horizontal rotation angle and the vertical rotation angle. The horizontal rotation angle refers to the angle of rotation of the eye around the vertical axis, which can be understood as the angle of rotation of the eye when the eye moves to the left or right. The vertical rotation angle refers to the angle of rotation of the eye around the horizontal axis, which can be understood as the angle of rotation of the eye when the eye moves up or down.

[0038] Head movement information refers to data information that describes the position and posture of the head in three-dimensional space. Head movement information includes but is not limited to head position and head posture. Head position refers to the coordinates of the specific position of the head in three-dimensional space, and the head position can be represented by three-dimensional coordinates. Head posture refers to the rotation state of the head in three-dimensional space, usually represented by yaw angle, pitch angle and roll angle. The yaw angle refers to the angle of rotation of the head around the vertical axis, which can be understood as the angle of left and right rotation of the head; the pitch angle refers to the angle of rotation of the head around the horizontal axis, which can be understood as the angle of nodding of the head up and down; the roll angle refers to the rotation of the head around the front-back axis, which can be understood as the angle of left-right tilt of the head.

[0039] Exemplarily, when the anti-motion sickness mode is turned on, the electronic device turns on the front camera and obtains eye movement information and head movement information corresponding to the image data to be displayed through the front camera.

[0040] In an exemplary embodiment, when the motion sickness prevention mode is turned on, the electronic device turns on the front camera, obtains current eye movement information and head movement information through the front camera, and uses the current eye movement information and head movement information as the eye movement information and head movement information corresponding to the next frame of image data to be displayed.

[0041] Step 104: determine the gaze point offset based on the eye movement information and the head movement information.

[0042] The gaze point offset refers to the change in the position of the gaze point due to eye movement and head movement. The gaze point offset can be expressed in two-dimensional coordinates.

[0043] Exemplarily, the electronic device calculates the gaze point offset based on the eye movement information and the head movement information.

[0044] Step 106: Determine a compensation offset corresponding to the image data to be displayed based on the gaze point offset and the image reduction ratio.

[0045] Among them, the image reduction ratio refers to the ratio of reduction of the image to be displayed. The image reduction ratio can be pre-set and can be determined according to the size of the electronic device display screen. For example, the image reduction ratio is 90%, the original length of the image to be displayed is L, and the original width is W, then the final display length is 0.9L, and the final display width is 0.9W.

[0046] Exemplarily, the electronic device obtains the image reduction ratio, multiplies the gaze point translation amount by the image reduction ratio, and obtains the compensation offset corresponding to the image data to be displayed.

[0047] Step 108: determining a target display position of the image data to be displayed based on the compensation offset.

[0048] The target display position refers to the position of the image to be displayed corresponding to the image data to be displayed in the display screen. The target display position can be the position of the center point of the image to be displayed corresponding to the image data to be displayed in the display screen, that is, the position of the center point of the image is used to represent the display position of the image in the display screen. The target display position can also be the position of each pixel in the image data to be displayed in the display screen, that is, the position of each pixel in the image is used to represent the display position of the image in the display screen.

[0049] Exemplarily, the electronic device obtains the original display position of the image corresponding to the image data to be displayed, and subtracts the compensation offset from the original display position of the image to be displayed to obtain the target display position of the image data to be displayed. The original display position of the image refers to the original position of the center point of the image to be displayed corresponding to the image data to be displayed.

[0050] In an exemplary embodiment, the electronic device obtains the original display position corresponding to each pixel in the image data to be displayed, determines the reduced display position corresponding to the pixel based on the reduction ratio and the original display position corresponding to the pixel, and subtracts the compensation offset from the reduced display position corresponding to the pixel to obtain the target display position of the pixel. The original display position corresponding to the pixel refers to the initial display position of the pixel; the reduced display position refers to the display position of the pixel on the display screen after the image to be displayed corresponding to the image data to be displayed is reduced.

[0051] Step 110: displaying the image data to be displayed based on the target display position.

[0052] Exemplarily, the electronic device displays the image data to be displayed according to the target display position.

[0053] In the above-mentioned image data display method, when a user uses an electronic device while riding in a vehicle, the movement of the vehicle will cause the user and the electronic device to produce the same movement, that is, the user and the electronic device are relatively still. At this time, the user does not need to adjust the position of the eye's gaze point, but the user's nervous system senses the user's movement and adjusts the position of the eye's gaze point according to the user's movement. Adjusting the position of the eye's gaze point causes the eyes to lose focus. In order to refocus the eyes, the position of the eye's gaze point needs to be adjusted again. Repeated adjustment of the position of the eye's gaze point causes nystagmus, which causes the user to experience motion sickness. When a user uses an electronic device while riding in a vehicle, the electronic device collects the user's eye movement information and head movement information, determines a gaze point offset based on the eye movement information and the head movement information, and the gaze point offset represents the offset of the gaze point caused by the user's eye movement and head movement. The gaze point offset in the opposite direction represents the offset of the gaze point adjusted by the user's neural network. The electronic device determines a compensation offset corresponding to the image data to be displayed based on the gaze point offset and the image reduction ratio, and the compensation offset represents the offset that needs to be adjusted for the image data to be displayed. A target display position of the image data to be displayed is determined based on the compensation offset, and the image data to be displayed is displayed based on the target display position. It can be understood that the offset of the image data to be displayed on the display screen caused by the use of the compensation offset is equivalent to the offset of the eye gaze point adjustment. Even if the nervous system adjusts the position of the eye gaze point, the user's eyes are still in a focused state, avoiding nystagmus, thereby achieving the effect of preventing motion sickness.

[0054] In an exemplary embodiment, determining the gaze point offset based on the eye movement information and the head movement information includes:

[0055] Based on the head position in the head movement information and the eye rotation angle in the eye movement information, the eye movement offset is determined; based on the head position and the head posture in the head movement information, the head movement offset is determined; based on the eye movement offset and the head movement offset, the gaze point offset is obtained.

[0056] The eye movement offset refers to the degree of offset of the gaze point caused by eye movement, and the head movement offset refers to the degree of offset of the gaze point caused by head movement.

[0057] Exemplarily, the electronic device obtains the head position from the head movement information, determines the straight-line distance between the eyeball and the display screen based on the head position, then obtains the eye rotation angle from the eye movement information, calculates the straight-line distance and the eye rotation angle to obtain the eye movement offset; the eye movement offset is added to the head movement offset to obtain the gaze point offset.

[0058] In an exemplary embodiment, when the gaze point position in the eye movement information and the head position are located in the same coordinate system, the straight-line distance between the eyeball and the display screen is as follows:

[0059]

[0060] Where D is the straight-line distance between the eyeball and the display screen; (x, y, z) is the head position.

[0061] In the present embodiment, since both eye movement and head movement will cause the offset of the gaze point, the eye movement offset is determined according to the head position in the head movement information and the eye rotation angle in the eye movement information, and the head movement offset is determined according to the head position and the head posture in the head movement information. The eye movement offset is then added to the head movement offset to obtain the gaze point offset. The gaze point offset represents the offset of the gaze point caused by the eye movement and head movement of the user, and the gaze point offset in the opposite direction represents the offset of the gaze point adjusted by the neural network, thereby providing accurate basic data for the subsequent determination of the compensation offset.

[0062] In an exemplary embodiment, when the gaze point position in the eye movement information is located in a first coordinate system and the head position is located in a second coordinate system, determining the eye movement offset based on the head position in the head movement information and the eye rotation angle in the eye movement information includes:

[0063] Based on the relative position relationship between the origin of the first coordinate system and the origin of the second coordinate system, determine the matching position corresponding to the head position in the first coordinate system; based on the matching position and the eye rotation angle, determine the eye movement offset; based on the matching position and the head posture in the head movement information, determine the head movement offset.

[0064] Among them, the first coordinate system refers to the display screen coordinate system, and the display screen coordinate system refers to a coordinate system with a corner of the display screen as the origin. For example, the first coordinate system takes the upper left corner of the display screen as the origin. The second coordinate system refers to the camera coordinate system, and the camera coordinate system refers to a coordinate system with the camera of the electronic device as the origin. For example, the second coordinate system takes the front camera of the electronic device as the origin. The relative position relationship refers to the distance and position difference between the origin of the first coordinate system and the origin of the second coordinate system in space. The relative position relationship can be expressed by three-dimensional coordinates. For example, the first coordinate takes the upper left corner of the electronic device display screen as the origin, and the second coordinate system takes the front camera of the electronic device as the origin. The relative position relationship represents the distance and position difference between the upper left corner of the display screen and the front camera in space. The matching position refers to the position of the head of the user of the electronic device in the first coordinate.

[0065] Exemplarily, when the gaze point position in the eye movement information is located in the first coordinate system and the head position is located in the second coordinate system, the electronic device obtains the relative position relationship between the origin of the first coordinate system and the origin of the second coordinate system, adds the head position to the relative position relationship, and obtains the matching position corresponding to the head position in the first coordinate system, and then determines the eye movement offset based on the matching position and the eye rotation angle, and determines the head movement offset based on the matching position and the head posture in the head movement information.

[0066] In an exemplary embodiment, the matching position (x', y', z') corresponding to the head position in the first coordinate system is as follows:

[0067] (x',y',z')=(x+Δx,y+Δy,z+Δz) Formula (2)

[0068] Among them, (x, y, z) is the head position, and (Δx, Δy, Δz) is the relative position relationship between the origin of the first coordinate system and the origin of the second coordinate system.

[0069] In this embodiment, when the gaze point position in the eye movement information obtained by the electronic device is located in the first coordinate system and the head position is located in the second coordinate system, that is, the gaze point and the head position are located in different coordinates, it is first necessary to determine the matching position corresponding to the head position in the first coordinate system based on the relative position relationship between the origin of the first coordinate system and the origin of the second coordinate system, which provides accurate basic data for the subsequent calculation of the eye movement offset and the head movement offset.

[0070] In an exemplary embodiment, Figure 2 As shown, the eyeball rotation angle includes the eyeball first direction rotation angle and the eyeball second direction rotation angle; based on the matching position and the eyeball rotation angle, the eyeball movement offset is determined, including:

[0071] Step 202: Determine the straight-line distance between the eyeball and the display screen based on the matching position.

[0072] Among them, the straight-line distance refers to the distance between the eyeball and the display screen.

[0073] Exemplarily, the electronic device uses the matching positions to calculate the straight-line distance between the eyeball and the display screen.

[0074] In an exemplary embodiment, the straight-line distance between the eyeball and the display screen is as follows:

[0075]

[0076] Where D is the straight-line distance between the eyeball and the display screen; (x', y', z') is the matching position.

[0077] Step 204: Determine the offset distance of the eye movement in the first direction based on the straight-line distance and the eye rotation angle in the first direction.

[0078] The first direction eye rotation angle refers to the horizontal rotation angle of the eye, that is, the angle of eye rotation when the eye moves left or right. The first direction eye movement offset distance refers to the horizontal offset of the gaze point caused by the eye movement.

[0079] Exemplarily, the electronic device multiplies the straight-line distance by the tangent value of the eye's rotation angle in the first direction to obtain the eye's displacement distance in the first direction.

[0080] In an exemplary embodiment, the offset distance of the eye movement in the first direction is as follows:

[0081] Δx eye =D×tan(α yaw ) Formula (4)

[0082] Where Δx eye is the offset distance of the first direction of eye movement; D is the straight-line distance between the eye and the display screen; α yaw The first rotation angle of the eyeball.

[0083] Step 206: Determine the offset distance of the eye movement in the second direction based on the straight-line distance and the eye rotation angle in the second direction.

[0084] The eyeball second direction rotation angle refers to the vertical rotation angle of the eyeball, that is, the angle of eyeball rotation when the eye moves upward or downward. The eyeball movement second direction offset distance refers to the vertical offset of the gaze point caused by the eyeball movement.

[0085] Exemplarily, the electronic device multiplies the straight-line distance by the tangent value of the eye's rotation angle in the second direction to obtain the eye's displacement distance in the second direction.

[0086] In an exemplary embodiment, the offset distance of the eye movement in the second direction is as follows:

[0087] Δy eye =D×tan(α pitch ) Formula (5)

[0088] Among them, Δy eye is the offset distance of the second direction of eye movement; D is the straight-line distance between the eye and the display screen; α pitch The second rotation angle of the eyeball.

[0089] Step 208: Obtain an eye movement offset based on the eye movement first direction offset distance and the eye movement second direction offset distance.

[0090] The eye movement offset refers to the offset of the gaze point caused by eye movement. The eye movement offset can be expressed in two-dimensional coordinates. For example, the offset distance of the first direction of eye movement is Δx eye , the second direction of eye movement offset distance is Δy eye , then the eye movement offset is (Δx eye ,Δy eye ).

[0091] Exemplarily, the electronic device obtains the eye movement offset according to the eye movement first direction offset distance and the eye movement second direction offset distance.

[0092] In this embodiment, the electronic device determines the eye movement offset based on the first direction rotation angle and the second direction rotation angle of the eye, and the straight-line distance between the eye and the display screen. The eye movement offset characterizes the offset of the gaze point caused by the eye movement, and provides basic data for the subsequent determination of the gaze point offset.

[0093] In an exemplary embodiment, Figure 3 As shown, the head posture includes the first direction rotation angle of the head, the second direction rotation angle of the head and the third direction rotation angle of the head; the matching position includes the matching first direction position, the matching second direction position and the matching third direction position; based on the matching position and the head posture in the head movement information, the head movement offset is determined, including:

[0094] Step 302: Determine a first lateral offset distance based on matching the third direction position and the first direction rotation angle of the head.

[0095] Among them, the matching third direction position refers to the position of the user's head on the Z axis of the first coordinate system. The first direction rotation angle of the head refers to the yaw angle of the user's head, that is, the angle of the user's head turning left and right. The first lateral offset distance refers to the horizontal offset of the gaze point caused by the left and right rotation of the head.

[0096] Exemplarily, the electronic device multiplies the matched third direction position by the tangent value of the rotation angle of the head in the first direction to obtain the first lateral offset distance.

[0097] In an exemplary embodiment, the first lateral offset distance is as follows:

[0098] Δx yaw =z'×tan(β yaw ) Formula (6)

[0099] Among them, Δy yaw is the first lateral offset distance; z' is the matching third direction position; β yawThe rotation angle of the head in the first direction.

[0100] Step 304: Determine a first longitudinal offset distance based on matching the third direction position and the second direction rotation angle of the head.

[0101] The second rotation angle of the head refers to the pitch angle of the user's head, that is, the angle of the user's head nodding up and down. The first longitudinal offset distance refers to the vertical offset of the gaze point caused by the head nodding up and down.

[0102] Exemplarily, the electronic device multiplies the matched third direction position by the negative of the direct tangent value of the second direction rotation angle of the head to obtain the first longitudinal offset distance.

[0103] In an exemplary embodiment, the first longitudinal offset distance is as follows:

[0104] Δy pitch = -z'×tan(β pitch ) Formula (7)

[0105] Among them, Δy pitch is the first longitudinal offset distance; z' is the matching third direction position; β pitch The second rotation angle of the head.

[0106] Step 306: Determine a second longitudinal offset distance based on matching the first direction position and the third direction rotation angle of the head.

[0107] Among them, the first direction position matching refers to the position of the user's head on the X-axis of the first coordinate system. The third direction rotation angle of the head refers to the roll angle of the user's head, that is, the angle at which the user's head tilts left and right. The second longitudinal offset distance refers to the vertical offset of the gaze point caused by the left and right tilt of the head.

[0108] Exemplarily, the electronic device multiplies the matched first direction position by the negative of the direct tangent value of the third direction rotation angle of the head to obtain the second longitudinal offset distance.

[0109] In an exemplary embodiment, the second longitudinal offset distance is as follows:

[0110] Δy roll = -x'×tan(β roll ) Formula (8)

[0111] Among them, Δy roll is the second longitudinal offset distance; x' is the matching first direction position; β roll The rotation angle of the head in the third direction.

[0112] Step 308: Determine a second lateral offset distance based on matching the second direction position and the third direction rotation angle of the head.

[0113] The second direction position is the position of the user's head on the Y axis of the first coordinate system. The second lateral offset distance is the horizontal offset of the gaze point caused by the left and right tilt of the head.

[0114] Exemplarily, the electronic device multiplies the matched second direction position by the negative of the direct tangent value of the third direction rotation angle of the head to obtain the second longitudinal offset distance.

[0115] In an exemplary embodiment, the second lateral offset distance is as follows:

[0116] Δx roll =y'×tan(β roll ) Formula (9)

[0117] Where Δx roll is the second longitudinal offset distance; y' is the matching second direction position; β roll is the rotation angle of the head in the third direction.

[0118] Step 310: Obtain the offset distance of the head movement in the first direction based on the first lateral offset distance and the second lateral offset distance.

[0119] The first direction offset distance of the head movement refers to the horizontal offset of the gaze point caused by the head movement.

[0120] Exemplarily, the electronic device adds the first lateral offset distance to the second lateral offset distance to obtain the offset distance in the first direction of the head movement.

[0121] In an exemplary embodiment, the offset distance of the head movement in the first direction is as follows:

[0122] Δx head =Δx yaw +Δx roll Formula (10)

[0123] Where Δx head Δx is the offset distance of the first direction of head movement; yaw is the first lateral offset distance; Δx roll is the second lateral offset distance.

[0124] Step 312: Obtain a second direction offset distance of the head movement based on the first longitudinal offset distance and the second longitudinal offset distance.

[0125] The second direction offset distance of the head movement refers to the vertical offset of the gaze point caused by the head movement.

[0126] Exemplarily, the electronic device adds the first longitudinal offset distance to the second longitudinal offset distance to obtain the offset distance in the second direction of the head movement.

[0127] In an exemplary embodiment, the offset distance of the second direction of head movement is as follows:

[0128] Δy head =Δy pitch +Δy roll Formula (11)

[0129] Among them, Δy head Δy is the offset distance of the second direction of head movement; pitch is the first longitudinal offset distance; Δy roll is the second longitudinal offset distance.

[0130] Step 314: Obtain a head movement offset based on the offset distance in the first direction of the head movement and the offset distance in the second direction of the head movement.

[0131] Exemplarily, the electronic device obtains the head movement offset according to the offset distance in the first direction of the head movement and the offset distance in the second direction of the head movement. For example, the offset distance in the first direction of the head movement is Δx head , the second direction offset distance of head movement is Δy head , then the head movement offset is (Δx head ,Δy head ).

[0132] In this embodiment, the head movement offset is determined based on the first direction rotation angle, the second direction rotation angle and the third direction rotation angle of the head in the head posture, and the matching first direction position, the matching second direction position and the matching third direction position in the matching position. The head movement offset characterizes the offset of the gaze point caused by the head movement, and provides basic data for the subsequent determination of the gaze point offset.

[0133] In an exemplary embodiment, determining a target display position of image data to be displayed based on the compensation offset includes:

[0134] For each pixel in the image data to be displayed, the reduced display position of the pixel is obtained; the reduced display position refers to the display position of the image data to be displayed on the display screen after it is reduced based on the image reduction ratio; based on the difference between the reduced display position of the pixel and the compensation offset, the target display position of the pixel is obtained.

[0135] Among them, a pixel refers to a basic unit constituting an image, and it can be understood that the image data to be displayed includes multiple pixels, and multiple pixels can constitute the image to be displayed. The reduced display position refers to the display position of a pixel on the display screen after the image to be displayed corresponding to the image data to be displayed is reduced based on the image reduction ratio. The reduced display position of a pixel can be determined based on the initial display position of the pixel and the image reduction ratio.

[0136] Exemplarily, for each pixel in the image data to be displayed, the electronic device obtains the reduced display position of the pixel, and subtracts the compensation offset from the reduced display position of the pixel to obtain the target display position of the pixel.

[0137] In an exemplary embodiment, the target display position of the pixel is as follows:

[0138] (x new ,y new )=(x re, y re )-(Δx,Δy)×A Formula (12)

[0139] Among them, (x new ,y new ) is the target display position of the pixel; (x re, y re ) is the reduced display position of the pixel; (Δx, Δy) is the gaze point offset; A is the image reduction ratio; (Δx, Δy)×A is the compensation offset.

[0140] In this embodiment, the target display position of the image data to be displayed is determined based on the compensation offset. It can be understood that the compensation offset is used to make the offset of the image data to be displayed in the display screen equal to the offset for adjusting the eye gaze point. Even if the nervous system adjusts the position of the eye gaze point, the user's eyes are still in a focused state, avoiding nystagmus, thereby achieving the effect of alleviating motion sickness symptoms.

[0141] In an exemplary embodiment, Figure 4 As shown, before obtaining the eye movement information and head movement information corresponding to the image data to be displayed, it also includes:

[0142] Step 402, obtaining a first angular acceleration set and a first linear acceleration set at a first moment, and a second angular acceleration set and a second linear acceleration set corresponding to a second moment; the first moment is before the second moment.

[0143] Among them, the first angular acceleration set refers to the angular acceleration set obtained at the first moment, and the first angular acceleration set includes the first angular accelerations in three directions, namely the first lateral angular acceleration, the first longitudinal angular acceleration and the first vertical angular acceleration. The first lateral angular acceleration is the angular acceleration in the X-axis direction at the first moment, the first longitudinal angular acceleration is the angular acceleration in the Y-axis direction at the first moment, and the first vertical angular acceleration is the angular acceleration in the Z-axis direction at the first moment. The first linear acceleration set refers to the linear acceleration set obtained at the first moment, and the first linear acceleration set includes the first linear accelerations in three directions, namely the first lateral linear acceleration, the first longitudinal linear acceleration and the first vertical linear acceleration. The first lateral linear acceleration is the linear acceleration in the X-axis direction at the first moment, the first longitudinal linear acceleration is the linear acceleration in the Y-axis direction at the first moment, and the first vertical linear acceleration is the linear acceleration in the Z-axis direction at the first moment.

[0144] The second angular acceleration set refers to the angular acceleration set obtained at the second moment, and the second angular acceleration set includes the second angular accelerations in three directions, namely the second lateral angular acceleration, the second longitudinal angular acceleration, and the second vertical angular acceleration. The second lateral angular acceleration is the angular acceleration in the X-axis direction at the second moment, the second longitudinal angular acceleration is the angular acceleration in the Y-axis direction at the second moment, and the second vertical angular acceleration is the angular acceleration in the Z-axis direction at the second moment. The second linear acceleration set refers to the linear acceleration set obtained at the second moment, and the second linear acceleration set includes the second linear accelerations in three directions, namely the second lateral linear acceleration, the second longitudinal linear acceleration, and the second vertical linear acceleration. The second lateral linear acceleration is the linear acceleration in the X-axis direction at the second moment, the second longitudinal linear acceleration is the linear acceleration in the Y-axis direction at the second moment, and the second vertical linear acceleration is the linear acceleration in the Z-axis direction at the second moment.

[0145] For example, before the anti-motion sickness mode is turned on, the electronic device obtains a first angular acceleration set at a first moment through an accelerometer, obtains a first linear acceleration set at the first moment through a gyroscope, and after a preset time, obtains a second angular acceleration set corresponding to a second moment through an accelerometer, and obtains a second linear acceleration set at the second moment through a gyroscope. The preset time refers to a preset time, which can be set according to actual needs and is not limited here.

[0146] In an exemplary embodiment, when the display screen of the electronic device is in a bright screen state, the electronic device obtains a first angular acceleration set at a first moment through an accelerometer, obtains a first linear acceleration set at the first moment through a gyroscope, and after a preset time, obtains a second angular acceleration set corresponding to a second moment through the accelerometer, and obtains a second linear acceleration set at the second moment through the gyroscope. That is, the electronic device obtains the first angular acceleration set and the first linear acceleration set at the first moment, and the second angular acceleration set and the second linear acceleration set corresponding to the second moment only when the electronic device is in use.

[0147] Step 404: for the first angular acceleration and the second angular acceleration in the same direction in the first angular acceleration set and the second angular acceleration set, determine the angular acceleration difference value between the first angular acceleration and the second angular acceleration.

[0148] The angular acceleration difference value refers to the difference between the first angular acceleration and the second angular acceleration in the same direction.

[0149] Exemplarily, the electronic device uses the absolute value of the difference between the first angular acceleration and the second angular acceleration in the same direction in the first angular acceleration set and the second angular acceleration set as the angular acceleration difference value between the first angular acceleration and the second angular acceleration in the same direction.

[0150] Step 406: for the first linear acceleration and the second linear acceleration in the same direction in the first linear acceleration set and the second linear acceleration set, determine a linear acceleration difference value between the first linear acceleration and the second linear acceleration.

[0151] The linear acceleration difference value refers to the difference between the first linear acceleration and the second linear acceleration in the same direction.

[0152] Exemplarily, the electronic device uses the absolute value of the difference between the first linear acceleration and the second linear acceleration in the same direction in the first linear acceleration set and the second linear acceleration set as the linear acceleration difference value between the first linear acceleration and the second linear acceleration.

[0153] Step 408: When at least one angular acceleration difference value is greater than the angular acceleration difference threshold, or at least one linear acceleration difference value is greater than the linear acceleration difference threshold, the anti-motion sickness mode is enabled.

[0154] Among them, the angular acceleration difference threshold refers to the threshold used for comparison with the angular acceleration difference value, and the angular acceleration difference threshold refers to a pre-set threshold used to determine whether to turn on the anti-motion sickness mode. The linear acceleration difference threshold refers to the threshold used for comparison with the linear acceleration difference value, and the linear acceleration difference threshold refers to a pre-set threshold used to determine whether to turn on the anti-motion sickness mode. The anti-motion sickness mode refers to a mode that prevents users from experiencing motion sickness. Motion sickness refers to discomfort symptoms caused by a mismatch between perceived body movement and actual movement or a conflict between sensory inputs. Motion sickness usually occurs when riding in vehicles, ships, airplanes and other means of transportation, or when using virtual reality equipment. Common symptoms include dizziness, nausea, vomiting, sweating, fatigue, etc. It can be understood that when users use electronic devices while riding in vehicles, ships or airplanes and other means of transportation, the anti-motion sickness mode is used to prevent users from experiencing motion sickness.

[0155] Exemplarily, the electronic device compares each angular acceleration difference value with the angular acceleration difference threshold, and compares each linear acceleration difference value with the linear acceleration difference threshold, and if at least one angular acceleration difference value is greater than the angular acceleration difference threshold, or at least one linear acceleration difference value is greater than the linear acceleration difference threshold, the anti-motion sickness mode is turned on.

[0156] In an exemplary embodiment, the electronic device compares each angular acceleration difference value with the angular acceleration difference threshold, and compares each linear acceleration difference value with the linear acceleration difference threshold. If at least one angular acceleration difference value is greater than the angular acceleration difference threshold, or at least one linear acceleration difference value is greater than the linear acceleration difference threshold, it is determined that the vehicle is in a turbulent state and the anti-motion sickness mode is turned on.

[0157] In this embodiment, when there is at least one angular acceleration difference value greater than the angular acceleration difference threshold, or at least one linear acceleration difference value greater than the linear acceleration difference threshold, the anti-motion sickness mode is turned on, that is, the angular acceleration difference value is greater than the angular acceleration difference threshold, or the linear acceleration difference value is greater than the linear acceleration difference threshold, it is judged that the user is using an electronic device on a vehicle, thereby turning on the anti-motion sickness mode.

[0158] In an exemplary embodiment, Figure 5 As shown, the image data display method also includes:

[0159] Step 502: When displaying motion reference image data, obtain a set of linear accelerations corresponding to the image data to be displayed and an initial display position of the motion reference image data.

[0160] The motion reference image data refers to a data set representing the motion reference image, the motion reference image refers to an image located around the image to be displayed corresponding to the image data to be displayed, the motion parameter image may be located on at least one of the upper side, lower side, left side and right side of the image to be displayed, the shape and number of the motion reference image may be set according to actual needs, and are not limited here, for example, Figure 6 The display screen of the electronic device shown in the figure, 602 is the image to be displayed corresponding to the image data to be displayed, and the circular bubbles located on the left and right sides of the image to be displayed 602 are the motion reference images corresponding to the motion reference image data. The linear acceleration set corresponding to the image data to be displayed refers to the linear acceleration set used to determine the updated display position of the motion reference image displayed simultaneously with the image to be displayed, and the linear acceleration set includes the first direction acceleration, the second direction acceleration and the third direction acceleration. The first direction acceleration is the horizontal linear acceleration, that is, the linear acceleration in the X-axis direction; the second direction acceleration is the longitudinal linear acceleration, that is, the linear acceleration in the Y-axis direction; the third direction acceleration is the vertical direction linear acceleration, that is, the linear acceleration in the Z-axis direction. The initial display position refers to the display position corresponding to the motion reference image data at the moment of obtaining the linear acceleration set. The initial display position can be the position of the center point of the motion reference image corresponding to the motion reference image data in the display screen. If there are multiple motion reference images, there are multiple initial display positions, and each motion reference image corresponds to an initial display position; the initial display position can be the position of each pixel point in the motion reference image data in the display screen.

[0161] Exemplarily, when a motion reference image corresponding to the motion reference image data is displayed on a display screen, the electronic device obtains a set of linear accelerations corresponding to the image data to be displayed and an initial display position of the motion reference image data while obtaining eye movement information and head movement information corresponding to the image data to be displayed.

[0162] Step 504 : determining the lateral motion direction, longitudinal motion direction and motion speed of the motion reference image data based on the linear acceleration set.

[0163] The horizontal motion direction refers to the motion direction of the motion reference image corresponding to the motion reference image data on the X axis. The vertical motion direction refers to the motion direction of the motion reference image corresponding to the motion reference image data on the Y axis. The motion speed refers to the speed of the motion reference image corresponding to the motion reference image data.

[0164] Exemplarily, the electronic device calculates the first direction acceleration, the second direction acceleration, and the third direction acceleration in the linear acceleration set to obtain the lateral motion direction, the longitudinal motion direction, and the motion speed of the motion reference image data.

[0165] Step 506: Determine the lateral running distance of the motion reference image data based on the lateral motion direction and motion speed.

[0166] The lateral movement distance refers to the distance that the motion reference image corresponding to the motion reference image data moves in the X-axis direction.

[0167] Exemplarily, the electronic device calculates the product of the lateral motion direction, the motion speed and the motion duration to obtain the lateral running distance of the motion reference image data. The motion duration refers to the duration of the motion reference image movement, and the motion duration can be the display interval duration between two frames of images.

[0168] Step 508: Determine the longitudinal running distance of the motion reference image data based on the longitudinal motion direction and motion speed.

[0169] The longitudinal movement distance refers to the distance that the motion reference image corresponding to the motion reference image data moves in the Y-axis direction.

[0170] Exemplarily, the electronic device calculates the product of the longitudinal motion direction, the motion speed and the motion duration to determine the longitudinal running distance of the motion reference image data.

[0171] Step 510: determining an updated display position of the motion reference image data based on the initial display position, the horizontal running distance, and the vertical running distance.

[0172] The updated display position refers to the updated display position of the motion reference image corresponding to the motion reference image data, which can be understood as the display position of the motion reference image on the display screen when the image to be displayed corresponding to the image data to be displayed is displayed on the display screen.

[0173] Exemplarily, the electronic device adds the horizontal running distance to the first direction display position in the initial display position, and adds the vertical running distance to the second direction display position in the initial display position, to obtain the updated display position of the motion reference image data.

[0174] In an exemplary embodiment, updating the display position (New Position x, New Position y) is as follows:

[0175] New Position x=Current Position x+v×Direction x×Δt Formula (13)

[0176] New Position y=Current Position y+v×Direction y×Δt Formula (14)

[0177] Among them, (Current Position x, Current Position y) is the initial display position; v is the movement speed of the motion reference image data; Direction x is the lateral movement direction of the motion reference image data; Direction y is the longitudinal movement direction of the motion reference image data; Δt is the movement duration.

[0178] Step 512, displaying the motion reference image data based on the updated display position.

[0179] Exemplarily, the electronic device displays the motion reference image corresponding to the motion reference image data based on the updated display position.

[0180] In this embodiment, a motion reference image is displayed around the image to be displayed, and an updated display position of the motion reference image data is determined according to a set of linear accelerations corresponding to the image data to be displayed and an initial display position of the motion reference image data. The motion reference image data moves from the initial display position to the updated display position, reflecting the acceleration information of the operation of the vehicle. The human eye sees the operation reference image on the display screen move from the initial display position to the updated display position. The acceleration information input by the human eye is the acceleration information of the vehicle, which is the same as the acceleration information of the vehicle perceived by the user's vestibule, thereby avoiding motion sickness caused by sensory conflict and achieving the effect of further preventing motion sickness.

[0181] In an exemplary embodiment, Figure 7 As shown, based on the linear acceleration set, the lateral motion direction, longitudinal motion direction and motion speed of the motion reference image data are determined, including:

[0182] Step 702: Determine the total acceleration of the motion reference image data based on the first direction acceleration, the second direction acceleration and the third direction acceleration in the linear acceleration set.

[0183] Here, total acceleration refers to the total linear acceleration.

[0184] Exemplarily, the electronic device calculates the first direction acceleration, the second direction acceleration, and the third direction acceleration in the linear acceleration set to obtain the total acceleration of the motion reference image data.

[0185] In one exemplary embodiment, the total acceleration is as follows:

[0186]

[0187] Among them, a total is the total acceleration; (a x ,,a y ,a z ) is the linear acceleration set; ax , is the acceleration in the first direction; a y is the acceleration in the second direction; a z is the acceleration in the third direction.

[0188] Step 704: Obtain the lateral motion direction of the motion reference image data based on the ratio between the first direction acceleration and the total acceleration.

[0189] Exemplarily, the electronic device divides the first direction acceleration by the total acceleration to obtain the lateral movement direction of the motion reference image data.

[0190] Step 706: Obtain the longitudinal motion direction of the motion reference image data based on the ratio between the second direction acceleration and the total acceleration.

[0191] Exemplarily, the electronic device divides the second direction acceleration by the total acceleration to obtain the longitudinal motion direction of the motion reference image data.

[0192] Step 708: Determine the motion speed of the motion reference image data based on the total acceleration.

[0193] Exemplarily, the electronic device multiplies the total acceleration by the scale factor to obtain the motion speed of the motion reference image data. The scale factor refers to a value used to adjust the speed. It can be understood that the scale factor is used to make the motion speed of the motion reference image more reasonable on the display screen.

[0194] In an exemplary embodiment, the movement speed is as follows:

[0195] v=k×a total Formula (16)

[0196] Where, v is the speed of movement; a total is the total acceleration; k is the proportional factor.

[0197] In this embodiment, the lateral movement direction, longitudinal movement direction and movement speed of the motion reference image data are determined by the first direction acceleration, the second direction acceleration and the third direction acceleration in the linear acceleration set, providing basic data for the subsequent determination of the initial display position of the motion reference image data.

[0198] In an exemplary embodiment, a user of an electronic device may experience motion sickness while riding in a vehicle. There are three main mechanisms for motion sickness: the first is the sensory conflict theory, which is caused by the conflict between the motion information input by the eyes and the motion information perceived by the vestibule; the second is the nystagmus caused by the vestibular-ocular reflex (VOR). The schematic diagram of the connection between the vestibule and the eye is shown in FIG. Figure 8As shown; the third is the involuntary body movement that causes postural instability.

[0199] The flowchart of the image display method for preventing motion sickness in an electronic device is as follows Fig. 9 As shown, including:

[0200] The electronic device obtains a first angular acceleration set at a first moment through an accelerometer, obtains a first linear acceleration set at the first moment through a gyroscope, and obtains a second angular acceleration set corresponding to a second moment through an accelerometer after a preset time, and obtains a second linear acceleration set at the second moment through a gyroscope. The absolute value of the difference between the first angular acceleration and the second angular acceleration in the same direction in the first angular acceleration set and the second angular acceleration set is used as the angular acceleration difference value between the first angular acceleration and the second angular acceleration in the same direction; the electronic device uses the absolute value of the difference between the first linear acceleration and the second linear acceleration in the same direction in the first linear acceleration set and the second linear acceleration set as the linear acceleration difference value between the first linear acceleration and the second linear acceleration; the electronic device compares each angular acceleration difference value with an angular acceleration difference threshold value, and compares each linear acceleration difference value with a linear acceleration difference threshold value, and if at least one angular acceleration difference value is greater than the angular acceleration difference threshold value, or at least one linear acceleration difference value is greater than the linear acceleration difference threshold value, the anti-motion sickness mode is turned on.

[0201] When the anti-motion sickness mode is turned on, the electronic device reduces the display screen according to the image reduction ratio, and a motion reference image appears.

[0202] The front camera is turned on, and the eye movement information and head movement information corresponding to the next frame of image data to be displayed are obtained through the front camera. The eye movement information includes the gaze point position and the eye rotation angle, and the gaze point position is located in the first coordinate system with the upper left corner of the display screen as the origin; the head movement information includes the head position and head posture, and the head position is located in the second coordinate system with the front camera of the electronic device as the origin. The electronic device obtains the relative position relationship between the origin of the first coordinate system and the origin of the second coordinate system, adds the head position to the relative position relationship, and obtains the matching position corresponding to the head position in the first coordinate system.

[0203] The electronic device substitutes the matching position into formula (3) to obtain the straight-line distance between the eyeball and the display screen; multiplies the straight-line distance by the tangent value of the eyeball rotation angle in the first direction to obtain the eyeball movement first direction offset distance; multiplies the straight-line distance by the tangent value of the eyeball rotation angle in the second direction to obtain the eyeball movement second direction offset distance; and obtains the eyeball movement offset according to the eyeball movement first direction offset distance and the eyeball movement second direction offset distance.

[0204] The electronic device multiplies the matching third direction position in the matching position by the tangent value of the first direction rotation angle of the head in the head posture to obtain a first lateral offset distance; multiplies the matching third direction position by the negative of the straight tangent value of the second direction rotation angle of the head in the head posture to obtain a first longitudinal offset distance; multiplies the matching first direction position in the matching position by the negative of the straight tangent value of the third direction rotation angle of the head to obtain a second longitudinal offset distance; multiplies the matching second direction position in the matching position by the negative of the straight tangent value of the third direction rotation angle of the head to obtain a second longitudinal offset distance; obtains the first direction offset distance of the head movement based on the first lateral offset distance and the second lateral offset distance; obtains the second direction offset distance of the head movement based on the first longitudinal offset distance and the second longitudinal offset distance; obtains the head movement offset based on the first direction offset distance of the head movement and the second direction offset distance of the head movement.

[0205] The electronic device adds the eye movement offset to the head movement offset to obtain the gaze point offset. The image reduction ratio is obtained, and the gaze point translation amount is multiplied by the image reduction ratio to obtain the compensation offset corresponding to the image data to be displayed. For each pixel in the image data to be displayed, the electronic device obtains the original display position corresponding to the pixel, determines the reduced display position corresponding to the pixel based on the reduction ratio and the original display position corresponding to the pixel, and subtracts the compensation offset from the reduced display position corresponding to the pixel to obtain the target display position of the pixel.

[0206] The image to be displayed corresponding to the image data to be displayed is displayed based on the target display position.

[0207] In addition, when a motion reference image corresponding to the motion reference image data is displayed on a display screen, the electronic device obtains the linear acceleration set corresponding to the image data to be displayed and the initial display position of the motion reference image data through a gyroscope while obtaining the eye movement information and head movement information corresponding to the image data to be displayed.

[0208] Substitute the linear acceleration set into formula (15) to obtain the total acceleration of the motion reference image data; divide the first direction acceleration by the total acceleration to obtain the lateral motion direction of the motion reference image data; divide the second direction acceleration by the total acceleration to obtain the longitudinal motion direction of the motion reference image data; multiply the total acceleration by the scale factor to obtain the motion speed of the motion reference image data.

[0209] The electronic device calculates the product of the horizontal movement direction, the movement speed and the movement duration to obtain the horizontal running distance of the movement reference image data; calculates the product of the vertical movement direction, the movement speed and the movement duration to determine the vertical running distance of the movement reference image data; the electronic device adds the horizontal running distance to the first direction display position in the initial display position, and adds the vertical running distance to the second direction display position in the initial display position to obtain the updated display position of the movement reference image data.

[0210] The motion reference image corresponding to the motion reference image data is displayed based on the updated display position.

[0211] In the above-mentioned image data display method, when a user uses an electronic device while riding in a vehicle, the electronic device collects the user's eye movement information and head movement information, determines a gaze point offset based on the eye movement information and the head movement information, the gaze point offset represents the gaze point offset caused by the user's eye movement and head movement, the gaze point offset in the opposite direction represents the offset of the gaze point adjusted by the user's neural network, the electronic device determines a compensation offset corresponding to the image data to be displayed based on the gaze point offset and the image reduction ratio, the compensation offset represents the offset that needs to be adjusted for the image data to be displayed, determines a target display position of the image data to be displayed based on the compensation offset, and displays the image data to be displayed based on the target display position. It can be understood that the offset of the image data to be displayed on the display screen using the compensation offset is equivalent to the offset of the eye gaze point adjustment, even if the nervous system adjusts the position of the eye gaze point, the user's eyes are still in a focused state, avoiding nystagmus, thereby achieving the effect of preventing motion sickness. Moreover, a motion reference image is displayed around the image to be displayed, and an updated display position of the motion reference image data is determined according to a set of linear accelerations corresponding to the image data to be displayed and an initial display position of the motion reference image data. The motion reference image data moves from the initial display position to the updated display position, reflecting the acceleration information of the operation of the vehicle. The human eye sees the operation reference image on the display screen move from the initial display position to the updated display position. The acceleration information input by the human eye is the acceleration information of the vehicle, which is the same as the acceleration information of the vehicle perceived by the user's vestibule, thereby avoiding motion sickness caused by sensory conflict and achieving the effect of further preventing motion sickness.

[0212] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0213] Based on the same inventive concept, the embodiment of the present application also provides an image data display device for implementing the image data display method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more image data display device embodiments provided below can refer to the limitations on the image data display method above, and will not be repeated here.

[0214] In an exemplary embodiment, Fig.10 As shown, an image data display device is provided, comprising: an acquisition module 102, a gaze point offset determination module 104, a compensation offset determination module 106, a display position determination module 108 and a display module 110, wherein:

[0215] An acquisition module 102 is used to acquire eye movement information and head movement information corresponding to the image data to be displayed;

[0216] A gaze point offset determination module 104, configured to determine a gaze point offset based on the eye movement information and the head movement information;

[0217] A compensation offset determination module 106, configured to determine a compensation offset corresponding to the image data to be displayed based on the gaze point offset and the image reduction ratio;

[0218] A display position determination module 108, used to determine a target display position of the image data to be displayed based on the compensation offset;

[0219] The display module 110 is used to display the image data to be displayed based on the target display position.

[0220] In an exemplary embodiment, the gaze point offset determination module 104 is also used to: determine the eye movement offset based on the head position in the head movement information and the eye rotation angle in the eye movement information; determine the head movement offset based on the head position and the head posture in the head movement information; and obtain the gaze point offset based on the eye movement offset and the head movement offset.

[0221] In an exemplary embodiment, the gaze point offset determination module 104 is also used to: determine the matching position corresponding to the head position in the first coordinate system based on the relative position relationship between the origin of the first coordinate system and the origin of the second coordinate system; determine the eye movement offset based on the matching position and the eye rotation angle; determine the head movement offset based on the matching position and the head posture in the head movement information.

[0222] In an exemplary embodiment, the gaze point offset determination module 104 is also used to: determine the straight-line distance between the eyeball and the display screen based on the matching position; determine the offset distance of the eyeball movement in the first direction based on the straight-line distance and the eyeball rotation angle in the first direction; determine the offset distance of the eyeball movement in the second direction based on the straight-line distance and the eyeball rotation angle in the second direction; and obtain the eyeball movement offset based on the eyeball movement offset distance in the first direction and the eyeball movement offset distance in the second direction.

[0223] In an exemplary embodiment, the gaze point offset determination module 104 is also used to: determine a first lateral offset distance based on matching a third direction position and a first direction rotation angle of the head; determine a first longitudinal offset distance based on matching a third direction position and a second direction rotation angle of the head; determine a second longitudinal offset distance based on matching a first direction position and a third direction rotation angle of the head; determine a second lateral offset distance based on matching a second direction position and a third direction rotation angle of the head; obtain a first direction offset distance of the head movement based on the first lateral offset distance and the second lateral offset distance; obtain a second direction offset distance of the head movement based on the first longitudinal offset distance and the second longitudinal offset distance; obtain a head movement offset based on the first direction offset distance of the head movement and the second direction offset distance of the head movement.

[0224] In an exemplary embodiment, the display position determination module 108 is also used to: obtain the reduced display position of the pixel point for each pixel point in the image data to be displayed; the reduced display position refers to the display position of the image data to be displayed in the display screen after being reduced based on the image reduction ratio; obtain the target display position of the pixel point based on the difference between the reduced display position of the pixel point and the compensation offset.

[0225] In an exemplary embodiment, the image data display device also includes an activation module, which is used to: obtain a first angular acceleration set and a first linear acceleration set at a first moment, and a second angular acceleration set and a second linear acceleration set corresponding to a second moment; the first moment is before the second moment; for the first angular acceleration and the second angular acceleration in the same direction in the first angular acceleration set and the second angular acceleration set, determine an angular acceleration difference value between the first angular acceleration and the second angular acceleration; for the first linear acceleration and the second linear acceleration in the same direction in the first linear acceleration set and the second linear acceleration set, determine a linear acceleration difference value between the first linear acceleration and the second linear acceleration; when there is at least one angular acceleration difference value greater than an angular acceleration difference threshold, or at least one linear acceleration difference value greater than a linear acceleration difference threshold, activate the anti-motion sickness mode.

[0226] In an exemplary embodiment, the display position determination module 108 is also used to: when displaying motion reference image data, obtain a set of linear accelerations corresponding to the image data to be displayed, and an initial display position of the motion reference image data; determine the lateral motion direction, longitudinal motion direction and motion speed of the motion reference image data based on the linear acceleration set; determine the lateral running distance of the motion reference image data based on the lateral motion direction and the motion speed; determine the longitudinal running distance of the motion reference image data based on the longitudinal motion direction and the motion speed; determine an updated display position of the motion reference image data based on the initial display position, the lateral running distance and the longitudinal running distance; and display the motion reference image data based on the updated display position.

[0227] In an exemplary embodiment, the display position determination module 108 is also used to: determine the total acceleration of the motion reference image data based on the first direction acceleration, the second direction acceleration and the third direction acceleration in the linear acceleration set; obtain the lateral motion direction of the motion reference image data based on the ratio between the first direction acceleration and the total acceleration; obtain the longitudinal motion direction of the motion reference image data based on the ratio between the second direction acceleration and the total acceleration; and determine the motion speed of the motion reference image data based on the total acceleration.

[0228] Each module in the above-mentioned image data display device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in an electronic device in the form of hardware, or can be stored in a memory in an electronic device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0229] In an exemplary embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.11As shown. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and an external device. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, an image data display method is implemented. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the electronic device casing, or an external keyboard, touchpad or mouse.

[0230] Those skilled in the art will understand that Fig.11 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0231] In an exemplary embodiment, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0232] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0233] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0234] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0235] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0236] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0237] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for displaying image data, characterized in that: The method comprises: Obtaining eye movement information and head movement information corresponding to the image data to be displayed; Determining a gaze point offset based on the eye movement information and the head movement information; Determining a compensation offset corresponding to the image data to be displayed based on the gaze point offset and the image reduction ratio; Determining a target display position of the image data to be displayed based on the compensation offset; Based on the target display position, the image data to be displayed is displayed.

2. The method according to claim 1, characterized in that The determining of the gaze point offset based on the eye movement information and the head movement information comprises: Determining an eye movement offset based on the head position in the head movement information and the eye rotation angle in the eye movement information; Determining a head movement offset based on the head position and the head posture in the head movement information; A gaze point offset is obtained based on the eye movement offset and the head movement offset.

3. The method according to claim 2, characterized in that When the gaze point position in the eye movement information is located in a first coordinate system and the head position is located in a second coordinate system, determining the eye movement offset based on the head position in the head movement information and the eye rotation angle in the eye movement information includes: Determine a matching position corresponding to the head position in the first coordinate system based on a relative position relationship between the origin of the first coordinate system and the origin of the second coordinate system; determining an eye movement offset based on the matching position and the eye rotation angle; The determining of the head movement offset based on the head position and the head posture in the head movement information comprises: A head movement offset is determined based on the matching position and the head posture in the head movement information.

4. The method according to claim 3, characterized in that The eyeball rotation angle includes an eyeball first direction rotation angle and an eyeball second direction rotation angle; and determining the eyeball movement offset based on the matching position and the eyeball rotation angle includes: Based on the matching position, determining the straight-line distance between the eyeball and the display screen; Determine the offset distance of the eyeball movement in the first direction based on the straight-line distance and the eyeball rotation angle in the first direction; Determine the offset distance of the eye movement in the second direction based on the straight-line distance and the eye rotation angle in the second direction; Based on the eye movement first direction offset distance and the eye movement second direction offset distance, an eye movement offset is obtained.

5. The method according to claim 3, characterized in that: The head posture includes a first direction rotation angle of the head, a second direction rotation angle of the head, and a third direction rotation angle of the head; the matching position includes a matching first direction position, a matching second direction position, and a matching third direction position; and determining the head movement offset based on the matching position and the head posture in the head movement information includes: Determining a first lateral offset distance based on the matching third direction position and the first direction rotation angle of the head; Determining a first longitudinal offset distance based on the matching third direction position and the second direction rotation angle of the head; Determining a second longitudinal offset distance based on the matching first direction position and the third direction rotation angle of the head; Determining a second lateral offset distance based on the matching second direction position and the third direction rotation angle of the head; Based on the first lateral offset distance and the second lateral offset distance, obtaining an offset distance in a first direction of head movement; Based on the first longitudinal offset distance and the second longitudinal offset distance, obtaining an offset distance in a second direction of head movement; Based on the offset distance of the head movement in the first direction and the offset distance of the head movement in the second direction, a head movement offset is obtained.

6. The method according to claim 1, characterized in that The step of determining a target display position of the image data to be displayed based on the compensation offset comprises: For each pixel in the image data to be displayed, obtaining a reduced display position of the pixel; the reduced display position refers to a display position of the image data to be displayed on the display screen after the image data to be displayed is reduced based on the image reduction ratio; The target display position of the pixel is obtained based on the difference between the reduced display position of the pixel and the compensation offset.

7. The method according to claim 1, characterized in that Before acquiring the eye movement information and the head movement information corresponding to the image data to be displayed, the method further includes: Acquire a first angular acceleration set and a first linear acceleration set at a first moment, and a second angular acceleration set and a second linear acceleration set corresponding to a second moment, wherein the first moment is before the second moment; For a first angular acceleration and a second angular acceleration in the same direction in the first angular acceleration set and the second angular acceleration set, determine an angular acceleration difference value between the first angular acceleration and the second angular acceleration; For the first linear acceleration and the second linear acceleration in the same direction in the first linear acceleration set and the second linear acceleration set, determine a linear acceleration difference value between the first linear acceleration and the second linear acceleration; In a case where at least one of the angular acceleration difference values ​​is greater than the angular acceleration difference threshold, or at least one of the linear acceleration difference values ​​is greater than the linear acceleration difference threshold, the anti-motion sickness mode is turned on.

8. The method according to claim 1, characterized in that The method further comprises: In the case of displaying motion reference image data, obtaining a set of linear accelerations corresponding to the image data to be displayed and an initial display position of the motion reference image data; Determining the lateral motion direction, longitudinal motion direction and motion speed of the motion reference image data based on the linear acceleration set; Determining a lateral running distance of the motion reference image data based on the lateral motion direction and the motion speed; Determining a longitudinal running distance of the motion reference image data based on the longitudinal motion direction and the motion speed; Determining an updated display position of the motion reference image data based on the initial display position, the horizontal running distance, and the vertical running distance; The motion reference image data is displayed based on the updated display position.

9. The method according to claim 8, characterized in that The determining, based on the linear acceleration set, the lateral motion direction, the longitudinal motion direction and the motion speed of the motion reference image data comprises: Determining a total acceleration of the motion reference image data based on the first direction acceleration, the second direction acceleration, and the third direction acceleration in the linear acceleration set; Obtaining a lateral motion direction of the motion reference image data based on a ratio between the first direction acceleration and the total acceleration; Obtaining the longitudinal motion direction of the motion reference image data based on a ratio between the second direction acceleration and the total acceleration; Based on the total acceleration, a motion speed of the motion reference image data is determined.

10. An image data display device, characterized in that: The device comprises: An acquisition module, used for acquiring eye movement information and head movement information corresponding to the image data to be displayed; A gaze point offset determination module, used to determine a gaze point offset based on the eye movement information and the head movement information; A compensation offset determination module, used to determine the compensation offset corresponding to the image data to be displayed based on the gaze point offset and the image reduction ratio; A display position determination module, used to determine a target display position of the image data to be displayed based on the compensation offset; A display module is used to display the image data to be displayed based on the target display position.

11. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images