A display control method and system for HMD device

Abstract

The present invention is applicable to the field of virtual reality technology, and in particular to a display control method and system for HMD equipment, the method comprising: obtaining video data to be displayed, decoding the video data to be displayed, and obtaining continuous picture frames; performing pixel position statistics on continuous picture frames, determining the picture motion speed, and judging whether to perform anti-dizziness adjustment; when performing anti-dizziness adjustment, performing picture dynamic area recognition, and determining the attention heat of each position in the picture; determining the picture adjustment area based on the attention heat, adjusting the pixels in the picture adjustment area, generating anti-dizziness replacement frames, and replacing the corresponding pictures in the continuous picture frames. The present invention takes the area where the user's attention is located as the center, constructs multiple picture adjustment areas, and by fixing the picture adjustment area, helps users reduce dizziness, improves user experience, and minimizes the impact of the picture adjustment area on immersion.

Description

Technical Field

[0001] The present invention belongs to the field of virtual reality technology, and in particular relates to a display control method and system for an HMD device. Background Art

[0002] HMD (Head-Mounted Display) is a display device worn on the head, mainly used in virtual reality (VR), augmented reality (AR) and mixed reality (MR) fields. It provides users with an immersive audio-visual experience through high-resolution display screens, advanced optical systems and multiple sensors, making users feel as if they are in a virtual environment. HMD devices are widely used in many fields such as gaming and entertainment, education and training, industrial design, and medical health, bringing users unprecedented ways of interaction and experience.

[0003] When using current HMD devices, when the image changes at a high speed, it is easy to cause dizziness, affecting the user experience. Summary of the invention

[0004] The present invention aims to provide a display control method for an HMD device, aiming to solve the problem that when the screen changes at a high speed during the use of the current HMD device, it is easy to cause dizziness to people, affecting the user experience.

[0005] The present invention is implemented as follows: a display control method for an HMD device, the method comprising:

[0006] Obtaining video data to be displayed, decoding the video data to be displayed, and obtaining continuous picture frames;

[0007] Perform pixel position statistics on continuous picture frames, determine the picture motion speed based on the statistical results, and determine whether to perform anti-dizziness adjustment according to the picture motion speed;

[0008] When performing anti-dizziness adjustment, the dynamic area of ​​the picture is identified for the continuous picture frames to determine the attention intensity of each position in the picture;

[0009] Based on the attention heat, the picture adjustment area is determined, the pixels in the picture adjustment area are adjusted, the anti-dizziness replacement frame is generated, and the corresponding picture in the continuous picture frames is replaced.

[0010] Preferably, the step of performing pixel position statistics on continuous picture frames, determining picture motion speed based on the statistical results, and judging whether to perform anti-dizziness adjustment according to the picture motion speed specifically includes:

[0011] Counting adjacent picture frames, dividing the picture into strip-shaped areas to obtain a plurality of strip-shaped pixel areas, sampling the strip-shaped pixel areas to obtain a sampled pixel set;

[0012] Comparing sampled pixel sets corresponding to adjacent picture frames, determining content matching relationships within adjacent picture frames, and determining pixel displacement amounts based on the content matching relationships;

[0013] The picture movement speed is calculated based on the pixel displacement, and the picture movement speed is compared with a preset reference value to determine whether anti-dizziness adjustment is performed.

[0014] Preferably, when performing the anti-dizziness adjustment, the step of identifying the dynamic area of ​​the continuous picture frames and determining the attention intensity of each position in the picture specifically includes:

[0015] Performing grayscale processing on adjacent picture frames to obtain a first grayscale image and a second grayscale image, performing pixel-by-pixel recognition on the first grayscale image and the second grayscale image, and counting the grayscale of each pixel;

[0016] Construct a sliding selection, traverse the entire frame through the sliding selection, and count the grayscale change rate corresponding to all pixels in the sliding selection in real time. The attention heat is represented by the grayscale change rate;

[0017] The pixel areas corresponding to the sliding selection at each moment are sorted based on the grayscale change rate, and the position with the highest grayscale change rate is taken as the position with the highest attention heat.

[0018] Preferably, the steps of determining the picture adjustment area based on the attention heat, adjusting the pixels in the picture adjustment area, generating an anti-sickness replacement frame, and replacing the corresponding picture in the continuous picture frames include:

[0019] A circular area is constructed with the position with the highest attention heat as the center, a preset number of adjacent picture frames are selected, and pixel information statistics are performed on pixels outside the circular area to obtain pixel statistics results;

[0020] Determine the overlapped area based on the pixel statistics result, extract the overlapped area, count the gray value differences of adjacent pixels in the overlapped area, and select the position with the lowest contrast as the image adjustment area;

[0021] The colors of the picture adjustment area are enhanced and uniformed, and anti-dizziness replacement frames are generated to replace the corresponding pictures in the continuous picture frames.

[0022] Preferably, the number of the picture adjustment areas is at least four groups, and the relative positions between the picture adjustment areas are fixed.

[0023] Another object of the present invention is to provide a display control system for an HMD device, the system comprising:

[0024] A data decoding module, used for acquiring video data to be displayed, decoding the video data to be displayed, and obtaining continuous picture frames;

[0025] A picture speed calculation module is used to count the pixel positions of continuous picture frames, determine the picture motion speed based on the statistical results, and determine whether to perform anti-dizziness adjustment according to the picture motion speed;

[0026] The attention heat calculation module is used to identify the dynamic area of ​​the continuous picture frames and determine the attention heat of each position in the picture when performing anti-dizziness adjustment;

[0027] The picture replacement module is used to determine the picture adjustment area based on the attention heat, adjust the pixels in the picture adjustment area, generate an anti-dizziness replacement frame, and replace the corresponding picture in the continuous picture frames.

[0028] Preferably, the picture speed calculation module includes:

[0029] A pixel sampling unit is used to count adjacent picture frames, divide the picture into strip-shaped areas to obtain a plurality of strip-shaped pixel areas, and sample the strip-shaped pixel areas to obtain a sampled pixel set;

[0030] A displacement statistics unit is used to compare sampled pixel sets corresponding to adjacent picture frames, determine content matching relationships within adjacent picture frames, and determine pixel displacements based on the content matching relationships;

[0031] The adjustment determination unit is used to calculate the picture motion speed based on the pixel displacement, compare the picture motion speed with a preset reference value, and determine whether to perform anti-dizziness adjustment.

[0032] Preferably, the attention heat calculation module includes:

[0033] A grayscale statistics unit is used to perform grayscale processing on adjacent picture frames to obtain a first grayscale image and a second grayscale image, perform pixel-by-pixel recognition on the first grayscale image and the second grayscale image, and count the grayscale of each pixel;

[0034] The change rate calculation unit is used to construct a sliding selection area, traverse the entire picture frame through the sliding selection area, and count the grayscale change rates corresponding to all pixels in the sliding selection area in real time. The attention heat is represented by the grayscale change rate;

[0035] The attention recognition unit is used to sort the pixel areas corresponding to the sliding selection area at each moment based on the grayscale change rate, and the position with the highest grayscale change rate is taken as the position with the highest attention heat.

[0036] Preferably, the picture replacement module includes:

[0037] A pixel information statistics unit is used to construct a circular area with the position with the highest attention heat as the center, select a preset number of adjacent picture frames, and perform pixel information statistics on pixels outside the circular area to obtain pixel statistics results;

[0038] An adjustment area selection unit is used to determine the overlap area based on the pixel statistics result, extract the overlap area, count the gray value difference of adjacent pixels in the overlap area, and select the position with the lowest contrast as the picture adjustment area;

[0039] The picture replacement unit is used to enhance and equalize the color of the picture adjustment area, generate anti-dizziness replacement frames, and replace corresponding pictures in continuous picture frames.

[0040] Preferably, the number of the picture adjustment areas is at least four groups, and the relative positions between the picture adjustment areas are fixed.

[0041] A display control method for an HMD device provided by the present invention can calculate the moving speed of the picture content according to the change of the picture by analyzing continuous pictures. When the moving speed is greater than a preset value, multiple picture adjustment areas are constructed with the user's attention area as the center. By fixing the picture adjustment area, the user's dizziness is helped to be reduced, the user experience is improved, and the influence of the picture adjustment area on the immersion is minimized. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flow chart of a display control method for an HMD device provided by an embodiment of the present invention;

[0043] Figure 2 A flowchart of the steps of performing pixel position statistics on continuous picture frames, determining picture motion speed based on the statistical results, and determining whether to perform anti-dizziness adjustment according to the picture motion speed provided by an embodiment of the present invention;

[0044] Figure 3 A flowchart of the steps of identifying the dynamic area of ​​a picture on continuous picture frames and determining the attention heat of each position in the picture when performing anti-dizziness adjustment provided by an embodiment of the present invention;

[0045] Figure 4 A flowchart of the steps of determining a picture adjustment area based on attention heat, adjusting pixels in the picture adjustment area, generating an anti-dizziness replacement frame, and replacing corresponding pictures in continuous picture frames provided in an embodiment of the present invention;

[0046] Figure 5An architecture diagram of a display control system for an HMD device provided by an embodiment of the present invention;

[0047] Figure 6 A structural diagram of a picture speed calculation module provided by an embodiment of the present invention;

[0048] Figure 7 An architecture diagram of an attention heat calculation module provided by an embodiment of the present invention;

[0049] Figure 8 This is an architecture diagram of a picture replacement module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0051] like Figure 1 FIG. 1 is a flow chart of a display control method for an HMD device provided by an embodiment of the present invention, wherein the method comprises:

[0052] S100, obtaining video data to be displayed, decoding the video data to be displayed, and obtaining continuous picture frames.

[0053] In this step, the video data to be displayed is obtained. When the HMD device is used, the video data to be displayed is decoded and processed. After decoding, the video data to be displayed is processed into continuous picture frames. The continuous picture frames include multiple continuous picture frames. The above continuous picture frames can be directly used for playback.

[0054] S200, performing pixel position statistics on continuous picture frames, determining picture motion speed based on the statistical results, and determining whether to perform anti-dizziness adjustment according to the picture motion speed.

[0055] In this step, pixel position statistics are performed on continuous picture frames. The video data to be displayed is unprocessed. When a user with 3D sickness watches it directly, it may cause discomfort to the user. Therefore, the video data to be displayed may be preprocessed before playing it, that is, the continuous picture frames are processed, and the change speed of the current picture is evaluated by comparing adjacent picture frames. Specifically, the picture frames are divided into regions, and the pixel positions in the same region are compared to determine the matching relationship between pixels. The picture motion speed is calculated based on the positional relationship between pixel groups with matching relationships. If the picture motion speed exceeds the preset value, it is determined that the picture changes too much and is likely to cause dizziness to the user, and anti-dizziness adjustments are performed on these picture frames.

[0056] S300, when performing anti-dizziness adjustment, the dynamic area of ​​the picture is identified for the continuous picture frames to determine the attention intensity of each position in the picture.

[0057] In this step, when performing anti-dizziness adjustment, dynamic area recognition is performed on continuous picture frames. When using an HMD device, an important reason for dizziness is that the movement state felt by the user's body does not match the movement state felt by the eyes. In the present invention, an adjustment area fixed relative to the user's focus point is constructed in the picture. By forming a relatively fixed point in the user's observation picture, the user's body and visual sense are balanced, and multiple adjacent picture frames are analyzed. According to the degree of change of the picture content, the area of ​​interest to the user is determined, that is, the attention heat of each position in the picture is analyzed. The higher the attention heat of the area, the greater the probability that the user will notice the area.

[0058] S400, determining a picture adjustment area based on the attention heat, adjusting pixels in the picture adjustment area, generating an anti-dizziness replacement frame, and replacing a corresponding picture in a continuous picture frame.

[0059] In this step, the picture adjustment area is determined based on the attention heat, and the area with the highest attention heat is selected. This position is regarded as the area where the user's attention is focused, that is, the user is looking at this area. Therefore, the outer area excluding this area is selected as the picture adjustment area to avoid affecting the user's viewing experience. The picture adjustment area is constructed around the user's field of vision. Through adjustment, the picture change in the picture adjustment area is greatly reduced, so that multiple fixed reference areas are formed in the picture. When the user uses the HMD device, a fixed reference point will be formed in the field of vision, which promotes the balance of the user's physical and visual senses. After adjusting the picture adjustment area, the original picture frame is replaced with the adjusted picture frame to obtain a new continuous picture frame. The obtained new continuous picture frame is played, which can relieve dizziness.

[0060] like Figure 2 As shown, as a preferred embodiment of the present invention, the steps of performing pixel position statistics on continuous picture frames, determining the picture motion speed based on the statistical results, and judging whether to perform anti-dizziness adjustment according to the picture motion speed specifically include:

[0061] S201, counting adjacent picture frames, dividing the picture into strip-shaped areas to obtain a plurality of strip-shaped pixel areas, sampling the strip-shaped pixel areas to obtain a sampled pixel set.

[0062] In this step, statistics are taken for adjacent picture frames. When the statistics are taken, the picture is first divided. Specifically, the picture is divided in two ways. The first way is to divide it in a rightward direction. Each time a column of pixels in the picture is selected, a vertical strip area is obtained. For example, if the picture resolution is 1000 pixels * 1000 pixels, 1000 strip areas with a resolution of 1 pixel * 1000 pixels will be obtained. Of course, the width of the strip area can be adjusted as needed and can be set to N pixels. Then the resolution of the corresponding strip area is N pixels * 1000 pixels. When sampling the strip pixel area, a pixel sampling area of ​​a preset range is selected each time along the long side direction of the strip pixel area, such as N * 100, or 100 * N. Each time sampling is performed, one pixel is offset. Each strip pixel area corresponds to multiple pixel sampling areas to obtain a sampled pixel set.

[0063] S202, comparing sampled pixel sets corresponding to adjacent picture frames, determining content matching relationships within adjacent picture frames, and determining pixel displacement amounts according to the content matching relationships.

[0064] In this step, the sampling pixel sets corresponding to adjacent picture frames are compared. The two adjacent picture frames are defined as picture frame A and picture frame B, and the strip pixel areas corresponding to the two are strip pixel areas a and b, respectively. The sampling pixel sets corresponding to the strip pixel areas a and b are Q and E, respectively. The corresponding pixel sampling areas in the Q sampling pixel set and the E sampling pixel set are compared one by one, that is, a pixel sampling area in the Q sampling pixel set is retrieved each time and compared with the pixel sampling areas in all other E sampling pixel sets. The number of identical pixels contained in the two (the difference in pixel information is considered to be the same pixel within a preset range) is counted, and the proportion of identical pixels is used as the similarity to evaluate the similarity relationship. The two groups of pixel sampling areas with the highest similarity are considered to be the same area, and the pixel offset of each identical pixel in the two adjacent picture frames is calculated. The pixel offset is characterized by the number of pixels between a pair of identical pixels. The average offset of all identical pixels is calculated to obtain the pixel displacement.

[0065] S203, calculating the picture motion speed based on the pixel displacement, comparing the picture motion speed with a preset reference value, and determining whether to perform anti-dizziness adjustment.

[0066] In this step, the picture movement speed is calculated based on the pixel displacement, and the picture movement speed is determined according to the ratio of pixel offset to time, such as 120 pixels / milliseconds. Whether to perform anti-dizziness adjustment is determined according to a preset reference value. If the set reference value is 100 pixels / milliseconds, then when the picture movement speed is 120 pixels / milliseconds, it is determined that the current picture is changing too fast and anti-dizziness adjustment is required.

[0067] like Figure 3 As shown, as a preferred embodiment of the present invention, when performing anti-dizziness adjustment, the step of performing dynamic area recognition on continuous picture frames and determining the attention heat of each position in the picture specifically includes:

[0068] S301, grayscale processing is performed on adjacent picture frames to obtain a first grayscale image and a second grayscale image, pixel-by-pixel recognition is performed on the first grayscale image and the second grayscale image, and the grayscale value of each pixel is counted.

[0069] In this step, grayscale processing is performed on two or more consecutive picture frames. The purpose of grayscale processing is to reduce the amount of data contained in the image to increase the data processing speed. After grayscale processing, taking two picture frames as an example, a first grayscale image and a second grayscale image are obtained, and the grayscale values ​​of all pixels in the image are identified.

[0070] S302, construct a sliding selection, traverse the entire picture frame through the sliding selection, and count the grayscale change rates corresponding to all pixels in the sliding selection in real time. The attention heat is represented by the grayscale change rate.

[0071] In this step, a sliding selection is constructed, and the size of the sliding selection is a preset value, such as a 50*50 rectangular area. The entire picture frame is traversed by the sliding selection, and two or more 50*50 pixel matrices are obtained each time. The two pixel matrices are respectively derived from two or more adjacent picture frames. The average grayscale value B of the pixels at the same position on the pixel matrix is ​​counted, that is, the sum of the grayscale values ​​of the pixel in different pixel matrices is calculated, and then the average value is calculated. Subsequently, the difference between the grayscale value of the pixel in different pixel matrices and the average grayscale value is calculated to obtain the grayscale difference Mi, and the grayscale change rate of a single pixel is , where n is the number of pixel matrices, i is the serial number of the pixel matrix, and the grayscale change rate of the pixel matrix is ​​the average value of the grayscale change rates of all single pixels, which is used to characterize the attention heat.

[0072] S303, sorting the pixel areas corresponding to the sliding selection area at each moment based on the grayscale change rate, and taking the position with the highest grayscale change rate as the position with the highest attention heat.

[0073] In this step, the pixel areas corresponding to the sliding selection at each moment are sorted based on the grayscale change rate. Each pixel matrix corresponds to an attention heat, and the area corresponding to the pixel matrix with the highest attention heat is selected as the area where the user is looking at.

[0074] like Figure 4As a preferred embodiment of the present invention, the steps of determining the picture adjustment area based on the attention heat, adjusting the pixels in the picture adjustment area, generating an anti-sickness replacement frame, and replacing the corresponding picture in the continuous picture frames include:

[0075] S401, constructing a circular area with the position with the highest attention heat as the center, selecting a preset number of adjacent picture frames, and performing pixel information statistics on pixels outside the circular area to obtain pixel statistics results.

[0076] In this step, a circular area is constructed with the position with the highest attention as the center. The radius of the circular area is a preset value. For example, if the radius is defined as 500 pixels, a circular area with a diameter of 1000 pixels is determined. This area is the area where the user's attention is located. A preset number of adjacent frames are selected, such as 5 consecutive adjacent frames each time. The pixels outside the circular area in each frame are extracted, and pixel information statistics are performed to determine the single-pixel grayscale change rate of each pixel.

[0077] S402, determining the overlapped area based on the pixel statistics result, extracting the overlapped area, performing statistics on the grayscale value differences of adjacent pixels in the overlapped area, and selecting the position with the lowest contrast as the picture adjustment area.

[0078] In this step, the overlapping area is determined based on the pixel statistics results, and the pixels are screened according to the preset grayscale change rate to screen out all pixels whose single-pixel grayscale change rate is lower than the preset grayscale change rate. From the connected domain formed by all the screened pixels, multiple areas of preset sizes are constructed, which are the overlapping areas. The overlapping area can be a circular range with a diameter of 100. The grayscale value differences of adjacent pixels in the overlapping area are counted, and the overlapping area with the lowest contrast and brightness higher than the preset value is selected as the picture adjustment area.

[0079] S403, enhancing and homogenizing the color of the picture adjustment area, generating an anti-dizziness replacement frame, and replacing the corresponding picture in the continuous picture frames.

[0080] In this step, the color of the picture adjustment area is enhanced and uniformed to further reduce the contrast between the pixels in the picture adjustment area so that the pixel colors in the picture adjustment area are closer. The color of the picture adjustment area is enhanced so that the color difference between the picture adjustment area and the nearby pixels is increased, so that the user's peripheral vision can feel the existence of the picture adjustment area without affecting the user's attention range. An anti-dizziness replacement frame is generated to replace the corresponding picture in the continuous picture frames.

[0081] like Figure 5As shown, a display control system for an HMD device provided by an embodiment of the present invention includes:

[0082] The data decoding module 100 is used to obtain the video data to be displayed, and decode the video data to be displayed to obtain continuous picture frames.

[0083] In the present system, the data decoding module 100 obtains the video data to be displayed. When the HMD device is used, the video data to be displayed is decoded and processed. After decoding, the video data to be displayed is processed into continuous picture frames. The continuous picture frames include multiple continuous picture frames. The continuous picture frames can be directly used for playback.

[0084] The picture speed calculation module 200 is used to perform pixel position statistics on continuous picture frames, determine the picture motion speed based on the statistical results, and determine whether to perform anti-dizziness adjustment according to the picture motion speed.

[0085] In the present system, the picture speed calculation module 20 performs pixel position statistics on continuous picture frames. The video data to be displayed is unprocessed. When a user with 3D dizziness watches it directly, it may cause discomfort to the user. Therefore, the video data to be displayed may be preprocessed before being played, that is, the continuous picture frames are processed, and the change speed of the current picture is evaluated by comparing adjacent picture frames. Specifically, the picture frames are divided into regions, and the pixel positions in the same region are compared to determine the matching relationship between pixels. The picture motion speed is calculated based on the positional relationship between pixel groups with matching relationships. If the picture motion speed exceeds the preset value, it is determined that the picture changes too much and is likely to cause dizziness to the user, and anti-dizziness adjustments are performed on these picture frames.

[0086] The attention heat calculation module 300 is used to identify the dynamic area of ​​the continuous picture frames when performing anti-dizziness adjustment, and determine the attention heat of each position in the picture.

[0087] In this system, when the attention heat calculation module 300 performs anti-dizziness adjustment, it identifies the dynamic area of ​​the picture for continuous picture frames. When using the HMD device, an important reason for the dizziness is that the movement state felt by the user's body does not match the movement state felt by the eyes. In the present invention, an adjustment area fixed relative to the user's focus point is constructed in the picture. By forming a relatively fixed point in the user's observation picture, the user's body and visual sense are balanced, and multiple adjacent picture frames are analyzed. According to the degree of change of the picture content, the area of ​​interest to the user is determined, that is, the attention heat of each position in the picture is analyzed. The higher the attention heat of the area, the greater the probability that the user will notice the area.

[0088] The picture replacement module 400 is used to determine the picture adjustment area based on the attention heat, adjust the pixels in the picture adjustment area, generate an anti-dizziness replacement frame, and replace the corresponding picture in the continuous picture frames.

[0089] In the present system, the picture replacement module 400 determines the picture adjustment area based on the attention heat, and selects the area with the highest attention heat. This position is regarded as the area where the user's attention is focused, that is, the user is looking at this area. Therefore, the outer area excluding this area is selected as the picture adjustment area to avoid affecting the user's viewing experience. The picture adjustment area is constructed around the user's field of vision. Through adjustment, the picture change in the picture adjustment area is greatly reduced, so that multiple fixed reference areas are formed in the picture. When the user uses the HMD device, a fixed reference point will be formed in the field of vision, which promotes the balance of the user's physical and visual senses. After adjusting the picture adjustment area, the original picture frame is replaced by the adjusted picture frame to obtain a new continuous picture frame. The obtained new continuous picture frame is played, which can relieve dizziness.

[0090] like Figure 6 As shown, as a preferred embodiment of the present invention, the picture speed calculation module 200 includes:

[0091] The pixel sampling unit 201 is used to perform statistics on adjacent picture frames, divide the picture into strip-shaped areas to obtain a plurality of strip-shaped pixel areas, and sample the strip-shaped pixel areas to obtain a sampled pixel set.

[0092] In this module, the pixel sampling unit 201 performs statistics on adjacent picture frames. When performing statistics, the picture is first divided. Specifically, it is divided in two ways. The first way is to divide it in a rightward manner. Each time a column of pixels in the picture is selected, a vertical strip area is obtained. For example, if the picture resolution is 1000 pixels * 1000 pixels, 1000 strip areas with a resolution of 1 pixel * 1000 pixels will be obtained. Of course, the width of the strip area can be adjusted as needed and can be set to N pixels. Then the resolution of the corresponding strip area is N pixels * 1000 pixels. When sampling the strip pixel area, a pixel sampling area of ​​a preset range is selected each time along the long side direction of the strip pixel area, such as N * 100, or 100 * N. Each time sampling is performed, one pixel is offset. Each strip pixel area corresponds to multiple pixel sampling areas to obtain a sampled pixel set.

[0093] The displacement statistics unit 202 is used to compare the sampled pixel sets corresponding to the adjacent picture frames, determine the content matching relationship in the adjacent picture frames, and determine the pixel displacement according to the content matching relationship.

[0094] In this module, the displacement statistics unit 202 compares the sampled pixel sets corresponding to adjacent picture frames. The two adjacent picture frames are defined as picture frame A and picture frame B, and the strip pixel areas corresponding to the two are strip pixel areas a and b, respectively. The sampled pixel sets corresponding to the strip pixel areas a and b are Q and E, respectively. The corresponding pixel sampling areas in the Q sampled pixel set and the E sampled pixel set are compared one by one, that is, a pixel sampling area in the Q sampled pixel set is retrieved each time, and it is compared with the pixel sampling areas in all other E sampled pixel sets, and the number of identical pixels contained in the two (the difference in pixel information is considered to be the same pixel within a preset range) is counted, and the proportion of identical pixels is used as the similarity to evaluate the similarity relationship, and the two groups of pixel sampling areas with the highest similarity are considered to be the same area, and the pixel offset of each identical pixel in the two adjacent picture frames is calculated. The pixel offset is characterized by the number of pixels between a pair of identical pixels, and the average offset of all identical pixels is calculated to obtain the pixel displacement.

[0095] The adjustment determination unit 203 is used to calculate the picture motion speed based on the pixel displacement, compare the picture motion speed with a preset reference value, and determine whether to perform anti-dizziness adjustment.

[0096] In this module, the adjustment judgment unit 203 calculates the picture motion speed based on the pixel displacement, and determines the picture motion speed according to the ratio of the pixel offset to the time, such as 120 pixels / milliseconds, and determines whether to perform anti-dizziness adjustment according to a preset reference value. If the set reference value is 100 pixels / milliseconds, then when the picture motion speed is 120 pixels / milliseconds, it is determined that the current picture changes too fast and anti-dizziness adjustment is required.

[0097] like Figure 7 As shown, as a preferred embodiment of the present invention, the attention heat calculation module 300 includes:

[0098] The grayscale statistics unit 301 is used to perform grayscale processing on adjacent picture frames to obtain a first grayscale image and a second grayscale image, perform pixel-by-pixel recognition on the first grayscale image and the second grayscale image, and count the grayscale of each pixel.

[0099] In this module, the grayscale statistics unit 301 performs grayscale processing on two or more consecutive picture frames. The purpose of grayscale processing is to reduce the amount of data contained in the image to improve the data processing speed. After grayscale processing, taking two picture frames as an example, a first grayscale image and a second grayscale image are obtained, and the grayscale values ​​of all pixels in the image are identified.

[0100] The change rate calculation unit 302 is used to construct a sliding selection area, traverse the entire picture frame through the sliding selection area, and count the grayscale change rates corresponding to all pixels in the sliding selection area in real time. The attention heat is represented by the grayscale change rate.

[0101] In this module, the change rate calculation unit 302 constructs a sliding selection area, the size of which is a preset value, such as a 50*50 rectangular area. The entire picture frame is traversed by the sliding selection area, and two or more 50*50 pixel matrices are obtained each time. The two pixel matrices are respectively derived from two or more adjacent picture frames. The average grayscale value B of the pixels at the same position on the pixel matrix is ​​counted, that is, the sum of the grayscale values ​​of the pixel in different pixel matrices is calculated, and then the average value is calculated. Subsequently, the difference between the grayscale value of the pixel in different pixel matrices and the average grayscale value is calculated to obtain the grayscale difference Mi, and the grayscale change rate of a single pixel is Among them, n is the number of pixel matrices, i is the serial number of the pixel matrix, and the grayscale change rate of the pixel matrix is ​​the average value of the grayscale change rates of all single pixels, which is used to characterize the attention heat.

[0102] The attention recognition unit 303 is used to sort the pixel areas corresponding to the sliding selection area at each moment based on the grayscale change rate, and take the position with the highest grayscale change rate as the position with the highest attention heat.

[0103] In this module, the attention recognition unit 303 sorts the pixel areas corresponding to the sliding selection at each moment based on the grayscale change rate. Each pixel matrix corresponds to an attention heat, and the area corresponding to the pixel matrix with the highest attention heat is selected as the area where the user is looking at.

[0104] like Figure 8 As shown, as a preferred embodiment of the present invention, the picture replacement module 400 includes:

[0105] The pixel information statistics unit 401 is used to construct a circular area with the position with the highest attention heat as the center, select a preset number of adjacent picture frames, and perform pixel information statistics on pixels outside the circular area to obtain pixel statistics results.

[0106] In this module, the pixel information statistics unit 401 constructs a circular area with the position with the highest attention as the center. The radius of the circular area is a preset value. For example, if the radius is defined as 500 pixels, a circular area with a diameter of 1000 pixels is determined. This area is the area where the user's attention is located. A preset number of adjacent picture frames are selected, such as 5 consecutive adjacent picture frames each time, and the pixels outside the circular area in each picture frame are extracted, and pixel information statistics are performed to determine the single-pixel grayscale change rate of each pixel.

[0107] The adjustment area selection unit 402 is used to determine the overlap area based on the pixel statistics result, extract the overlap area, count the gray value difference of adjacent pixels in the overlap area, and select the position with the lowest contrast as the picture adjustment area.

[0108] In this module, the adjustment area selection unit 402 determines the overlapping area based on the pixel statistics result, screens the pixels according to the preset grayscale change rate, screens out all pixels whose single-pixel grayscale change rate is lower than the preset grayscale change rate, and constructs a plurality of areas of preset sizes, namely the overlapping areas, from the connected domain formed by all the screened pixels. The overlapping area can be a circular range with a diameter of 100. The grayscale value difference of adjacent pixels in the overlapping area is counted, and the overlapping area with the lowest contrast and brightness higher than the preset value is selected as the picture adjustment area.

[0109] The picture replacement unit 403 is used to enhance and equalize the color of the picture adjustment area, generate an anti-dizziness replacement frame, and replace the corresponding picture in the continuous picture frames.

[0110] In this module, the picture replacement unit 403 enhances and homogenizes the color of the picture adjustment area, further reduces the contrast between the pixels in the picture adjustment area, makes the pixel colors in the picture adjustment area closer, and enhances the color of the picture adjustment area, so that the picture adjustment area has a color difference with nearby pixels, so that the user's peripheral vision can feel the existence of the picture adjustment area, but does not affect the user's attention range, generates an anti-dizziness replacement frame, and replaces the corresponding picture in the continuous picture frames.

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A display control method for an HMD device, characterized in that: The method comprises: Obtaining video data to be displayed, decoding the video data to be displayed, and obtaining continuous picture frames; Perform pixel position statistics on continuous picture frames, determine the picture motion speed based on the statistical results, and determine whether to perform anti-dizziness adjustment according to the picture motion speed; When performing anti-dizziness adjustment, the dynamic area of ​​the picture is identified for the continuous picture frames to determine the attention intensity of each position in the picture; Based on the attention heat, the picture adjustment area is determined, the pixels in the picture adjustment area are adjusted, the anti-dizziness replacement frame is generated, and the corresponding picture in the continuous picture frames is replaced.

2. The display control method for an HMD device according to claim 1, characterized in that: The step of performing pixel position statistics on continuous picture frames, determining picture motion speed based on the statistical results, and judging whether to perform anti-dizziness adjustment according to the picture motion speed specifically includes: Counting adjacent picture frames, dividing the picture into strip-shaped areas to obtain a plurality of strip-shaped pixel areas, sampling the strip-shaped pixel areas to obtain a sampled pixel set; Comparing sampled pixel sets corresponding to adjacent picture frames, determining content matching relationships within adjacent picture frames, and determining pixel displacement amounts based on the content matching relationships; The picture movement speed is calculated based on the pixel displacement, and the picture movement speed is compared with a preset reference value to determine whether anti-dizziness adjustment is performed.

3. The display control method for an HMD device according to claim 1, characterized in that: When performing the anti-dizziness adjustment, the step of identifying the dynamic area of ​​the continuous picture frames and determining the attention heat of each position in the picture specifically includes: Performing grayscale processing on adjacent picture frames to obtain a first grayscale image and a second grayscale image, performing pixel-by-pixel recognition on the first grayscale image and the second grayscale image, and counting the grayscale of each pixel; Construct a sliding selection, traverse the entire frame through the sliding selection, and count the grayscale change rate corresponding to all pixels in the sliding selection in real time. The attention heat is represented by the grayscale change rate; The pixel areas corresponding to the sliding selection at each moment are sorted based on the grayscale change rate, and the position with the highest grayscale change rate is taken as the position with the highest attention heat.

4. The display control method for an HMD device according to claim 1, characterized in that: The steps of determining the picture adjustment area based on the attention heat, adjusting the pixels in the picture adjustment area, generating an anti-dizziness replacement frame, and replacing the corresponding picture in the continuous picture frames include: A circular area is constructed with the position with the highest attention heat as the center, a preset number of adjacent picture frames are selected, and pixel information statistics are performed on pixels outside the circular area to obtain pixel statistics results; Determine the overlapped area based on the pixel statistics result, extract the overlapped area, count the gray value differences of adjacent pixels in the overlapped area, and select the position with the lowest contrast as the image adjustment area; The colors of the picture adjustment area are enhanced and uniformed, and anti-dizziness replacement frames are generated to replace the corresponding pictures in the continuous picture frames.

5. The display control method for an HMD device according to claim 1, characterized in that: The number of the picture adjustment areas is at least four groups, and the relative positions between the picture adjustment areas are fixed.

6. A display control system for an HMD device, characterized in that: The system comprises: A data decoding module, used for acquiring video data to be displayed, decoding the video data to be displayed, and obtaining continuous picture frames; A picture speed calculation module is used to count the pixel positions of continuous picture frames, determine the picture motion speed based on the statistical results, and determine whether to perform anti-dizziness adjustment according to the picture motion speed; The attention heat calculation module is used to identify the dynamic area of ​​the continuous picture frames and determine the attention heat of each position in the picture when performing anti-dizziness adjustment; The picture replacement module is used to determine the picture adjustment area based on the attention heat, adjust the pixels in the picture adjustment area, generate an anti-dizziness replacement frame, and replace the corresponding picture in the continuous picture frames.

7. The display control system for an HMD device according to claim 6, characterized in that: The picture speed calculation module comprises: A pixel sampling unit is used to count adjacent picture frames, divide the picture into strip-shaped areas to obtain a plurality of strip-shaped pixel areas, and sample the strip-shaped pixel areas to obtain a sampled pixel set; A displacement statistics unit is used to compare sampled pixel sets corresponding to adjacent picture frames, determine content matching relationships within adjacent picture frames, and determine pixel displacements based on the content matching relationships; The adjustment determination unit is used to calculate the picture motion speed based on the pixel displacement, compare the picture motion speed with a preset reference value, and determine whether to perform anti-dizziness adjustment.

8. The display control system for an HMD device according to claim 6, characterized in that: The attention heat calculation module includes: A grayscale statistics unit is used to perform grayscale processing on adjacent picture frames to obtain a first grayscale image and a second grayscale image, perform pixel-by-pixel recognition on the first grayscale image and the second grayscale image, and count the grayscale of each pixel; The change rate calculation unit is used to construct a sliding selection area, traverse the entire picture frame through the sliding selection area, and count the grayscale change rates corresponding to all pixels in the sliding selection area in real time. The attention heat is represented by the grayscale change rate; The attention recognition unit is used to sort the pixel areas corresponding to the sliding selection area at each moment based on the grayscale change rate, and the position with the highest grayscale change rate is taken as the position with the highest attention heat.

9. The display control system for an HMD device according to claim 6, characterized in that: The picture replacement module comprises: A pixel information statistics unit is used to construct a circular area with the position with the highest attention heat as the center, select a preset number of adjacent picture frames, and perform pixel information statistics on pixels outside the circular area to obtain pixel statistics results; An adjustment area selection unit is used to determine the overlap area based on the pixel statistics result, extract the overlap area, count the gray value difference of adjacent pixels in the overlap area, and select the position with the lowest contrast as the picture adjustment area; The picture replacement unit is used to enhance and equalize the color of the picture adjustment area, generate anti-dizziness replacement frames, and replace corresponding pictures in continuous picture frames.

10. The display control system for an HMD device according to claim 6, characterized in that: The number of the picture adjustment areas is at least four groups, and the relative positions between the picture adjustment areas are fixed.

Images