Method for enhancing visual effect of near-to-eye display equipment by enhancing perception difference

Abstract

The invention relates to the technical field of image data processing, in particular to a method for improving the visual effect of near-to-eye display equipment by enhancing perception difference. The method comprises the following steps: carrying out edge detection on a binocular image to be displayed by near-eye display equipment, and taking an area enclosed by an obtained edge curve as an initial detail area; traversing outwards layer by layer from the initial detail region; for each pixel point in the layer traversed each time, determining the detail degree of the pixel point according to the incidence relation between the gradient of the pixel point and the gradient of the edge curve; determining the pixel points of which the detail degrees accord with a preset condition as extensible pixel points; obtaining a detail area based on the initial detail area and each extensible pixel point; zooming the detail area according to a first zooming algorithm, and zooming the area outside the detail area according to a second zooming algorithm to obtain a zoomed binocular image; the loss degree of the first scaling algorithm is smaller than that of the second scaling algorithm. By adopting the method, the visual effect of the near-to-eye display equipment can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and particularly to a method for enhancing perceptual differences to improve the visual effect of a near-eye display device. Background Art

[0002] With the rapid development of technologies such as virtual reality (VR) and augmented reality (AR), near-eye display devices have been widely used. The near-eye display device reconstructs a virtual scene at a relatively short distance from the human eye through a microdisplay and a relay optical path, enabling the human eye to view the image on the microdisplay within a comfortable range and allowing the user to have an immersive experience.

[0003] Since the screen of the microdisplay in the near-eye display device is small, the displayed image often needs to be greatly reduced to adapt to the microdisplay. During the process of image reduction, the number of image pixel points decreases, resulting in obvious loss of details such as the edges of the image texture, which greatly affects the visual effect. Summary of the Invention

[0004] In order to solve the technical problem of poor visual effect of the near-eye display device, the purpose of the present invention is to provide a method for enhancing perceptual differences to improve the visual effect of the near-eye display device, and the specific technical solution adopted is as follows: The present invention provides a method for enhancing perceptual differences to improve the visual effect of a near-eye display device, and the method includes: Performing edge detection on the binocular images to be displayed by the near-eye display device, and taking the area enclosed by the obtained edge curves as the initial detail area; Traversing layer by layer outward from the initial detail area; For each pixel point in each layer traversed each time, determining the detail degree of the pixel point according to the correlation between the gradient of the pixel point and the gradient of the edge curve; Determining the pixel points whose detail degree meets the preset conditions as expandable pixel points; Obtaining a detail area based on the initial detail area and each of the expandable pixel points; Scaling the detail area according to a first scaling algorithm, and scaling the area outside the detail area according to a second scaling algorithm to obtain a scaled binocular image; the loss degree of the first scaling algorithm is smaller than that of the second scaling algorithm.

[0005] According to the method for enhancing perceptual differences to improve the visual effect of the near-eye display device provided by the present invention, the determining the detail degree of the pixel point according to the correlation between the gradient of the pixel point and the gradient of the edge curve includes: Determine the angle between the perpendicular line from the pixel point to the edge curve and the gradient direction of the pixel point; Determine the level of detail of the pixel point according to the gradient value of the pixel point, the average gradient value of the edge curve, and the angle.

[0006] According to the method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, the determining the level of detail of the pixel point according to the gradient value of the pixel point, the average gradient value of the edge curve, and the angle includes: Determine the ratio of the gradient value of the pixel point to the average gradient value of the edge curve to obtain a gradient ratio; Determine the level of detail of the pixel point according to the gradient ratio and the cosine value of the angle.

[0007] According to the method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, the determining the pixel points with the level of detail meeting the preset conditions as the pixel points to be expanded includes: Determine the pixel points with the level of detail greater than or equal to the preset level-of-detail threshold as the pixel points to be expanded.

[0008] According to the method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, the scaling the detail area according to the first scaling algorithm and scaling the area outside the detail area according to the second scaling algorithm to obtain the scaled binocular images includes: Determine the length ratio and width ratio between the microdisplay in the near-eye display device and the binocular images; If the length ratio and the width ratio are not equal, determine the smaller ratio of the length ratio and the width ratio as the scaling ratio; If the length ratio and the width ratio are equal, determine any one of the length ratio and the width ratio as the scaling ratio; Scale the detail area according to the first scaling algorithm in accordance with the scaling ratio, and scale the area outside the detail area according to the second scaling algorithm in accordance with the scaling ratio to obtain the scaled binocular images.

[0009] According to the method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, the binocular images include a left-eye image and a right-eye image; After the scaling the detail area according to the first scaling algorithm and scaling the area outside the detail area according to the second scaling algorithm to obtain the scaled binocular images, the method further includes: Match the detail areas in the scaled left-eye image and right-eye image; Determine whether the brightness difference within the matching detailed regions in the scaled left-eye image and right-eye image is within the range recognizable by the human eye; If so, reduce the brightness difference within the matching detailed regions in the scaled left-eye image and right-eye image; If not, enhance the brightness difference within the matching detailed regions in the scaled left-eye image and right-eye image.

[0010] According to the method for improving the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, the determining whether the brightness difference within the matching detailed regions in the scaled left-eye image and right-eye image is within the range recognizable by the human eye includes: Determine the maximum brightness difference within the matching detailed regions in the scaled left-eye image and right-eye image; If the maximum brightness difference is greater than or equal to the minimum visual difference of the human eye, determine that the brightness difference is within the range recognizable by the human eye; If the maximum brightness difference is less than the minimum visual difference of the human eye, determine that the brightness difference is not within the range recognizable by the human eye.

[0011] According to the method for improving the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, the enhancing the brightness difference within the matching detailed regions in the scaled left-eye image and right-eye image includes: For the matching detailed regions in the scaled left-eye image and right-eye image respectively, determine the quantization value of the detailed information degree of the detailed region according to the brightness values of each pixel point within the detailed region and the degree of detail; Increase the brightness of the detailed region with a larger quantization value of the detailed information degree and decrease the brightness of the detailed region with a smaller quantization value of the detailed information degree.

[0012] According to the method for improving the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, the determining the quantization value of the detailed information degree of the detailed region according to the brightness values of each pixel point within the detailed region and the degree of detail includes: According to the brightness values of each pixel point within the detailed region, determine the average brightness value and the brightness range of the detailed region; According to the degree of detail of each pixel point within the detailed region, determine the sum of the degrees of detail of the detailed region; According to the average brightness value, the brightness range, and the sum of the degrees of detail, determine the quantization value of the detailed information degree of the detailed region.

[0013] According to the method for enhancing the visual effect of a near-eye display device by realizing enhanced perceptual differences provided by the present invention, reducing the brightness difference within the matching detailed regions in the scaled left-eye image and right-eye image includes: Comparing the overall brightness magnitudes in the matching detailed regions in the scaled left-eye image and right-eye image; Reducing the brightness of the detailed region with a greater overall brightness and increasing the brightness of the detailed region with a smaller overall brightness until the brightness difference within the matching detailed regions is outside the range distinguishable by the human eye.

[0014] The present invention has the following beneficial effects: Edge detection is performed on the binocular images to be displayed by the near-eye display device, and the region enclosed by the obtained edge curves is used as the initial detailed region. Then, traversing layer by layer outward from the initial detailed region, for each pixel point in each layer traversed each time, according to the correlation between the gradient of the pixel point and the gradient of the edge curve, the detail level of the pixel point is determined, and the pixel points whose detail levels meet the preset conditions are determined as expandable pixel points. Based on the initial detailed region and each expandable pixel point, the detailed region is obtained, realizing the accurate determination of the detailed region with color changes, avoiding the problem of serious detail loss caused by only taking the high-gradient regions extracted by edge detection as the detailed regions and ignoring the regions with color changes around the edges. The accurately extracted detailed region is scaled according to the first scaling algorithm, and the regions outside the detailed region are scaled according to the second scaling algorithm. The loss degree of the first scaling algorithm is smaller than that of the second scaling algorithm, which can ensure less detail loss in the image during the scaling process, thereby improving the visual effect of the near-eye display device. Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0016] Figure 1 It is a schematic flowchart of a method for enhancing the visual effect of a near-eye display device by realizing enhanced perceptual differences provided by an embodiment of the present invention; Figure 2 It is an effect example diagram of images under different brightnesses provided by an embodiment of the present invention; Figure 3 It is a schematic overall flowchart of a method for enhancing the visual effect of a near-eye display device by realizing enhanced perceptual differences provided by an embodiment of the present invention. Detailed Embodiments

[0017] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following specifically describes, in conjunction with the accompanying drawings and preferred embodiments, a method for enhancing the visual effect of a near-eye display device by enhancing the perceptual difference, including its specific implementation manner, structure, features, and effects. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0019] The following specifically describes the specific solution of a method for enhancing the visual effect of a near-eye display device by enhancing the perceptual difference provided by the present invention in conjunction with the accompanying drawings.

[0020] Please refer to Figure 1 , which shows a schematic flowchart of a method for enhancing the visual effect of a near-eye display device by enhancing the perceptual difference provided by an embodiment of the present invention, including the following steps: Step 101: Perform edge detection on the binocular images to be displayed by the near-eye display device, and use the region enclosed by the obtained edge curves as the initial detail region.

[0021] Among them, a near-eye display device is a display device that reconstructs a virtual scene at a relatively short distance from the human eye through a microdisplay and a relay optical path, enabling the human eye to view the image on the microdisplay within a comfortable range and giving the user an immersive experience as if on the scene. The binocular images include two images, namely the left-eye image and the right-eye image, which respectively represent the picture materials to be displayed on the left and right independent microdisplays of the near-eye display device.

[0022] In one embodiment, first preprocess the binocular images, and then perform edge detection on the preprocessed binocular images, and use the region enclosed by the obtained edge curves as the initial detail region. The preprocessing can include grayscale processing and denoising processing.

[0023] Step 102: Traverse layer by layer outward from the initial detail region.

[0024] Specifically, first traverse to the first layer adjacent to the periphery of the initial detail region, then traverse to the layer adjacent outside the first layer as the second layer, then traverse to the layer adjacent outside the second layer as the third layer, and so on. Stop traversing until there are no expandable pixel points in the traversed layer.

[0025] Step 103: For each pixel point in each layer traversed each time, determine the level of detail of the pixel point according to the correlation between the gradient of the pixel point and the gradient of the edge curve.

[0026] In one embodiment, for each pixel point in each layer traversed each time, determine the level of detail of the pixel point according to the correlation between the magnitude of the gradient of the pixel point and the average gradient value of the edge curve, and the correlation between the direction of the gradient of the pixel point and the direction of the perpendicular line from the pixel point of the edge curve to the edge curve.

[0027] Step 104: Determine the pixel points whose level of detail meets the preset conditions as expandable pixel points.

[0028] Among them, the greater the level of detail of the pixel point, the more likely the pixel point is to be determined as an expandable pixel point, and the preset conditions can be set based on this principle.

[0029] In one embodiment, compare the level of detail of the pixel point with a preset level of detail threshold, and determine whether the pixel point is an expandable pixel point according to the comparison result. Among them, the preset level of detail threshold can be user-defined. For example, the preset level of detail threshold can be set to 0.65.

[0030] Step 105: Obtain the detail area based on the initial detail area and each expandable pixel point.

[0031] Step 106: Scale the detail area according to the first scaling algorithm, and scale the area outside the detail area according to the second scaling algorithm to obtain the scaled binocular image; the loss degree of the first scaling algorithm is smaller than that of the second scaling algorithm.

[0032] Among them, the loss degree of the first scaling algorithm is smaller than that of the second scaling algorithm, and the second scaling algorithm is faster to calculate and the scaled image occupies less space. For example: the first scaling algorithm can be bicubic interpolation. The second scaling algorithm can be nearest neighbor interpolation.

[0033] In one embodiment, determine the scaling ratio according to the length ratio and width ratio between the microdisplay in the near-eye display device and the binocular image, and then scale the detail area according to the first scaling algorithm in accordance with the scaling ratio, and scale the area outside the detail area according to the second scaling algorithm in accordance with the scaling ratio to obtain the scaled binocular image.

[0034] In one embodiment, after obtaining the scaled binocular images, the detailed regions in the scaled left-eye image and right-eye image are matched, and the brightness difference of the matched detailed regions in the scaled left-eye image and right-eye image is enhanced when they are not recognizable by the human eye (i.e., outside the recognizable range of the human eye), so as to obtain the binocular images with enhanced perception difference, and the binocular images with enhanced perception difference are displayed on the near-eye display device.

[0035] For the method for enhancing the visual effect of the near-eye display device by enhancing the perception difference mentioned above, edge detection is performed on the binocular images to be displayed on the near-eye display device, and the region enclosed by the obtained edge curves is used as the initial detailed region. Then, it traverses layer by layer outward from the initial detailed region. For each pixel point in each layer traversed each time, according to the correlation between the gradient of the pixel point and the gradient of the edge curve, the detail level of the pixel point is determined. The pixel points whose detail level meets the preset conditions are determined as expandable pixel points, and the detailed region is obtained based on the initial detailed region and each expandable pixel point. This realizes accurately determining the detailed region with color changes, avoiding the problem of serious detail loss caused by only taking the high-gradient region extracted by edge detection as the detailed region and ignoring the regions with color changes around the edge. The accurately extracted detailed region is scaled according to the first scaling algorithm, and the region outside the detailed region is scaled according to the second scaling algorithm. The loss degree of the first scaling algorithm is smaller than that of the second scaling algorithm, which can ensure less detail loss of the image during the scaling process, thereby improving the visual effect of the near-eye display device.

[0036] In one embodiment, determining the detail level of a pixel point according to the correlation between the gradient of the pixel point and the gradient of the edge curve includes: determining the included angle between the perpendicular line from the pixel point to the edge curve and the gradient direction of the pixel point; and determining the detail level of the pixel point according to the gradient value of the pixel point, the average gradient value of the edge curve, and the included angle.

[0037] It can be understood that the larger the gradient value of the pixel point, the greater the degree of color change at the position of the pixel point, and the greater the detail level of the pixel point. The smaller the average gradient value of the edge curve, the greater the degree of color change at the position of the pixel point compared with the color change degree of the edge curve, and the greater the detail level of the pixel point. The smaller the included angle between the perpendicular line from the pixel point to the edge curve and the gradient direction of the pixel point, the more consistent the color change trend at the position of the pixel point with the color change trend of the edge curve, and the greater the detail level of the pixel point. Therefore, in one embodiment, the detail level of the pixel point is positively correlated with the gradient value of the pixel point, negatively correlated with the average gradient value of the edge curve, and negatively correlated with the included angle.

[0038] In the above embodiments, by determining the angle between the perpendicular line from the pixel point to the edge curve and the gradient direction of the pixel point, and based on the gradient value of the pixel point, the average gradient value of the edge curve, and the angle, the level of detail of the pixel point can be accurately determined.

[0039] In one embodiment, determining the level of detail of a pixel point according to the gradient value of the pixel point, the average gradient value of the edge curve, and the angle includes: determining the ratio of the gradient value of the pixel point to the average gradient value of the edge curve to obtain a gradient ratio; and determining the level of detail of the pixel point according to the gradient ratio and the cosine value of the angle.

[0040] In one embodiment, based on the grayscale image of the binocular image, with the length and width of the pixel point as the unit length, an image rectangular coordinate system is established. Assuming that the coordinate positions of a total of pixel points are , where represents the index of the pixel point, .

[0041] In one embodiment, the level of detail of the pixel point is positively correlated with the gradient ratio and positively correlated with the cosine value of the angle.

[0042] In one embodiment, the level of detail of the pixel point is determined according to the product of the gradient ratio and the cosine value of the angle. The formula is as follows:

[0043] where represents the level of detail of the th pixel point. represents the gradient of the th pixel point. represents the gradient value of the th pixel point. represents the average gradient value of the edge curve. represents the th pixel point to the perpendicular line of the edge curve and the th pixel point between the gradient direction of the angle. represents the th pixel point to the perpendicular line of the edge curve and the th pixel point between the cosine value of the angle of the gradient direction.

[0044] It can be understood that the gradient ratio represents the degree of color change at the position of the pixel point compared to the degree of color change of the edge curve. The larger this value, the more the degree of color change at the pixel point position approximates or even exceeds the degree of color change at the edge curve position, and the greater the level of detail of the pixel point. The cosine value of the angle between the perpendicular line from the pixel point to the edge curve and the gradient direction of the pixel point The larger it is (i.e., the closer it is to 1), the smaller the angle between the perpendicular line from the pixel to the edge curve and the gradient direction of the pixel The smaller it is (i.e., the closer it is to 0), it indicates that the color change trend at the position of the pixel conforms more to the color change trend at the position of the edge curve, and then the detail degree of the pixel is greater. The detail degree of the pixel The greater it is, the greater the degree of color change at the position of the pixel and the more it conforms to the change trend in the detail area, and the more it should be expanded into the detail area.

[0045] In the above embodiments, the ratio of the gradient value of the pixel to the average gradient value of the edge curve is determined to obtain a gradient ratio. According to the gradient ratio and the cosine value of the angle, the detail degree of the pixel can be accurately determined.

[0046] In one embodiment, determining the pixels with a detail degree meeting a preset condition as expandable pixels includes: determining the pixels with a detail degree greater than or equal to a preset detail degree threshold as expandable pixels.

[0047] Among them, the preset detail degree threshold can be user-defined. For example, the preset detail degree threshold can be set to 0.65.

[0048] In one embodiment, if the detail degree of the pixel is less than the preset detail degree threshold, no operation is performed on the pixel.

[0049] In the above embodiments, determining the pixels with a detail degree greater than or equal to the preset detail degree threshold as expandable pixels can accurately determine the expandable pixels from each traversed layer.

[0050] In one embodiment, scaling the detail area according to a first scaling algorithm and scaling the area outside the detail area according to a second scaling algorithm to obtain a scaled binocular image includes: determining the length ratio and width ratio between the microdisplay in the near-eye display device and the binocular image; if the length ratio and the width ratio are not equal, determining the smaller ratio of the length ratio and the width ratio as the scaling ratio; if the length ratio and the width ratio are equal, determining any one of the length ratio and the width ratio as the scaling ratio; scaling the detail area according to the first scaling algorithm in accordance with the scaling ratio, and scaling the area outside the detail area according to the second scaling algorithm in accordance with the scaling ratio to obtain a scaled binocular image.

[0051] Among them, the length ratio is the ratio between the number of pixels in the length direction of the microdisplay in the near-eye display device and the number of pixels in the length direction of the binocular image. The width ratio is the ratio between the number of pixels in the width direction of the microdisplay in the near-eye display device and the number of pixels in the width direction of the binocular image.

[0052] For example, assume that the number of pixels in the length and width of the binocular image are \(n\) and \(m\) respectively, and the number of pixels in the length and width of the microdisplay in the near-eye display device are \(N\) and \(M\) respectively. Then the length ratio is , and the width ratio is . If the length ratio is equal to the width ratio, that is , then either the length ratio or the width ratio is determined as the scaling ratio, that is, the scaling ratio . If the length ratio is not equal to the width ratio, that is , then the smaller ratio of the length ratio and the width ratio is determined as the scaling ratio, that is, the scaling ratio .

[0053] In the above embodiments, if the length ratio is equal to the width ratio, then either the length ratio or the width ratio is determined as the scaling ratio; if the length ratio is not equal to the width ratio, then the smaller ratio of the length ratio and the width ratio is determined as the scaling ratio, which can ensure the maximum utilization rate of the microdisplay as much as possible.

[0054] In one embodiment, the binocular image includes a left-eye image and a right-eye image; after scaling the detail area according to the first scaling algorithm and scaling the area outside the detail area according to the second scaling algorithm to obtain the scaled binocular image, the method further includes: matching the detail areas in the scaled left-eye image and right-eye image; determining whether the brightness difference in the matching detail areas in the scaled left-eye image and right-eye image is within the range recognizable by the human eye; if so, reducing the brightness difference in the matching detail areas in the scaled left-eye image and right-eye image; if not, enhancing the brightness difference in the matching detail areas in the scaled left-eye image and right-eye image.

[0055] In one embodiment, corner detection is performed on the left-eye image and the right-eye image to obtain the corner information in the left-eye image and the right-eye image. For example, the SIFT corner detection algorithm can be used for corner detection. Then, based on the corner information in the left-eye image and the right-eye image, the detail areas in the left-eye image and the right-eye image are matched (paired) so that the detail areas with consistent relative positions and consistent internal corner information are matched together.

[0056] It can be understood that the image brightness also has a certain impact on the information contained in the image. Figure 2 From left to right in Figure 2It can be seen that details in the image are more obvious in brighter conditions, while object contours in the image are more obvious in darker conditions. In a near-eye display device, independent micro-displays are provided for each of the user's two eyes. The images displayed by the two micro-displays simulate binocular vision and are not exactly the same. By enhancing the binocular brightness difference perception of the same object within the brightness range imperceptible to the human eye, the user can synthesize different information obtained from both eyes to obtain a more stereoscopic and realistic usage experience.

[0057] Therefore, in the above embodiment, the detail regions in the scaled left-eye image and right-eye image are matched; it is determined whether the brightness difference within the matched detail regions in the scaled left-eye image and right-eye image is within the range recognizable by the human eye. If so, the brightness difference within the matched detail regions in the scaled left-eye image and right-eye image is reduced; if not, the brightness difference within the matched detail regions in the scaled left-eye image and right-eye image is enhanced. This realizes enhancing the brightness difference within the matched detail regions in the scaled left-eye image and right-eye image when it is outside the range recognizable by the human eye (i.e., outside the range recognizable by the human eye), increases the amount of information in the binocular images, improves the sense of three-dimensionality and realism, and thus improves the visual effect of the near-eye display device.

[0058] In one embodiment, determining whether the brightness difference within the matched detail regions in the scaled left-eye image and right-eye image is within the range recognizable by the human eye includes: determining the maximum brightness difference within the matched detail regions in the scaled left-eye image and right-eye image; if the maximum brightness difference is greater than or equal to the minimum visual difference of the human eye, it is determined that the brightness difference is within the range recognizable by the human eye; if the maximum brightness difference is less than the minimum visual difference of the human eye, it is determined that the brightness difference is not within the range recognizable by the human eye.

[0059] Among them, the minimum visual difference of the human eye is the minimum brightness difference that can be recognized by the human eye determined according to Weber's Law.

[0060] In one embodiment, for each detail region respectively, the difference between the maximum brightness within the detail region in the scaled right-eye image and the minimum brightness within the detail region in the scaled left-eye image is determined to obtain a first difference, and the difference between the maximum brightness within the detail region in the scaled left-eye image and the minimum brightness within the detail region in the scaled right-eye image is determined to obtain a second difference. The maximum value is selected from the first difference and the second difference as the maximum brightness difference of the detail region.

[0061] For example: Let the maximum brightness and minimum brightness in a certain detail region in the left-eye image be and , respectively, and the maximum brightness and minimum brightness in the matched detail region in the right-eye image be and , the minimum visible parallax of the human eye is , then the maximum brightness difference of this detail area is . If the maximum brightness difference is greater than or equal to the minimum visible parallax of the human eye, that is , it is determined that the brightness difference is within the recognizable range of the human eye. It is necessary to reduce the brightness difference so that the human eye cannot recognize the brightness difference to avoid affecting the display effect; if the maximum brightness difference is less than the minimum visible parallax of the human eye, that is , it is determined that the brightness difference is not within the recognizable range of the human eye, that is, the human eye cannot recognize the brightness difference, and the brightness difference can be enhanced as much as possible within to increase the amount of information in the binocular images, enhancing the stereoscopic effect and realism.

[0062] In the above embodiments, the maximum brightness difference in the matching detail areas of the scaled left-eye image and right-eye image is determined. If the maximum brightness difference is greater than or equal to the minimum visible parallax of the human eye, it is determined that the brightness difference is within the recognizable range of the human eye. If the maximum brightness difference is less than the minimum visible parallax of the human eye, it is determined that the brightness difference is not within the recognizable range of the human eye, achieving accurate judgment of whether to enhance or reduce the brightness difference according to Weber's law.

[0063] In one embodiment, enhancing the brightness difference in the matching detail areas of the scaled left-eye image and right-eye image includes: respectively for the matching detail areas in the scaled left-eye image and right-eye image, determining the quantization value of the detail information degree of the detail area according to the brightness values and detail degrees of each pixel point in the detail area; increasing the brightness of the detail area with a larger quantization value of the detail information degree and decreasing the brightness of the detail area with a smaller quantization value of the detail information degree.

[0064] Specifically, according to the brightness values and detail degrees of each pixel point in the detail area of the left-eye image, determine the quantization value of the detail information degree of this detail area in the left-eye image. According to the brightness values and detail degrees of each pixel point in the matching detail area of the right-eye image, determine the quantization value of the detail information degree of this detail area in the right-eye image. Increase the brightness of the detail area with a larger quantization value of the detail information degree in the left-eye image and right-eye image, and decrease the brightness of the detail area with a smaller quantization value of the detail information degree.

[0065] In one embodiment, the overall brightness level and brightness stability degree in the detail area can be determined according to the brightness values of each pixel point in the detail area, and the overall detail degree in the detail area can be determined according to the detail degrees of each pixel point in the detail area. According to the overall brightness level, brightness stability degree and overall detail degree in the detail area, determine the quantization value of the detail information degree of the detail area. The quantization value of the detail information degree of the detail area is positively correlated with the overall brightness level, brightness stability degree and overall detail degree.

[0066] In one embodiment, the brightness of the detail region with a larger quantization value of the detail information degree can be gradually increased in a preset step, and the brightness of the detail region with a smaller quantization value of the detail information degree can be gradually decreased in a preset step until the maximum brightness difference between the two detail regions is equal to the minimum visual difference of the human eye, and then the increase or decrease is stopped. For example, the preset step can be set to 1.

[0067] For example, assume the minimum visual difference of the human eye , the maximum brightness of the detail region in the left-eye image is 8, the minimum brightness of the corresponding detail region in the right-eye image is 7, and the maximum brightness difference between the two is 1. Therefore, the maximum brightness difference is less than the minimum visual difference of the human eye , and the brightness difference needs to be enhanced. Assume the preset step is 1, then increase the brightness of the detail region in the left-eye image with a higher brightness by 1, and decrease the brightness of the corresponding detail region in the right-eye image with a lower brightness by 1. After adjustment, the brightness of the two becomes 9 and 6 respectively, and the maximum brightness difference is 3, still less than ∆l. Continue to adjust, and the brightness of the two is obtained as 10 and 5 respectively. At this time, the maximum brightness difference is 5, and the maximum brightness difference is equal to the minimum visual difference ∆l of the human eye, and the adjustment is stopped.

[0068] In the above embodiment, for the matching detail regions in the scaled left-eye image and right-eye image respectively, according to the brightness values and detail degrees of the pixel points in the detail regions, the quantization value of the detail information degree of the detail region is determined. The brightness of the detail region with a larger quantization value of the detail information degree is increased, and the brightness of the detail region with a smaller quantization value of the detail information degree is decreased. The information situation included in the detail region is fully considered, and the brightness difference between the matching detail regions in the scaled left-eye image and right-eye image can be accurately enhanced, thereby increasing the amount of information in the binocular image, enhancing the stereoscopic effect and realism, improving the visual effect of the near-eye display device, and avoiding the problem of poor visual effect caused by insufficient consideration in the way of directly increasing the brightness of the detail region with a larger brightness in the left-eye image and right-eye image and decreasing the brightness of the detail region with a smaller brightness.

[0069] In one embodiment, determining the quantization value of the detail information degree of the detail region according to the brightness values and detail degrees of the pixel points in the detail region includes: determining the average brightness value and brightness range of the detail region according to the brightness values of the pixel points in the detail region; determining the sum of the detail degrees of the detail region according to the detail degrees of the pixel points in the detail region; and determining the quantization value of the detail information degree of the detail region according to the average brightness value, brightness range and sum of the detail degrees.

[0070] Among them, the brightness range is the difference between the maximum brightness and the minimum brightness in the detail region.

[0071] In one embodiment, the quantization value of the detail information degree of the detail area is positively correlated with the average brightness, negatively correlated with the brightness range, and positively correlated with the sum of the detail degrees.

[0072] In one embodiment, the ratio of the average brightness to the brightness range of the detail area can be calculated, and then the ratio is multiplied by the sum of the detail degrees to obtain the quantization value of the detail information degree of the detail area. Taking the calculation of the quantization value of the detail information degree of the detail area in the left-eye image as an example, the formula is as follows:

[0073] Where, represents the quantization value of the detail information degree of the detail area in the left-eye image. represents the average brightness of the detail area in the left-eye image. represents the brightness range of the detail area in the left-eye image. represents the detail degree of the th pixel point in the detail area of the left-eye image. represents the number of pixel points in the detail area of the left-eye image. represents the sum of the detail degrees of the detail area in the left-eye image.

[0074] It can be understood that the average brightness of the detail area can measure the overall brightness level within the detail area, and the brightness range of the detail area can inversely measure the brightness stability degree of the detail area. The larger the brightness range, the worse the brightness stability degree. Therefore, the ratio of the average brightness to the brightness range of the detail area is larger, indicating that the brightness within the detail area is larger and the brightness is stable. can measure the overall detail degree of the detail area. In the above formula, the brightness performance value is used as the overall detail degree for correction to obtain the quantization value of the detail information degree of the detail area. The larger the quantization value of the detail information degree, the higher the information degree within the detail area of the left-eye image.

[0075] Similarly, the quantization value of the detail information degree of the detail area in the right-eye image can be calculated. Compare the quantization value of the detail information degree of the detail area in the left-eye image and the quantization value of the detail information degree of the detail area in the right-eye image, increase the brightness of the detail area in the image corresponding to the larger value, and decrease the brightness of the detail area in the image corresponding to the smaller value.

[0076] In the above embodiments, according to the average brightness, brightness range, and the sum of the degrees of detail of the detail area, the quantization value of the detail information degree of the detail area can be accurately determined. The information situation contained in the detail area is fully considered, and the brightness difference of the matching detail areas in the scaled left-eye image and right-eye image can be accurately enhanced, thereby increasing the amount of information in the binocular image, enhancing the three-dimensional sense and realism, improving the visual effect of the near-eye display device, and avoiding the problem of poor visual effect caused by insufficient consideration in the way of directly increasing the brightness of the detail area with greater brightness and decreasing the brightness of the detail area with smaller brightness in the left-eye image and right-eye image.

[0077] In one embodiment, reducing the brightness difference in the matching detail areas of the scaled left-eye image and right-eye image includes: comparing the overall brightness of the matching detail areas in the scaled left-eye image and right-eye image; reducing the brightness of the detail area with greater overall brightness and increasing the brightness of the detail area with smaller overall brightness until the brightness difference in the matching detail areas is not within the range recognizable by the human eye.

[0078] In one embodiment, if the value of the maximum brightness difference is the difference between the maximum brightness in this detail area of the scaled right-eye image and the minimum brightness in this detail area of the scaled left-eye image, that is , then the overall brightness of the detail area in the right-eye image is greater than the overall brightness of the corresponding detail area in the left-eye image; conversely, if the value of the maximum brightness difference is the difference between the maximum brightness in this detail area of the scaled left-eye image and the minimum brightness in this detail area of the scaled right-eye image, that is , then the overall brightness of the detail area in the left-eye image is greater than the overall brightness of the corresponding detail area in the right-eye image. Reduce the brightness of the detail area with greater overall brightness and increase the brightness of the detail area with smaller overall brightness until the brightness difference in the matching detail areas is not within the range recognizable by the human eye, that is .

[0079] In the above embodiments, reducing the brightness of the detail area with greater overall brightness and increasing the brightness of the detail area with smaller overall brightness in the scaled left-eye image and right-eye image until the brightness difference in the matching detail areas is not within the range recognizable by the human eye can prevent the human eye from recognizing the brightness difference between the left-eye image and the right-eye image, which affects the visual effect.

[0080] Combining the above embodiments, as Figure 3 shown, is the overall flowchart of the method for enhancing the perceptual difference to improve the visual effect of the near-eye display device provided by the present invention, including the following steps: Step 301, obtain the binocular image captured by the binocular camera.

[0081] Step 302: Preprocess the binocular images and obtain the initial detail regions through edge detection.

[0082] Step 303: Expand the detail regions in combination with the color change conditions of the pixel points around the initial detail regions, and scale the binocular images according to the detail regions.

[0083] Step 304: Match the same detail regions in the binocular images and enhance the reasonable perceptual differences between them.

[0084] Step 305: Use the scaled and enhanced binocular images for near-eye display devices.

[0085] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0086] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application.

[0087] It should be noted that: The above sequence of the embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0088] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other, and the key point of each embodiment is to illustrate the differences from other embodiments.

Claims

1. A method for enhancing the visual effect of a near-eye display device by enhancing perceptual differences, characterized in that, The method includes: Performing edge detection on the binocular images to be displayed by the near-eye display device, and taking the area enclosed by the obtained edge curves as the initial detail area; Traversing layer by layer outward from the initial detail area; For each pixel point in each layer traversed each time, determining the detail level of the pixel point according to the correlation between the gradient of the pixel point and the gradient of the edge curve; Determining the pixel points whose detail level meets the preset conditions as expandable pixel points; Obtaining a detail area based on the initial detail area and each of the expandable pixel points; Scaling the detail area according to a first scaling algorithm, and scaling the area outside the detail area according to a second scaling algorithm to obtain a scaled binocular image; the loss degree of the first scaling algorithm is smaller than that of the second scaling algorithm.

2. The method for enhancing the visual effect of a near-eye display device by realizing the enhanced perception difference according to claim 1, wherein, The determining the detail level of the pixel point according to the correlation between the gradient of the pixel point and the gradient of the edge curve includes: Determining the angle between the perpendicular line from the pixel point to the edge curve and the gradient direction of the pixel point; Determining the detail level of the pixel point according to the gradient value of the pixel point, the average gradient value of the edge curve, and the angle.

3. The method for enhancing the visual effect of a near-eye display device by realizing the perception difference according to claim 2, wherein The determining the detail level of the pixel point according to the gradient value of the pixel point, the average gradient value of the edge curve, and the angle includes: Determining the ratio of the gradient value of the pixel point to the average gradient value of the edge curve to obtain a gradient ratio; Determining the detail level of the pixel point according to the gradient ratio and the cosine value of the angle.

4. The method for enhancing the visual effect of a near-eye display device by enhancing perceptual differences according to claim 1, characterized in that, The determining the pixel points whose detail level meets the preset conditions as expandable pixel points includes: Determining the pixel points whose detail level is greater than or equal to a preset detail level threshold as expandable pixel points.

5. The method for enhancing the visual effect of a near-eye display device by realizing the perception difference according to claim 1, wherein, The scaling the detail area according to a first scaling algorithm, and scaling the area outside the detail area according to a second scaling algorithm to obtain a scaled binocular image includes: Determining the length ratio and width ratio between the microdisplay in the near-eye display device and the binocular image; If the length ratio and the width ratio are not equal, then determining the smaller ratio of the length ratio and the width ratio as the scaling ratio; If the length ratio and the width ratio are equal, then determining any one of the length ratio and the width ratio as the scaling ratio; Scaling the detail area according to the first scaling algorithm in accordance with the scaling ratio, and scaling the area outside the detail area according to the second scaling algorithm in accordance with the scaling ratio to obtain a scaled binocular image.

6. The method for enhancing the visual effect of a near-eye display device by realizing the perception difference according to any one of claims 1 to 5, characterized in that, The binocular images include a left-eye image and a right-eye image; After the scaling the detail area according to a first scaling algorithm, and scaling the area outside the detail area according to a second scaling algorithm to obtain a scaled binocular image, the method further includes: Matching the detail areas in the scaled left-eye image and right-eye image; Judging whether the brightness difference in the matched detail areas in the scaled left-eye image and right-eye image is within the range recognizable by the human eye; If so, reduce the brightness difference within the matched detail regions in the scaled left-eye image and right-eye image; If not, enhance the brightness difference within the matched detail regions in the scaled left-eye image and right-eye image.

7. The method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference according to claim 6, wherein The determination of whether the brightness difference within the matched detail regions in the scaled left-eye image and right-eye image is within the range recognizable by the human eye includes: Determine the maximum brightness difference within the matched detail regions in the scaled left-eye image and right-eye image; If the maximum brightness difference is greater than or equal to the minimum visual difference of the human eye, determine that the brightness difference is within the range recognizable by the human eye; If the maximum brightness difference is less than the minimum visual difference of the human eye, determine that the brightness difference is not within the range recognizable by the human eye.

8. The method for enhancing the visual effect of a near-eye display device by realizing the perception difference according to claim 6, characterized in that, The enhancement of the brightness difference within the matched detail regions in the scaled left-eye image and right-eye image includes: For the matched detail regions in the scaled left-eye image and right-eye image respectively, determine the quantization value of the detail information degree of the detail region according to the brightness values of each pixel point within the detail region and the degree of detail; Increase the brightness of the detail region with a larger quantization value of the detail information degree, and decrease the brightness of the detail region with a smaller quantization value of the detail information degree.

9. The method for enhancing the visual effect of a near-eye display device by realizing the perception difference according to claim 8, wherein, The determination of the quantization value of the detail information degree of the detail region according to the brightness values of each pixel point within the detail region and the degree of detail includes: Determine the average brightness and brightness range of the detail region according to the brightness values of each pixel point within the detail region; Determine the sum of the degrees of detail of the detail region according to the degree of detail of each pixel point within the detail region; Determine the quantization value of the detail information degree of the detail region according to the average brightness, the brightness range, and the sum of the degrees of detail.

10. The method for enhancing the visual effect of a near-eye display device by realizing the enhanced perception difference according to claim 8, wherein, The reduction of the brightness difference within the matched detail regions in the scaled left-eye image and right-eye image includes: Compare the overall brightness magnitudes of the matched detail regions in the scaled left-eye image and right-eye image; Reduce the brightness of the detail region with a larger overall brightness, and increase the brightness of the detail region with a smaller overall brightness until the brightness difference within the matched detail regions is not within the range recognizable by the human eye.

Images