A method for enhancing perceptual differences to improve the visual effects of near-eye display devices

Abstract

The present invention relates to the field of image data processing technology, and in particular to a method for enhancing perceptual differences to achieve visual effect improvement of near-eye display devices. The method comprises: performing edge detection on a binocular image to be displayed by a near-eye display device, and using the area enclosed by the obtained edge curve as an initial detail area; traversing outward layer by layer from the initial detail area; for each pixel point in the layer traversed each time, determining the detail level of the pixel point based on the correlation between the gradient of the pixel point and the gradient of the edge curve; determining the pixel point whose detail level meets a preset condition as an expandable pixel point; obtaining a detail area based on the initial detail area and each expandable pixel point; scaling the detail area according to a first scaling algorithm, and scaling the area outside the detail area according to a second scaling algorithm to obtain a scaled binocular image; the first scaling algorithm has a smaller loss than the second scaling algorithm. The method can improve the visual effect of a near-eye display device.

Description

Technical Field

[0001] The present invention relates to the field of image data processing technology, and in particular to a method for enhancing perceptual differences to improve the visual effects of near-eye display devices. Background Art

[0002] With the rapid development of technologies such as virtual reality (VR) and augmented reality (AR), near-eye display devices have gained widespread application. These devices use microdisplays and relay optical paths to reconstruct virtual scenes at a relatively close distance to the human eye, allowing the human eye to view images on the microdisplay within a comfortable range, providing users with an immersive experience.

[0003] Since the microdisplay screen in near-eye display devices is relatively small, the displayed image often needs to be significantly reduced to adapt to the microdisplay. During the image reduction process, the number of image pixels decreases, resulting in obvious loss of details such as image texture edges, greatly affecting the visual effect. Summary of the Invention

[0004] In order to solve the technical problem of poor visual effects of near-eye display devices, the purpose of the present invention is to provide a method for enhancing the perceived difference to improve the visual effects of near-eye display devices. The technical solutions adopted are as follows:

[0005] The present invention provides a method for enhancing perceptual differences to improve the visual effects of a near-eye display device, the method comprising:

[0006] Performing edge detection on the binocular image to be displayed by the near-eye display device, and using the area enclosed by the obtained edge curve as the initial detail area;

[0007] Traversing outward layer by layer from the initial detail area;

[0008] For each pixel point in the layer traversed each time, determining the detail level of the pixel point according to the correlation relationship between the gradient of the pixel point and the gradient of the edge curve;

[0009] Determining pixels whose detail levels meet preset conditions as expandable pixels;

[0010] Obtaining a detail area based on the initial detail area and each of the expandable pixel points;

[0011] The detail area is scaled according to a first scaling algorithm, and the area outside the detail area is scaled according to a second scaling algorithm to obtain a scaled binocular image; the first scaling algorithm has a smaller loss than the second scaling algorithm.

[0012] According to the method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, determining the detail level of the pixel point based on the correlation between the gradient of the pixel point and the gradient of the edge curve includes:

[0013] Determine the angle between the perpendicular line from the pixel point to the edge curve and the gradient direction of the pixel point;

[0014] The detail level of the pixel point is determined according to the gradient value of the pixel point, the average gradient value of the edge curve, and the angle.

[0015] According to the method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, determining the detail level of the pixel point based on the gradient value of the pixel point, the average gradient value of the edge curve, and the angle includes:

[0016] Determine the ratio of the gradient value of the pixel point to the average gradient value of the edge curve to obtain a gradient ratio;

[0017] The detail level of the pixel is determined according to the gradient ratio and the cosine value of the angle.

[0018] According to the method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, the pixel points whose detail levels meet preset conditions are determined as expandable pixel points, including:

[0019] Pixels whose detail level is greater than or equal to a preset detail level threshold are determined as expandable pixels.

[0020] According to the method for enhancing the visual effect of a near-eye display device by enhancing the perception difference provided by the present invention, the detail area is scaled according to a first scaling algorithm, and the area outside the detail area is scaled according to a second scaling algorithm to obtain a scaled binocular image, including:

[0021] Determining a length ratio and a width ratio between a microdisplay in the near-eye display device and the binocular image;

[0022] If the length ratio is not equal to the width ratio, determining the smaller ratio between the length ratio and the width ratio as the scaling ratio;

[0023] If the length ratio is equal to the width ratio, determining any one of the length ratio and the width ratio as the scaling ratio;

[0024] The detail area is scaled in accordance with the scaling ratio according to a first scaling algorithm, and the area outside the detail area is scaled in accordance with the scaling ratio according to a second scaling algorithm to obtain a scaled binocular image.

[0025] According to the method for enhancing the visual effect of a near-eye display device by enhancing the perception difference provided by the present invention, the binocular image includes a left-eye image and a right-eye image;

[0026] After scaling the detail area according to the first scaling algorithm and scaling the area outside the detail area according to the second scaling algorithm to obtain the scaled binocular image, the method further includes:

[0027] Matching the detail areas in the scaled left and right images;

[0028] Determining whether the brightness difference in the matched detail area in the scaled left-eye image and the right-eye image is within a range recognizable by the human eye;

[0029] If yes, reducing the brightness difference in the matched detail area in the scaled left image and the right image;

[0030] If not, the brightness difference in the matched detail area in the scaled left image and the right image is enhanced.

[0031] According to the method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, determining whether the brightness difference in the matching detail area in the scaled left-eye image and the right-eye image is within the range recognizable by the human eye includes:

[0032] Determining a maximum brightness difference within the matched detail area in the scaled left image and the right image;

[0033] If the maximum brightness difference is greater than or equal to the minimum visual difference of the human eye, it is determined that the brightness difference is within the range recognizable by the human eye;

[0034] If the maximum brightness difference is smaller than the minimum visual difference of the human eye, it is determined that the brightness difference is not within the range recognizable by the human eye.

[0035] According to the method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, the method of enhancing the brightness difference within the matching detail area in the scaled left-eye image and the right-eye image includes:

[0036] For each of the matched detail areas in the scaled left image and the right image, determine a quantized value of the detail information level of the detail area according to the brightness value of each pixel in the detail area and the detail level;

[0037] The brightness of the detail area with a larger quantized value of the detail information level is increased, and the brightness of the detail area with a smaller quantized value of the detail information level is reduced.

[0038] According to the method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, determining a quantitative value of the detail information degree of the detail area according to the brightness value of each pixel point in the detail area and the detail degree, includes:

[0039] Determine the average brightness value and the extreme brightness difference of the detail area according to the brightness value of each pixel in the detail area;

[0040] Determining a sum of the detail levels of the detail area according to the detail levels of each pixel in the detail area;

[0041] A detail information degree quantization value of the detail area is determined according to the sum of the brightness average value, the brightness extreme difference, and the detail degree.

[0042] According to the method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference provided by the present invention, reducing the brightness difference in the matching detail area in the scaled left-eye image and the right-eye image includes:

[0043] Comparing the overall brightness of the matched detail areas in the scaled left image and the right image;

[0044] The brightness of the detail area with greater overall brightness is reduced, and the brightness of the detail area with smaller overall brightness is increased, until the brightness difference in the matching detail areas is no longer within the range recognizable by the human eye.

[0045] The present invention has the following beneficial effects: edge detection is performed on a binocular image to be displayed on a near-eye display device, and the area enclosed by the obtained edge curve is used as an initial detail area. Then, traversing is performed layer by layer outward from the initial detail area. For each pixel point in the layer traversed each time, the detail level of the pixel point is determined according to the correlation between the gradient of the pixel point and the gradient of the edge curve. The pixel points whose detail levels meet preset conditions are determined as expandable pixels. The detail area is obtained based on the initial detail area and each expandable pixel point, thereby accurately determining the detail area with color change, avoiding the problem of only taking the high-gradient area extracted by edge detection as the detail area and ignoring the area around the edge with the same color change, which leads to a more serious detail loss. The accurately extracted detail area is scaled according to the first scaling algorithm, and the area outside the detail area is scaled according to the second scaling algorithm. The first scaling algorithm has a smaller loss than the second scaling algorithm, which can ensure that less image details are lost during the scaling process, thereby improving the visual effect of the near-eye display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flowchart of a method for enhancing perceptual differences to improve the visual effects of a near-eye display device provided by one embodiment of the present invention;

[0048] Figure 2 This is an example diagram of the effects of images at different brightness levels provided by an embodiment of the present invention;

[0049] Figure 3 A schematic diagram of the overall process of a method for enhancing perceptual differences to improve the visual effects of a near-eye display device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation method, structure, features and effects of a method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference proposed by the present invention. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0051] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0052] The following describes in detail a specific solution of a method for enhancing perceptual differences to improve the visual effects of a near-eye display device provided by the present invention in conjunction with the accompanying drawings.

[0053] See also Figure 1 , which shows a flow chart of a method for enhancing perceived difference to improve the visual effect of a near-eye display device provided by an embodiment of the present invention, including the following steps:

[0054] Step 101 : performing edge detection on a binocular image to be displayed by a near-eye display device, and taking an area enclosed by the obtained edge curve as an initial detail area.

[0055] A near-eye display (NED) uses a microdisplay and relay optical paths to reconstruct virtual scenes at a relatively close distance to the human eye, allowing the human eye to view images on the microdisplay within a comfortable range, providing the user with an immersive experience. A binocular image consists of two images: a left-eye image and a right-eye image, representing the image material to be displayed on the left and right independent NED microdisplays.

[0056] In one embodiment, the binocular image is preprocessed first, and then edge detection is performed on the preprocessed binocular image, and the area surrounded by the obtained edge curve is used as the initial detail area. The preprocessing may include grayscale processing and denoising processing.

[0057] Step 102: traverse outward layer by layer from the initial detail area.

[0058] Specifically, the algorithm first traverses to the first layer adjacent to the initial detail area, then traverses to the next layer outside the first layer as the second layer, then traverses to the next layer outside the second layer as the third layer, and so on. The algorithm stops traversing when no expandable pixels exist in the traversed layer.

[0059] Step 103 : for each pixel point in the layer traversed each time, determine the detail level of the pixel point according to the correlation between the gradient of the pixel point and the gradient of the edge curve.

[0060] In one embodiment, for each pixel point in the layer traversed each time, the detail level of the pixel point is determined based on the correlation between the gradient magnitude of the pixel point and the average gradient value of the edge curve, and the correlation between the gradient direction of the pixel point and the direction of the perpendicular line from the pixel point to the edge curve.

[0061] Step 104 : Determine the pixels whose detail levels meet the preset conditions as expandable pixels.

[0062] The greater the detail level of a pixel point, the more likely the pixel point is to be determined as an expandable pixel point. Preset conditions can be set based on this principle.

[0063] In one embodiment, the detail level of a pixel is compared with a preset detail level threshold, and whether the pixel is an expandable pixel is determined based on the comparison result. The preset detail level threshold can be user-defined, for example, the preset detail level threshold can be set to 0.65.

[0064] Step 105 : Obtain a detail area based on the initial detail area and each expandable pixel point.

[0065] Step 106 : Scale the detail area according to a first scaling algorithm, and scale the area outside the detail area according to a second scaling algorithm to obtain a scaled binocular image; the first scaling algorithm has a smaller loss than the second scaling algorithm.

[0066] The first scaling algorithm has a lower loss than the second scaling algorithm, and the second scaling algorithm is faster to calculate than the first scaling algorithm, and the scaled image occupies less space. For example, the first scaling algorithm may be bicubic interpolation, and the second scaling algorithm may be nearest neighbor interpolation.

[0067] In one embodiment, a scaling ratio is determined based on a length ratio and a width ratio between a microdisplay in a near-eye display device and a binocular image. Then, the detail area is scaled in accordance with the scaling ratio using a first scaling algorithm, and the area outside the detail area is scaled in accordance with the scaling ratio using a second scaling algorithm to obtain a scaled binocular image.

[0068] In one embodiment, after obtaining the scaled binocular image, detail areas in the scaled left-eye image and the scaled right-eye image are matched, and the brightness difference of the matched detail areas in the scaled left-eye image and the scaled right-eye image is enhanced when the human eye cannot perceive the difference (i.e., outside the range of human eye perception), to obtain a binocular image with enhanced perceptual difference, and the binocular image with enhanced perceptual difference is displayed on a near-eye display device.

[0069] The above-mentioned method for enhancing the perceived difference to achieve visual effect improvement of near-eye display devices performs edge detection on the binocular image to be displayed by the near-eye display device, uses the area enclosed by the obtained edge curve as the initial detail area, and then traverses outward layer by layer from the initial detail area. For each pixel point in the layer traversed each time, the detail level of the pixel point is determined based on the correlation between the gradient of the pixel point and the gradient of the edge curve. The pixel points whose detail level meets the preset conditions are determined as expandable pixels. The detail area is obtained based on the initial detail area and each expandable pixel point, thereby accurately determining the detail area with color change, avoiding the problem of only using the high gradient area extracted by edge detection as the detail area while ignoring the area around the edge with the same color change, resulting in a relatively serious detail loss. The accurately extracted detail area is scaled according to the first scaling algorithm, and the area outside the detail area is scaled according to the second scaling algorithm. The first scaling algorithm has a smaller loss than the second scaling algorithm, which can ensure that less image detail is lost during the scaling process, thereby improving the visual effect of the near-eye display device.

[0070] In one embodiment, the detail level of a pixel point is determined based on the correlation between the gradient of the pixel point and the gradient of the edge curve, including: determining the angle between the perpendicular line from the pixel point to the edge curve and the gradient direction of the pixel point; and determining the detail level of the pixel point based on the gradient value of the pixel point, the average gradient value of the edge curve, and the angle.

[0071] It can be understood that the larger the gradient value of a pixel point, the greater the degree of color change at the pixel point position, and the greater the degree of detail of the pixel point. The smaller the average gradient value of the edge curve, the greater the degree of color change at the pixel point position compared to the degree of color change of the edge curve, and the greater the degree of detail of the pixel point. The smaller the angle between the perpendicular line from the pixel point to the edge curve and the gradient direction of the pixel point, the more the color change trend at the pixel point position conforms to the color change trend of the edge curve, and the greater the degree of detail of the pixel point. Therefore, in one embodiment, the degree of detail of the pixel point is positively correlated with the gradient value of the pixel point, negatively correlated with the average gradient value of the edge curve, and negatively correlated with the angle.

[0072] In the above embodiment, the angle between the perpendicular line from the pixel to the edge curve and the gradient direction of the pixel is determined. Based on the gradient value of the pixel, the average gradient value of the edge curve, and the angle, the detail level of the pixel can be accurately determined.

[0073] In one embodiment, the detail level of a pixel is determined based on the gradient value of the pixel, the average gradient value of the edge curve, and the angle, including: determining the ratio of the gradient value of the pixel to the average gradient value of the edge curve to obtain a gradient ratio; and determining the detail level of the pixel based on the gradient ratio and the cosine value of the angle.

[0074] In one embodiment, based on the grayscale image of the binocular image, an image rectangular coordinate system is established with the length and width of the pixel as the unit length. The coordinate position of the pixel point is ,in, Represents the index of the pixel point, .

[0075] In one embodiment, the detail level of a pixel is positively correlated with the gradient ratio and the cosine value of the angle.

[0076] In one embodiment, the detail level of a pixel is determined based on the product of the gradient ratio and the cosine value of the angle. The formula is as follows:

[0077]

[0078] in, Indicates the The level of detail per pixel. Indicates the The gradient of a pixel. Indicates the The gradient value of a pixel. Indicates the average gradient value of the edge curve. Indicates the The perpendicular line from the pixel point to the edge curve is The angle between the gradient directions of the pixels. Indicates the The perpendicular line from the pixel point to the edge curve is The cosine of the angle between the gradient directions of the pixels.

[0079] It can be understood that the gradient ratio Indicates the degree of color change at the pixel location compared to the color change of the edge curve. The larger the value, the closer the color change at the pixel location is to or even greater than the color change at the edge curve, and the greater the detail of the pixel. The cosine value of the angle between the perpendicular line from the pixel to the edge curve and the gradient direction of the pixel The larger it is (i.e. the closer it is to 1), the smaller the angle between the vertical line from the pixel to the edge curve and the gradient direction of the pixel. The smaller it is (i.e. the closer it is to 0), the more the color change trend at the pixel position conforms to the color change trend at the edge curve position, and the greater the detail level of the pixel. The larger the value is, the greater the degree of color change at the pixel position and the more consistent it is with the change trend in the detail area, and the more it should be expanded into the detail area.

[0080] In the above embodiment, the ratio of the gradient value of the pixel point to the average gradient value of the edge curve is determined to obtain the gradient ratio. According to the gradient ratio and the cosine value of the angle, the detail level of the pixel point can be accurately determined.

[0081] In one embodiment, determining pixels whose detail levels meet a preset condition as expandable pixels includes: determining pixels whose detail levels are greater than or equal to a preset detail level threshold as expandable pixels.

[0082] The preset detail level threshold may be user-defined, for example, the preset detail level threshold may be set to 0.65.

[0083] In one embodiment, if the detail level of a pixel is less than a preset detail level threshold, no operation is performed on the pixel.

[0084] In the above embodiment, pixels whose detail level is greater than or equal to a preset detail level threshold are determined as expandable pixels, which can accurately determine expandable pixels from each traversed layer.

[0085] In one embodiment, scaling a detail area according to a first scaling algorithm and scaling an area outside the detail area according to a second scaling algorithm to obtain a scaled binocular image includes: determining a length ratio and a width ratio between a microdisplay in a near-eye display device and the binocular image; if the length ratio and the width ratio are not equal, determining the smaller of the length ratio and the width ratio as the scaling ratio; if the length ratio and the width ratio are equal, determining either the length ratio or the width ratio as the scaling ratio; scaling the detail area according to the first scaling algorithm to conform to the scaling ratio, and scaling the area outside the detail area according to the second scaling algorithm to conform to the scaling ratio, to obtain the scaled binocular image.

[0086] The length ratio is the ratio of the number of pixels in the length direction of the microdisplay in the near-eye display device to the number of pixels in the length direction of the binocular image. The width ratio is the ratio of the number of pixels in the width direction of the microdisplay in the near-eye display device to the number of pixels in the width direction of the binocular image.

[0087] For example, if the number of pixels in the length and width of the binocular image is n and m respectively, and the number of pixels in the length and width of the micro display in the near-eye display device is N and M respectively, then the length ratio is , the width ratio is If the length ratio is equal to the width ratio, then , then any ratio between the length ratio and the width ratio is determined as the scaling ratio, that is, the scaling ratio If the length ratio is not equal to the width ratio, , then the smaller ratio between the length ratio and the width ratio is determined as the scaling ratio, that is, the scaling ratio .

[0088] In the above embodiment, if the length ratio is equal to the width ratio, any one of the length ratio and the width ratio is determined as the scaling ratio; if the length ratio is not equal to the width ratio, the smaller ratio of the length ratio and the width ratio is determined as the scaling ratio, which can ensure the maximum possible utilization of the micro display.

[0089] In one embodiment, a binocular image includes a left-eye image and a right-eye image; after scaling a detail area according to a first scaling algorithm and scaling an area outside the detail area according to a second scaling algorithm to obtain a scaled binocular image, the method further includes: matching the detail areas in the scaled left and right images; determining whether a brightness difference within the matching detail areas in the scaled left and right images is within a range recognizable by the human eye; if so, reducing the brightness difference within the matching detail areas in the scaled left and right images; if not, enhancing the brightness difference within the matching detail areas in the scaled left and right images.

[0090] In one embodiment, corner detection is performed on the left and right images to obtain corner information in the left and right images. For example, the SIFT corner detection algorithm can be used for corner detection. Based on the corner information in the left and right images, detail regions in the left and right images are then matched (paired) to match detail regions that have consistent relative positions and internal corner information.

[0091] It can be understood that the image brightness also has a certain impact on the information contained in the image. Figure 2 From left to right in the figure are the original image, the image after the original image is brightened, and the image after the original image is dimmed. Figure 2 As can be seen, details are more apparent in brighter images, while outlines of objects are more distinct in darker images. In near-eye display devices, each eye is provided with an independent microdisplay. The images displayed by the two microdisplays simulate binocular vision, though they are not completely identical. This enhances the perception of binocular brightness differences for the same object in a brightness range imperceptible to the human eye, allowing the user to combine the different information received from both eyes for a more three-dimensional and realistic user experience.

[0092] Therefore, in the above embodiment, the detail areas in the scaled left-eye image and the right-eye image are matched; it is determined whether the brightness difference in the matching detail areas in the scaled left-eye image and the right-eye image is within the range recognizable to the human eye. If so, the brightness difference in the matching detail areas in the scaled left-eye image and the right-eye image is reduced; if not, the brightness difference in the matching detail areas in the scaled left-eye image and the right-eye image is enhanced. This achieves the goal of enhancing the brightness difference in the matching detail areas in the scaled left-eye image and the right-eye image when the human eye cannot recognize them (that is, outside the range recognizable to the human eye), increasing the amount of information in the binocular image, enhancing the stereoscopic effect and realism, and thus improving the visual effect of the near-eye display device.

[0093] In one embodiment, determining whether the brightness difference within the matching detail areas in the scaled left-eye image and the right-eye image is within a range recognizable to the human eye includes: determining a maximum brightness difference within the matching detail areas in the scaled left-eye image and the right-eye image; if the maximum brightness difference is greater than or equal to a minimum visual difference of the human eye, determining that the brightness difference is within a range recognizable to the human eye; and if the maximum brightness difference is less than the minimum visual difference of the human eye, determining that the brightness difference is not within a range recognizable to the human eye.

[0094] Among them, the minimum visual difference of the human eye is the minimum brightness difference that the human eye can recognize, determined according to Weber's Law.

[0095] In one embodiment, for each detail area, the difference between the maximum brightness within the detail area in the scaled right-eye image and the minimum brightness within the detail area in the scaled left-eye image is determined to obtain a first difference, the difference between the maximum brightness within the detail area in the scaled left-eye image and the minimum brightness within the detail area in the scaled right-eye image is determined to obtain a second difference, and the maximum value is selected from the first difference and the second difference as the maximum brightness difference value of the detail area.

[0096] For example: suppose the maximum brightness and minimum brightness of a detail area in the left image are and The maximum brightness and minimum brightness of the matching detail area in the right image are and , the minimum visual error of the human eye is , then the maximum brightness difference of the detail area is , if the maximum brightness difference is greater than or equal to the minimum visual difference of the human eye, that is , then it is determined that the brightness difference is within the range that the human eye can recognize, and the brightness difference needs to be reduced so that the human eye cannot recognize the brightness difference to avoid affecting the display effect; if the maximum brightness difference is less than the minimum visual difference of the human eye, that is , it is determined that the brightness difference is not within the range that the human eye can recognize, that is, the human eye cannot recognize the brightness difference. The brightness difference is enhanced as much as possible within the range, thereby increasing the amount of information in the binocular image and improving the stereoscopic effect and realism.

[0097] In the above embodiment, the maximum brightness difference in the matching detail areas of the scaled left-eye image and the right-eye image is determined. If the maximum brightness difference is greater than or equal to the minimum visual difference of the human eye, it is determined that the brightness difference is within the range recognizable by the human eye. If the maximum brightness difference is less than the minimum visual difference of the human eye, it is determined that the brightness difference is not within the range recognizable by the human eye. This achieves accurate judgment on whether to enhance or reduce the brightness difference based on Weber's theorem.

[0098] In one embodiment, enhancing the brightness difference within the matching detail areas in the scaled left and right images includes: determining, for each of the matching detail areas in the scaled left and right images, a detail information degree quantization value of the detail area based on the brightness value and detail degree of each pixel in the detail area; and increasing the brightness of the detail area having a larger detail information degree quantization value, and decreasing the brightness of the detail area having a smaller detail information degree quantization value.

[0099] Specifically, based on the brightness value and detail level of each pixel in the detail area in the left image, the detail information level quantization value of the detail area in the left image is determined; based on the brightness value and detail level of each pixel in the matching detail area in the right image, the detail information level quantization value of the detail area in the right image is determined. The brightness of the detail area with a larger detail information level quantization value in the left and right images is increased, and the brightness of the detail area with a smaller detail information level quantization value is reduced.

[0100] In one embodiment, the overall brightness level and brightness stability of the detail region can be determined based on the brightness value of each pixel in the detail region, and the overall detail level of the detail region can be determined based on the detail level of each pixel in the detail region. A quantized value of the detail information level of the detail region can be determined based on the overall brightness level, brightness stability, and overall detail level of the detail region. The quantized value of the detail information level of the detail region is positively correlated with the overall brightness level, brightness stability, and overall detail level.

[0101] In one embodiment, the brightness of the detail region with a larger quantized value of the detail information level can be gradually increased according to a preset step size, and the brightness of the detail region with a smaller quantized value of the detail information level can be gradually decreased according to a preset step size until the maximum brightness difference between the two detail regions equals the minimum visual difference of the human eye, and then the increase or decrease stops. For example, the preset step size can be set to 1.

[0102] For example: Assuming the minimum visual error of the human eye The maximum brightness of the detail area in the left image is 8, and the minimum brightness of the corresponding detail area in the right image is 7. The maximum brightness difference between the two is 1, so the maximum brightness difference is smaller than the minimum visual difference of the human eye. , the brightness difference needs to be enhanced. Assuming the preset step size is 1, increase the brightness of the brighter detail area of the left image by 1, and decrease the brightness of the corresponding detail area of the lower brightness right image by 1. After adjustment, the brightness of the two images becomes 9 and 6, respectively, with a maximum brightness difference of 3, which is still less than ∆l. Continue adjustment until the brightness of the two images is 10 and 5, respectively. At this point, the maximum brightness difference is 5, which is equal to the minimum visual difference of the human eye, ∆l, and the adjustment stops.

[0103] In the above embodiment, for the matching detail areas in the scaled left-eye image and the right-eye image, a detail information degree quantization value of the detail area is determined based on the brightness value and detail degree of each pixel in the detail area, the brightness of the detail area with a larger detail information degree quantization value is increased, and the brightness of the detail area with a smaller detail information degree quantization value is reduced. This fully considers the information contained in the detail area, and can accurately enhance the brightness difference of the matching detail areas in the scaled left-eye image and the right-eye image, thereby increasing the amount of information in the binocular image, enhancing the stereoscopic effect and realism, and improving the visual effect of the near-eye display device. It avoids the problem of poor visual effect caused by insufficient consideration due to the method of directly increasing the brightness of the detail area with larger brightness and decreasing the brightness of the detail area with smaller brightness in the left-eye image and the right-eye image.

[0104] In one embodiment, a quantitative value of the detail information level of the detail area is determined based on the brightness value and detail level of each pixel in the detail area, including: determining the brightness average value and brightness range of the detail area based on the brightness value of each pixel in the detail area; determining the sum of the detail levels of the detail area based on the detail level of each pixel in the detail area; and determining the quantitative value of the detail information level of the detail area based on the brightness average value, brightness range and sum of the detail level.

[0105] The brightness extreme difference is the difference between the maximum brightness and the minimum brightness in the detail area.

[0106] In one embodiment, the quantized value of the detail information level of the detail area is positively correlated with the average brightness, negatively correlated with the extreme brightness difference, and positively correlated with the sum of the detail levels.

[0107] In one embodiment, the ratio of the average brightness of the detail area to the extreme brightness difference can be calculated, and then the ratio can be multiplied by the sum of the detail levels to obtain the detail information level quantization value of the detail area. Taking the calculation of the detail information level quantization value of the detail area in the left eye image as an example, the formula is as follows:

[0108]

[0109] in, Indicates the quantitative value of the detail information level of the detail area in the left eye image. Indicates the average brightness of the detail area in the left eye image. Indicates the extreme brightness difference of the detail area in the left eye image. Indicates the first detail area in the left image. The level of detail per pixel. Indicates the number of pixels in the detail area of the left image. Indicates the sum of the detail levels of the detail areas in the left eye image.

[0110] It can be understood that the average brightness of the detail area can measure the overall brightness level in the detail area, and the brightness range of the detail area can inversely measure the brightness stability of the detail area. The larger the brightness range, the worse the brightness stability. Therefore, the ratio of the average brightness of the detail area to the brightness range is The larger the value, the greater the brightness in the detail area and the more stable the brightness. It can measure the overall detail level of the detail area. The above formula uses brightness as the value Overall level of detail Make corrections to obtain the quantitative value of the detail information level of the detail area Detail information degree quantification value The larger the value, the higher the level of information in the detail area of the left-eye image.

[0111] Similarly, the quantitative value of the detail information degree of the detail area in the right eye image can be calculated Compare the detail information quantization value of the detail area in the left image And the detail information quantization value of the detail area in the right image , the brightness of the detail area in the image corresponding to the larger value is increased, and the brightness of the detail area in the image corresponding to the smaller value is reduced.

[0112] In the above embodiment, the quantized value of the detail information level of the detail region can be accurately determined based on the sum of the average brightness value, the extreme brightness difference, and the detail level of the detail region. This fully considers the information contained within the detail region and accurately enhances the brightness difference between the matching detail regions in the scaled left and right images, thereby increasing the amount of information in the binocular image, enhancing the sense of three-dimensionality and realism, and improving the visual quality of the near-eye display device. This avoids the problem of poor visual quality caused by insufficient consideration caused by simply increasing the brightness of the brighter detail regions in the left and right images and decreasing the brightness of the smaller detail regions.

[0113] In one embodiment, reducing the brightness difference within the matching detail areas in the scaled left and right images includes: comparing the overall brightness of the matching detail areas in the scaled left and right images; reducing the brightness of the detail areas with greater overall brightness, and increasing the brightness of the detail areas with less overall brightness, until the brightness difference within the matching detail areas is no longer within a range perceptible to the human eye.

[0114] In one embodiment, if the maximum brightness difference is the difference between the maximum brightness in the detail area in the scaled right image and the minimum brightness in the detail area in the scaled left image, that is, , the overall brightness of the detail area in the right image is greater than the overall brightness of the corresponding detail area in the left image; conversely, if the value of the maximum brightness difference is the difference between the maximum brightness in the detail area in the scaled left image and the minimum brightness in the detail area in the scaled right image, that is, , the overall brightness of the detail area in the left image is greater than the overall brightness of the corresponding detail area in the right image. The brightness of the detail area with greater overall brightness is reduced, and the brightness of the detail area with smaller overall brightness is increased until the brightness difference in the matching detail areas is no longer within the range that the human eye can recognize. .

[0115] In the above embodiment, the brightness of the detail areas with greater overall brightness in the scaled left-eye image and the right-eye image is reduced, and the brightness of the detail areas with less overall brightness is increased until the brightness difference in the matching detail areas is no longer within the range recognizable by the human eye. This can prevent the human eye from recognizing the brightness difference between the left-eye image and the right-eye image, which would affect the visual effect.

[0116] Based on the above embodiments, Figure 3 FIG. 1 is a schematic diagram of the overall process of the method for enhancing the perceived difference to improve the visual effect of a near-eye display device provided by the present invention, including the following steps:

[0117] Step 301: Acquire a binocular image captured by a binocular camera.

[0118] Step 302: pre-process the binocular image and obtain an initial detail area through edge detection.

[0119] Step 303 : expanding the detail area based on the color changes of the pixels around the initial detail area, and scaling the binocular image according to the detail area.

[0120] Step 304 : Match the same detail area in the binocular images and enhance the reasonable perceptual difference between the two.

[0121] Step 305: Use the scaled and enhanced binocular image for a near-eye display device.

[0122] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the scope of the present application, and all such modifications and improvements fall within the scope of protection of the present application.

[0124] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0125] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A method for enhancing perceptual differences to improve the visual effects of near-eye display devices, characterized in that: The method comprises: Performing edge detection on the binocular image to be displayed by the near-eye display device, and using the area enclosed by the obtained edge curve as the initial detail area; Traversing outward layer by layer from the initial detail area; For each pixel point in the layer traversed each time, determining the detail level of the pixel point according to the correlation relationship between the gradient of the pixel point and the gradient of the edge curve; Determining pixels whose detail levels meet preset conditions as expandable pixels; Obtaining a detail area based on the initial detail area and each of the expandable pixel points; The detail area is scaled according to a first scaling algorithm, and the area outside the detail area is scaled according to a second scaling algorithm to obtain a scaled binocular image; the first scaling algorithm has a smaller loss than the second scaling algorithm.

2. The method for enhancing the visual effect of near-eye display devices by enhancing the perceived difference according to claim 1, characterized in that: The determining the detail level of the pixel point according to the correlation between the gradient of the pixel point and the gradient of the edge curve includes: Determine the angle between the perpendicular line from the pixel point to the edge curve and the gradient direction of the pixel point; The detail level of the pixel point is determined according to the gradient value of the pixel point, the average gradient value of the edge curve, and the angle.

3. The method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference according to claim 2, characterized in that: The determining the detail level of the pixel point according to the gradient value of the pixel point, the average gradient value of the edge curve, and the angle includes: Determine the ratio of the gradient value of the pixel point to the average gradient value of the edge curve to obtain a gradient ratio; The detail level of the pixel is determined according to the gradient ratio and the cosine value of the angle.

4. The method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference according to claim 1, characterized in that: The step of determining the pixel points whose detail levels meet a preset condition as expandable pixel points includes: Pixels whose detail level is greater than or equal to a preset detail level threshold are determined as expandable pixels.

5. The method for enhancing the visual effect of near-eye display devices by enhancing the perceived difference according to claim 1, characterized in that: Scaling the detail area according to a first scaling algorithm and scaling the area outside the detail area according to a second scaling algorithm to obtain a scaled binocular image includes: Determining a length ratio and a width ratio between a microdisplay in the near-eye display device and the binocular image; If the length ratio is not equal to the width ratio, determining the smaller ratio between the length ratio and the width ratio as the scaling ratio; If the length ratio is equal to the width ratio, determining any one of the length ratio and the width ratio as the scaling ratio; The detail area is scaled in accordance with the scaling ratio according to a first scaling algorithm, and the area outside the detail area is scaled in accordance with the scaling ratio according to a second scaling algorithm to obtain a scaled binocular image.

6. The method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference according to any one of claims 1 to 5, characterized in that: The binocular image includes a left-eye image and a right-eye image; After scaling the detail area according to the first scaling algorithm and scaling the area outside the detail area according to the second scaling algorithm to obtain the scaled binocular image, the method further includes: Matching the detail areas in the scaled left and right images; Determining whether the brightness difference in the matched detail area in the scaled left-eye image and the right-eye image is within a range recognizable by the human eye; If yes, reducing the brightness difference in the matched detail area in the scaled left image and the right image; If not, the brightness difference in the matched detail area in the scaled left image and the right image is enhanced.

7. The method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference according to claim 6, characterized in that: The determining whether the brightness difference in the matched detail area in the scaled left-eye image and the right-eye image is within a range recognizable by the human eye includes: Determining a maximum brightness difference within the matched detail area in the scaled left image and the right image; If the maximum brightness difference is greater than or equal to the minimum visual difference of the human eye, it is determined that the brightness difference is within the range recognizable by the human eye; If the maximum brightness difference is smaller than the minimum visual difference of the human eye, it is determined that the brightness difference is not within the range recognizable by the human eye.

8. The method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference according to claim 6, characterized in that: The enhancing the brightness difference in the matched detail area in the scaled left image and the right image includes: For each of the matched detail areas in the scaled left image and the right image, determine a quantized value of the detail information level of the detail area according to the brightness value of each pixel in the detail area and the detail level; The brightness of the detail area with a larger quantized value of the detail information level is increased, and the brightness of the detail area with a smaller quantized value of the detail information level is reduced.

9. The method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference according to claim 8, characterized in that: The determining, based on the brightness value of each pixel point in the detail area and the detail level, a quantitative value of the detail information level of the detail area includes: Determine the average brightness value and the extreme brightness difference of the detail area according to the brightness value of each pixel in the detail area; Determining a sum of the detail levels of the detail area according to the detail levels of each pixel in the detail area; A detail information degree quantization value of the detail area is determined according to the sum of the brightness average value, the brightness extreme difference, and the detail degree.

10. The method for enhancing the visual effect of a near-eye display device by enhancing the perceived difference according to claim 8, characterized in that: The reducing the brightness difference in the matched detail area in the scaled left image and the right image includes: Comparing the overall brightness of the matched detail areas in the scaled left image and the right image; The brightness of the detail area with greater overall brightness is reduced, and the brightness of the detail area with smaller overall brightness is increased, until the brightness difference in the matching detail areas is no longer within the range recognizable by the human eye.

Images