Image Detail Enhancement Method and System for a Binocular Near-Eye Display Device

Abstract

The present invention relates to the technical field of image processing, and particularly relates to an image detail enhancement method and system for a binocular near-eye display device. The technical solution of the present invention obtains a first image and a second image collected by the binocular near-eye display device under the current display task, divides both the first image and the second image into multiple regions, and determines an overlapping region and a non-overlapping region according to the feature similarity between the regions divided from the first image and the second image; for each divided region within the overlapping region and the non-overlapping region, a first enhancement coefficient and a second enhancement coefficient are respectively configured; according to the first enhancement coefficient and the second enhancement coefficient, the current display task of the binocular near-eye display device is executed. This technical solution can respectively configure corresponding enhancement coefficients based on the overlapping region and the non-overlapping region of the binocular near-eye images, enabling more accurate enhancement processing of the binocular near-eye images and improving the display quality of the binocular near-eye images.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and specifically relates to an image detail enhancement method and system for a binocular near-eye display device. Background Art

[0002] A binocular near-eye display device is a device designed to provide users with a near-eye display experience, such as AR (Augmented Reality) glasses, VR (Virtual Reality) helmets, etc. In recent years, it has been widely used in many fields such as entertainment, education, medical treatment, and industry. These devices provide users with an immersive visual experience by simulating or enhancing the real environment. However, due to the limitations of display technology, binocular near-eye display devices have problems such as insufficient resolution when presenting image details, which affects the visual experience of users. Summary of the Invention

[0003] In order to solve the technical problem of how to improve the image display quality of binocular near-eye images, the purpose of the present invention is to provide an image detail enhancement method and system for a binocular near-eye display device. The specific technical solutions adopted are as follows:

[0004] In a first aspect, an embodiment of the present invention provides an image detail enhancement method for a binocular near-eye display device. The method includes:

[0005] Obtain a first image and a second image collected by the binocular near-eye display device under the current display task, where the first image and the second image are two images collected at the same moment in the current display task from different angles;

[0006] Divide both the first image and the second image into multiple regions, and determine the overlapping regions and non-overlapping regions according to the feature similarity between the regions divided from the first image and the second image;

[0007] Configure a first enhancement coefficient and a second enhancement coefficient for each divided region in the overlapping regions and non-overlapping regions respectively;

[0008] Execute the current display task of the binocular near-eye display device according to the first enhancement coefficient and the second enhancement coefficient.

[0009] In an optional embodiment, determining the overlapping regions and non-overlapping regions according to the feature similarity between the regions divided from the first image and the second image includes:

[0010] Determine the regional gray value according to the gray distribution characteristics of the pixel points of the regions divided from the first image and the second image, and the shape similarity between the regions of the first image and the second image;

[0011] Determine the overlapping regions and non - overlapping regions between the first image and the second image according to the regional gray - scale values and shape similarities.

[0012] In an alternative embodiment, according to the gray - scale distribution characteristics of the pixel points in the regions divided by the first image and the second image, determine the regional gray - scale values, including:

[0013] Obtain the pixel gray - scale values, the number of valid pixel points, and the number of gray - scale values of each divided region on the first image and the second image;

[0014] According to the formula , obtain the regional gray - scale value of the th region on the th image, where is the pixel gray - scale value of the th region on the th image, is the number of valid pixel points of the th region on the th image, is the number of gray - scale values of the th region on the th image, is 1 or 2, is a natural number greater than 0, is a normalization function.

[0015] In an alternative embodiment, according to the gray - scale distribution characteristics of the pixel points in the regions divided by the first image and the second image, determine the shape similarity between each region of the first image and the second image, including:

[0016] Construct the edge chain code of the corresponding region according to the edge pixel points of each divided region on the first image and the second image;

[0017] According to the edge chain code of each divided region, determine the angle difference between adjacent pixel lines of each region, where the pixel line is the connection line between the geometric center of the edge chain code and the edge pixel point;

[0018] Perform pixel - point sampling and screening according to the angle difference between adjacent pixel lines of each region to obtain the target edge points of each region;

[0019] Perform polar - coordinate histogram transformation on all the target edge points of each region to obtain the shape feature vector of the corresponding region;

[0020] Obtain the shape similarity according to the similarity between the shape feature vectors of the regions divided on the first image and the shape feature vectors of the regions divided on the second image.

[0021] In an alternative embodiment, pixel sampling and screening are performed based on the angular difference between adjacent pixel lines in each region to obtain target edge points for each region, including:

[0022] When the angular difference of the current adjacent pixel lines is greater than a preset first threshold, it is determined that all edge pixel points corresponding to the current adjacent pixel lines are target edge points;

[0023] When the angular difference of the current adjacent pixel lines is less than or equal to the first threshold and the continuous cumulative quantity is greater than a preset second threshold, a preset number of edge pixel points are screened out from the current adjacent pixel lines as target edge points.

[0024] In an alternative embodiment, the shape similarity is obtained based on the similarity between the shape feature vectors of the regions divided on the first image and the shape feature vectors of the regions divided on the second image, including:

[0025] According to the formula , the distribution similarity of the th target edge point in the th region of the first image and the second image is obtained , where is the shape feature vector of the th target edge point in the th region divided on the first image, is the shape feature vector of the th target edge point in the th region divided on the second image, cos represents the cosine similarity function, is the number of pixel points of the th target edge point in the th region of the first image in the th histogram region, is the number of pixel points of the th target edge point in the th region of the second image in the th histogram region, is the number of histogram regions after dividing the polar coordinate histogram into regions;

[0026] According to the formula , the shape similarity of the th region of the first image and the second image is obtained , where is the number of polar coordinate histograms in the th region.

[0027] In an alternative embodiment, determining the overlapping region and non-overlapping region between the first image and the second image according to the regional gray value and shape similarity includes:

[0028] Obtaining a regional gray difference value according to the difference between the regional gray values of any two regions between the first image and the second image;

[0029] Obtaining a characteristic matching result of the position of the region according to the regional gray difference value and shape similarity of the regions divided by the first image and the second image;

[0030] Determining the regions corresponding to all the characteristic matching results greater than the third threshold as the overlapping region between the first image and the second image;

[0031] Determining the image regions of the first image and the second image outside the overlapping region as non-overlapping regions.

[0032] In an alternative embodiment, the regions divided in the overlapping region are the first regions, and the regions divided in the non-overlapping region are the second regions; for each divided region in the overlapping region and the non-overlapping region, configuring a first enhancement coefficient and a second enhancement coefficient respectively, including:

[0033] Performing mean processing on the gray values of all pixel points in each divided region to obtain the average gray value of each divided region;

[0034] Obtaining the first enhancement coefficient of the current first region according to the number of regions in the first region, the average gray value of the current first region, and the average gray value of the adjacent first regions of the current first region;

[0035] Obtaining the second enhancement coefficient of the current second region according to the average gray value of the current second region, the average gray value of the target first region, and the first enhancement coefficient, where the target first region is the first region closest to the current second region.

[0036] In an alternative embodiment, performing the current display task of the binocular near-eye display device according to the first enhancement coefficient and the second enhancement coefficient includes:

[0037] Determining the first enhanced gray value of each pixel point in the overlapping region according to the first enhancement coefficient and the gray value of each pixel point in the overlapping region;

[0038] Determining the second enhanced gray value of each pixel point in the non-overlapping region according to the second enhancement coefficient and the gray value of each pixel point in the non-overlapping region;

[0039] Controlling the binocular near-eye display device to perform stereo matching display according to the first enhanced gray value and the second enhanced gray value.

[0040] In a second aspect, an image detail enhancement system for a binocular near-eye display device provided by an embodiment of the present invention includes:

[0041] A shooting terminal for obtaining a first image and a second image collected by the binocular near-eye display device under a current display task, where the first image and the second image are two images collected at the same moment in the current display task from different angles;

[0042] A processing terminal connected to the shooting terminal. The processing terminal is configured to divide both the first image and the second image into multiple regions, and determine an overlapping region and a non-overlapping region according to the feature similarity between the regions divided from the first image and the second image; and configure a first enhancement coefficient and a second enhancement coefficient for each divided region in the overlapping region and the non-overlapping region, respectively;

[0043] A display terminal connected to the processing terminal. The display terminal is configured to execute the current display task of the binocular near-eye display device according to the first enhancement coefficient and the second enhancement coefficient.

[0044] Compared with the prior art, an image detail enhancement method and system for a binocular near-eye display device provided by the present invention have the following advantages:

[0045] The technical solution of the present invention obtains a first image and a second image collected by the binocular near-eye display device under a current display task. Since the first image and the second image are two images collected at the same moment in the current display task from different angles, both the first image and the second image can be divided into multiple regions, and an overlapping region and a non-overlapping region can be determined according to the feature similarity between the regions divided from the first image and the second image; a first enhancement coefficient and a second enhancement coefficient are configured for each divided region in the overlapping region and the non-overlapping region, respectively; and the current display task of the binocular near-eye display device is executed according to the first enhancement coefficient and the second enhancement coefficient. This technical solution can configure corresponding enhancement coefficients based on the overlapping region and the non-overlapping region of the binocular near-eye images, realize more accurate enhancement processing of the binocular near-eye images, make the three-dimensional sense of the binocular near-eye images stronger, and thus improve the display quality of the binocular near-eye images. Description of the Drawings

[0046] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0047] Figure 1Flowchart of an image detail enhancement method for a binocular near-eye display device provided by an embodiment of the present invention;

[0048] Figure 2 Schematic diagram of the principle of image acquisition by a binocular camera provided by an embodiment of the present invention;

[0049] Figure 3 Schematic diagram of the principle of shooting the first image and the second image provided by an embodiment of the present invention;

[0050] Figure 4 Schematic diagram of a 4-bit chain code provided by an embodiment of the present invention;

[0051] Figure 5 Schematic diagram of an 8-bit chain code provided by an embodiment of the present invention;

[0052] Figure 6 Schematic diagram of a polar histogram provided by an embodiment of the present invention;

[0053] Figure 7 Schematic diagram of the divided area of the first image provided by an embodiment of the present invention;

[0054] Figure 8 Schematic diagram of the divided area of the second image provided by an embodiment of the present invention;

[0055] Figure 9 Schematic diagram of the overlapping area and non-overlapping area on a binocular near-eye image provided by an embodiment of the present invention;

[0056] Figure 10 Schematic diagram of configuring an enhancement coefficient for the first image provided by an embodiment of the present invention;

[0057] Figure 11 Schematic diagram of configuring an enhancement coefficient for the second image provided by an embodiment of the present invention;

[0058] Figure 12 Implementation step diagram of the image detail enhancement method provided by an embodiment of the present invention;

[0059] Figure 13 Schematic diagram of the structure of an image detail enhancement system for a binocular near-eye display device provided by an embodiment of the present invention.

[0060] Explanation of reference numerals: 21 - left camera, 22 - right camera, 23 - first image, 24 - second image, 25 - shooting object, 26 - overlapping area, 27 - non-overlapping area, 28 - binocular near-eye image. Detailed implementation manner

[0061] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following will, in conjunction with the accompanying drawings and preferred embodiments, elaborate in detail on a method and system for enhancing image details of a binocular near-eye display device proposed according to the present invention, including its specific implementation manner, structure, features, and effects. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0063] Currently, when enhancing the images captured by a binocular near-eye display device, usually, stereo matching is first performed and then image enhancement. When the effect of the captured original images is poor, the result of stereo matching may be deviated, resulting in a deviation between the effect after image enhancement and the expected effect. The present invention analyzes and processes the two images captured under the current display task, identifies the overlapping area and non-overlapping area of the two images. Then, the image enhancement coefficients are calculated separately for them to complete the image enhancement process. Further, stereo matching is used to present the result in the binocular near-eye display device, making the features more distinct when the binocular near-eye images are displayed, helping to more accurately identify and match the corresponding feature points in the subsequent stereo matching process, thereby improving the accuracy of stereo image display matching and making the display effect presented by the binocular near-eye display device better. The following will specifically describe the specific solutions of a method and system for enhancing image details of a binocular near-eye display device provided by the present invention with reference to the accompanying drawings.

[0064] Please refer to Figure 1 , Figure 1 which is a flowchart of a method for enhancing image details of a binocular near-eye display device provided by an embodiment of the present invention. This method can be applied to the operation of a binocular near-eye display device, which can be a terminal device such as an AR glasses or a VR helmet, as long as it can run this enhancement method, and the device type is not specifically limited here. The method specifically includes:

[0065] S11. Obtain a first image and a second image captured by the binocular near-eye display device under the current display task, where the first image and the second image are two images captured at the same moment from different angles in the current display task.

[0066] Specifically, a left camera and a right camera (or called a binocular camera) are provided on the binocular near-eye display device. Please refer to Figure 2, 21 represents the left camera, and 22 represents the right camera. The left camera is used to capture the viewing field of the user's left eye, and the right camera 22 is used to capture the viewing field of the user's right eye. 23 represents the first image, and 24 represents the second image. The image captured by the left camera is the first image, and the image captured by the right camera is the second image. Since the quality of the lens directly affects the clarity and accuracy of the image during shooting, the use of the lens here needs to be selected according to the specific application scenario. Usually, two cameras are used to collect images in different positional relationships (such as in a straight line, on a plane, or in a three-dimensional distribution) to achieve a stereoscopic vision effect.

[0067] After obtaining the first image and the second image, noise reduction processing can also be performed on them. Through preprocessing, the image quality can be improved, providing a better basis for subsequent processing. When processing the two images collected by the binocular near-eye display device, since when wearing the binocular near-eye display device, people have a relatively high degree of attention to the middle position part in the current viewing angle and a relatively low degree of attention to the two side areas, the enhancement requirements for pixel points at different positions on the image are different. It is necessary to analyze the two images collected by the binocular cameras to improve the accuracy of image enhancement.

[0068] S12. Divide both the first image and the second image into multiple regions, and determine the overlapping regions and non-overlapping regions according to the feature similarity between the regions divided from the first image and the second image.

[0069] Specifically, when processing the two images collected by the binocular near-eye display device, since the left camera and the right camera capture the same part, due to the different shooting angles, the two collected images will have a certain positional deviation. Therefore, it is impossible to directly use the feature matching algorithm to perform feature matching on the two images collected by the binocular near-eye display device. However, from the analysis of the local and overall parts of the two images, the feature manifestations at different positions are basically unchanged. Based on this feature, the two images can be divided into regions, and the regional features of each region can be analyzed to help complete the matching between the images. The first image and the second image can be divided into multiple regions respectively based on the region growing method, or the region division of the first image and the second image can be implemented using an image partitioning model. The two images can be divided into regions based on the same division method, and no specific limitation is made here. It can be understood that since the two images are divided into regions in the same way, the divided regions have similar features, which is convenient for subsequent accurate processing.

[0070] After the region division of the two images is completed, it is possible to determine whether it is an overlapping region based on the feature similarity between the divided regions. When the feature similarity between the divided regions of the first image and the divided regions of the second image is relatively close, it indicates that the two regions are overlapping regions; otherwise, it indicates that the two regions are non-overlapping regions.

[0071] Exemplarily, step S12 includes sub-steps S12-1 to S12-2, which are specifically described as follows:

[0072] S12-1. Determine the regional gray value and the shape similarity between each region of the first image and the second image according to the gray distribution characteristics of the pixel points of the divided regions of the first image and the second image. For regions with the same feature representation on the two images, there is a certain similarity in their gray-scale representation. The regional gray value characterizes the gray-scale performance of the divided region. After performing gray-scale processing on both the first image and the second image, the regional gray value can be calculated based on the gray values of the pixel points of each region. The shape similarity characterizes the degree of shape similarity between each region of the first image and the second image, and the shape similarity can be determined based on the shape presentation characteristics of the divided regions.

[0073] In a specific implementation manner, determining the regional gray value includes S12-1-1 and S12-1-2, which are specifically described as follows:

[0074] S12-1-1. Obtain the pixel point gray value, the number of valid pixel points, and the number of gray values of each divided region on the first image and the second image. The pixel point gray value characterizes the gray value of the pixel points on each divided region, which can be obtained based on the gray-scale processing results of the first image and the second image; the number of valid pixel points is the number of pixel points with gray values in the divided region, which can be statistically obtained based on the pixel point gray values of each region; the number of gray values is the number of gray values included in a single region, which can also be statistically obtained based on the pixel point gray values. For example, in a certain region, there are 20 pixel points, among which, the gray values of 10 pixel points are 210, the gray values of 5 pixel points are 150, the gray values of 3 pixel points are 100, and the gray values of 2 pixel points are 0, then the number of valid pixel points is 18, and the number of gray values is 3.

[0075] S12-1-2. According to the formula , obtain the regional gray value of the th region on the th image, where is the pixel point gray value of the th region on the th image, is the number of valid pixel points of the th region on the The number of valid pixel points in an area is the number of gray values in the area on the th image, where is a natural number greater than 0. When is 1 it is the area gray value of the area on the first image, when is 2 it is the area gray value of the area on the second image, is a normalization function. It can be understood that the normalization function can make the number of gray values fall within a preset range to prevent the area gray value from exceeding the gray value range of 0 - 255.

[0076] Please refer to Figure 3 , where 23 represents the first image, 24 represents the second image, and 25 represents the object being photographed. Since there are local similarities between the first image and the second image, the above analysis mainly uses the gray-scale representation of the images collected by the binocular near-eye display device for calculation. There is an overlapping part in the images collected by the binocular near-eye display device, and there are spatial errors in the gray-scale distribution of the image features in this part, but there are still great similarities in the gray-scale representation.

[0077] Analyzing the similar parts in the images collected by the binocular near-eye display device based only on the gray-scale performance of the images is not accurate enough, and it may not be the same area due to other different performances. For each area in the images collected by the binocular near-eye display device, the shape performance of the same object under a small viewing angle change may be relatively similar. Based on this, in a specific implementation, determining the shape similarity of each area between the first image and the second image includes S12-1-3 to S12-1-5, which are specifically described as follows:

[0078] S12-1-3. According to the edge pixel points of each divided area on the first image and the second image, construct the edge chain code of the corresponding area. Edge pixel points are the pixel points located at the edge position of the divided area. Based on the edge pixel points, an edge chain code can be constructed. Please refer to Figure 4 and Figure 5 , and based on the edge pixel points of the th area on the th image, select the first pixel point on the leftmost vertical line as the starting pixel point, and construct the edge chain code in the clockwise direction. Figure 4 and Figure 5 respectively show 4-bit chain codes and 8-bit chain codes.

[0079] S12-1-4. Determine the angular difference between adjacent pixel lines of each region according to the edge chain code of each divided region, where the pixel line is the connection line between the geometric center of the edge chain code (i.e., the region center) and the edge pixel point. The angular change between the th pixel line and its adjacent pixel line is denoted as . Based on the angular change between adjacent pixel lines, sample the edge pixel points of the th region on the th image to calculate the change in the angular difference between the connection lines of adjacent pixel points: , , are the angles between the th pixel line and the th pixel line of the th region on the th image and the reference line respectively. The reference line can be a horizontal line or a vertical line, and is a natural number greater than 1.

[0080] S12-1-5. Perform pixel point sampling and screening according to the angular difference between adjacent pixel lines of each region to obtain the target edge points of each region. Through pixel point sampling and screening, effective shape features on the region can be obtained, reducing the computational redundancy of data.

[0081] The method of pixel point sampling and screening can be selected according to actual needs. For example, when the angular difference of the current adjacent pixel lines is greater than a preset first threshold, it is determined that all the edge pixel points corresponding to the current adjacent pixel lines are target edge points; when the angular difference of the current adjacent pixel lines is less than or equal to the first threshold and the continuous cumulative quantity is greater than a preset second threshold, a preset number of edge pixel points are selected from the current adjacent pixel lines as target edge points.

[0082] Specifically, set the first threshold . When , it indicates that the angular change difference between the adjacent pixel lines is relatively large, and all the corresponding edge pixel points need to be retained; otherwise, it indicates that the angular change between the adjacent pixel lines is relatively gentle, and it can be deleted during sampling. Further, when there are consecutive adjacent pixel lines with calculated , then pixel points are selected from these consecutive pixel points for deletion, and the remaining edge pixel points are sampled. Through the above method, characteristic edge pixel points can be screened out from all edge pixel points and used as target edge points.

[0083] S12-1-6. Perform polar coordinate histogram transformation on all target edge points in each region to obtain the shape feature vector corresponding to the region.

[0084] Specifically, construct a polar coordinate system centered on each sampled target edge point. In the polar coordinate system, divide the other target edge points on the edge of the current target edge point according to the angle and distance, and count the number of target edge points in each divided region to form a polar coordinate histogram. For example, if a certain region contains 8 target edge points, construct a polar coordinate system centered on each target edge point. Based on the lines connecting the current target edge point and other target edge points, multiple line segments are obtained. By the angles between the multiple line segments and the horizontal or vertical axis of the polar coordinate system, multiple angles are obtained. By statistically analyzing the multiple angles, the histogram of the current target edge point can be obtained. Please refer to Figure 6 and divide the angles once every ; divide the distance according to the divided distance of the angle on the polar coordinate system, as long as each small divided region can be a square or a rectangle. Figure 6 where the abscissa in is the angle and the ordinate is the distance. After dividing the polar coordinate histogram into regions, regions can be obtained. Each small region in the polar coordinate histogram is denoted as , where

[0085] is a natural number greater than 1. Combine the polar coordinate histograms of all target edge points in each region to form a high-dimensional feature vector, that is, the shape feature vector (or shape context descriptor). It can be understood that since the shape feature vector is obtained by transforming the polar coordinate histogram and each target edge point has a corresponding histogram, the shape feature vector can capture the shape edge structure and relative position information of the region. The shape feature vector is denoted as

[0086] . In the formula, represents the histogram vector of the th target edge point in the th region of the th image, and represents the number of pixel points in the th small region divided on the histogram.

[0087] S12-1-7. Obtain the shape similarity based on the similarity between the shape feature vectors of the divided regions on the first image and the shape feature vectors of the divided regions on the second image. The shape similarity characterizes the degree of similarity between the divided regions on the first image and the divided regions on the second image, and the shape similarity can be obtained based on the cosine similarity between the two shape feature vectors.

[0088] The shape similarity between two regions can be calculated based on the following method, specifically including:

[0089] In the first step, according to the formula , obtain the distribution similarity of the th target edge point in the th region of the first image and the second image, where is the shape feature vector of the th target edge point in the th region divided by the first image, is the shape feature vector of the th target edge point in the th region divided by the second image, cos represents the cosine similarity function, is the number of pixel points of the th target edge point in the th region divided by the first image in the th histogram region, is the number of pixel points of the th target edge point in the th region divided by the second image in the th histogram region, is the number of histogram regions after dividing the polar coordinate histogram into regions, is a natural number greater than 1.

[0090] It can be understood that the above formula calculates the similarity between the histogram vectors of the target edge points reflected on the collected images and the Euclidean distance between the two shape feature vectors. For the shape similarity analysis of the same region in two images, the higher the similarity of the single-dimensional corresponding vector features, and the smaller the Euclidean distance calculated by the corresponding histogram, it indicates that the shape features of a certain region in the two images in this dimension have small differences, and the shape similarity of the two regions is high.

[0091] In the second step, according to the formula , obtain the shape similarity of the th region of the first image and the second image, where is the The number of polar histograms in a region.

[0092] This formula is based on the feature performance within a single dimension (the histogram corresponding to one target edge point) analyzed above, and calculates the shape feature vector corresponding to the th region on the th image, and the shape similarity between the shape feature vector corresponding to the th region on the th image. For analyzing the shape similarity performance between an entire region and another region, analyzing only with the features of a single dimension may cause some information to be ignored. Therefore, it is necessary to analyze the shape feature performance of the region by combining the feature performances of multiple dimensions (the histograms corresponding to multiple target edge points). This calculation method can improve the accuracy of calculating the shape similarity between two regions on two images.

[0093] By repeating the above calculation steps, the regional gray values and shape similarities of each region on the first image and the second image can be obtained.

[0094] S12-2. Determine the overlapping regions and non-overlapping regions between the first image and the second image according to the regional gray values and shape similarities. The regional gray value reflects the brightness and darkness of the corresponding region, and the shape similarity can measure the similarity of the region shape. By comparing the gray values and shapes of these regions, find the similar regions in the first image and the second image. These similar regions are the overlapping regions, meaning they correspond to the same or similar parts in the image; while the non-similar regions are the non-overlapping regions, that is, the different parts in the image.

[0095] Sub-step S12-2 includes S12-2-1 and S12-2-4, which are specifically described as follows:

[0096] S12-2-1. Obtain the regional gray value difference according to the difference between the regional gray values of any two regions between the first image and the second image. The regional gray value of the first image is denoted as , and the regional gray value of the second image is denoted as . Taking the difference between the two can obtain the regional gray value difference.

[0097] S12-2-2. Obtain the characteristic matching result of the position of this region according to the regional gray value difference and shape similarity of the regions divided by the first image and the second image. Please refer to Figure 7 and Figure 8 , there are similar regions on the first image and the second image. Figure 7 shows Region 1 and Region 2 divided by the first image. Figure 8Regions 1 and 2 divided from the second image are shown. Whether the two regions are similar can be determined based on the regional gray - level difference and shape similarity of the two regions on the two images, and the similarity degree of the two regions is characterized based on the feature matching result.

[0098] For example, based on the formula , the feature matching result of the two regions is calculated , is the shape similarity of the two regions for comparison between the first image and the second image, is the regional gray - level value of the th region on the first image, is the regional gray - level value of the th region on the second image, is the normalization function.

[0099] It can be understood that the above formula is analyzed and calculated based on the shape similarity and gray - level performance difference of the two regions on the images collected by the binocular near - eye display device. For the overlapping part of the images collected by the device, its shape similarity is high and the gray - level performance similarity is high, indicating that a certain region of the two images collected from two angles is relatively coincident and the regional feature matching is good.

[0100] S12 - 2 - 3. Determine the regions corresponding to all feature matching results greater than the third threshold as the overlapping region between the first image and the second image. The third threshold can be set according to actual needs, as long as it can accurately distinguish the overlapping region of the two images. Based on the calculated results analyzed above, the third threshold can be set. When the calculated , it indicates that the regional feature matching result of the two regions is good. Then select the regions with calculated to construct a data sequence: . Based on the data sequence constructed each time, take the two regions on the two corresponding images of as the corresponding regions, represents the maximum - value function. If the regional feature matching calculated for a certain region on the image collected by the binocular near - eye display device and the region on another image cannot construct a data sequence, it means that there is no corresponding region for this region on the image, and this region is not an overlapping region. Repeat the above steps to complete the matching and recognition of the overlapping regions on the two images.

[0101] S12 - 2 - 4. Determine the image regions of the first image and the second image outside the overlapping region as non - overlapping regions. Please refer to Figure 9, 26 represents the overlapping area, 27 represents the non - overlapping area, 28 represents the binocular near - eye image. In the figure, area a, area b, and area c are all in the overlapping area. The area enveloped by the dashed line is the overlapping area, and the area outside the dashed line is the non - overlapping area.

[0102] So far, the matching recognition of the overlapping parts on the two images collected by the binocular near - eye display device, and the recognition of the non - overlapping parts have been completed.

[0103] S13. For each divided area in the overlapping area and the non - overlapping area, configure the first enhancement coefficient and the second enhancement coefficient respectively.

[0104] Specifically, it is necessary to perform graphic enhancement on the images collected by the binocular near - eye display device based on the recognized image results. However, when people wear the binocular near - eye display device, the degree of attention to different positions on the image is different, so the enhancement degree of images at different positions is also different. Generally speaking, the middle position of the line of sight will greatly attract the user's visual attention (that is, the overlapping part of the two images), and the enhancement degree of the images in the overlapping area should be greater than that of the non - overlapping areas on both sides. The corresponding enhancement coefficients can be configured based on the regional gray values of the overlapping area and the non - overlapping area. For example, input the regional gray value into a preset calculation model, and obtain the first enhancement coefficient of the divided area in the overlapping area and the second enhancement coefficient of the divided area in the non - overlapping area based on the output result of the calculation model.

[0105] In practical applications, since the first enhancement coefficient and the second enhancement coefficient are related to the enhanced display effect of the image, if there are deviations in the configuration of each enhancement coefficient, problems such as abruptness may occur in the overall image display. Therefore, the enhancement degree of the images in the non - overlapping parts on both sides should be analyzed based on the middle overlapping part.

[0106] Exemplarily, the divided area in the overlapping area is the first area, and the divided area in the non - overlapping area is the second area; step S13 includes sub - steps S13 - 1 to S13 - 3, which are specifically described as follows:

[0107] S13 - 1. Perform mean processing on the gray values of all pixel points in each divided area to obtain the average gray value of each divided area. The average value calculation of the pixel point gray values in each divided area can be implemented based on the mean processing model, and the average gray value of each divided area is obtained through the output result of the mean processing model.

[0108] S13 - 2. Obtain the first enhancement coefficient of the current first area according to the number of areas in the first area, the average gray value of the current first area, and the average gray value of the adjacent first area of the current first area.

[0109] Specifically, it can be based on the formula , calculate the average gray value of the pixel points in the overlapping area of the two images , where is the number of overlapping areas on the two images, is the gray value of the th area on the first image.

[0110] Furthermore, analyze the specific information of the th overlapping area on the image to calculate the first enhancement coefficient. Through the formula , calculate the first enhancement coefficient of the th area in the overlapping area , where is the average gray value of the th area adjacent to the th area on the th image, that is, the average gray value of the adjacent first area of the current first area, M is the total number of the first areas adjacent to the current first area.

[0111] It can be understood that repeating the above steps can obtain the image enhancement coefficients of all overlapping areas. The above calculation method analyzes the degree of enhancement of a certain area from its basic conditions, mainly using the difference between the average gray value of a certain area and the average gray value of the overall overlapping part, as well as the difference from the average gray value of the surrounding areas. When the gray value of a certain area on the image collected by the binocular near-eye display device is quite different from the overall gray value and quite different from the gray value of the surrounding areas, it indicates that this area is relatively prominent on the image, and a relatively large degree of image enhancement is performed on it. Please refer to Figure 10 and Figure 11 , the divided areas of the overlapping areas in the figure can be configured with a relatively large first enhancement coefficient based on the above method.

[0112] S13-3. Obtain the second enhancement coefficient of the current second area according to the average gray value of the current second area, the average gray value of the target first area, and the first enhancement coefficient, where the target first area is the first area closest to the current second area.

[0113] Specifically, to prevent the problem of abruptness during image display, the image enhancement coefficient of the th non-overlapping area on the th image can be calculated based on the image enhancement coefficient of the overlapping area.

[0114] For example, according to the formula , calculate the second enhancement coefficient of the current second area , that is, the th area on the The second enhancement coefficient of a non-overlapping region, where is the enhancement coefficient calculated for the first region closest to the th image and the th non-overlapping region, that is, the first enhancement coefficient of the target first region; is the average gray value of the corresponding region with the enhancement coefficient of , that is, the average gray value of the target first region; is the th image and the th non-overlapping region.

[0115] Based on the above formula, the enhancement coefficient of a certain non-overlapping region can be analyzed. Based on the enhancement coefficients of the regions that have been calculated nearby, and then combined with its gray-scale performance for analysis. When the gray value of a certain region is greater than the gray value of the region with a known enhancement coefficient nearby, in order to ensure that the original gray-scale performance law remains unchanged, a certain degree of stronger image enhancement is performed on it.

[0116] Repeat the above steps to obtain the image enhancement coefficients of all regions on the two images. Thus, the image enhancement coefficients of all regions on the two images are obtained.

[0117] S14. According to the first enhancement coefficient and the second enhancement coefficient, perform the current display task of the binocular near-eye display device.

[0118] Specifically, after obtaining the image enhancement coefficients of all regions on the two images, the images can be enhanced based on the enhancement coefficients of each region, and the enhanced two images are used to complete stereo matching, so as to present the stereo image to the user through the binocular near-eye display device.

[0119] Exemplarily, step S14 includes S14-1 to S14-3, which are specifically described as follows:

[0120] S14-1. According to the first enhancement coefficient and the gray value of each pixel point in the overlapping region, determine the first enhanced gray value of each pixel point in the overlapping region. Based on the formula , the first enhanced gray value of the th pixel point in the overlapping region can be calculated , is the gray value of the th pixel point in the overlapping region, is the first enhancement coefficient of the th pixel point, is a natural number greater than 1.

[0121] S14-2. Determine the second enhanced gray value of each pixel in the non-overlapping region according to the second enhancement coefficient and the gray value of each pixel in the non-overlapping region. Similarly, based on the formula , calculate the second enhanced gray value of the -th pixel in the non-overlapping region , is the gray value of the -th pixel in the non-overlapping region, is the second enhancement coefficient of the -th pixel, is a natural number greater than 1.

[0122] It should be noted that when calculating the first enhanced gray value and the second enhanced gray value , normalization processing can be performed based on the actual situation to make the calculated enhanced gray value within the gray value range of 0-255.

[0123] S14-3. Control the binocular near-eye display device to perform stereo matching display according to the first enhanced gray value and the second enhanced gray value. Specifically, an adaptive method and a specific similarity measure factor can be used to complete the stereo matching calculation of the two images and obtain a disparity space image (DSI, Disparity Space Image). An initial disparity image is obtained through two steps of cost aggregation and disparity calculation. Further, by steps such as proposing incorrect disparities, smoothing processing, and sub-pixel accuracy optimization, the stereo matching of the two images is completed. Thus, a depth image can be determined based on the disparity image, and the 3D information of the captured scene can be restored.

[0124] Next, the embodiments of the present invention will comprehensively describe the image detail enhancement method of the binocular near-eye display device. Please refer to Figure 12 , specifically including:

[0125] S901. Image acquisition. The first image is captured by the left camera, and the second image is captured by the right camera.

[0126] S902. Region division. Both the first image and the second image are divided into multiple regions.

[0127] S903. Identification of overlapping regions and non-overlapping regions. Based on the gray feature and shape similarity of the divided regions, the overlapping regions and non-overlapping regions of the first image and the second image are determined.

[0128] S904. Calculation of image enhancement coefficient. Enhancement coefficients are respectively configured for the overlapping regions and non-overlapping regions.

[0129] S905. Image enhancement. Image enhancement is performed on the first image and the second image based on the calculated image enhancement coefficients.

[0130] S906. Stereo matching. A matching process for stereoscopic display is performed on the first image and the second image.

[0131] S907. Presentation by a binocular near-eye display device.

[0132] Based on the same technical concept as the enhancement method, an image detail enhancement system for a binocular near-eye display device is further provided in an embodiment of the present invention. Please refer to Figure 13 , Figure 13 which is a schematic structural diagram of the enhancement system. The enhancement system includes a shooting terminal 101, a processing terminal 102, and a display terminal 103.

[0133] The shooting terminal 101 is configured to obtain a first image and a second image collected by the binocular near-eye display device under the current display task. Among them, the first image and the second image are two images collected at the same moment from different angles in the current display task. The processing terminal 102 is connected to the shooting terminal 101. The processing terminal 102 is configured to divide both the first image and the second image into multiple regions, and determine an overlapping region and a non-overlapping region according to the feature similarity between the regions divided from the first image and the second image. And for each divided region in the overlapping region and the non-overlapping region, a first enhancement coefficient and a second enhancement coefficient are respectively configured. The display terminal 103 is connected to the processing terminal 102. The display terminal 103 is configured to execute the current display task of the binocular near-eye display device according to the first enhancement coefficient and the second enhancement coefficient.

[0134] It should be noted that the shooting terminal 101 may be a shooting device composed of a left camera and a right camera. The processing terminal 102 may be a device composed of a data processing chip. The display terminal 103 may be a display screen capable of performing stereoscopic display.

[0135] The technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:

[0136] The technical solution of the embodiment of the present invention obtains the first image and the second image collected by the binocular near-eye display device under the current display task. Since the first image and the second image are two images collected at the same moment in the current display task from different angles, the first image and the second image can both be divided into multiple regions, and the overlapping region and the non-overlapping region are determined according to the feature similarity between the divided regions of the first image and the second image; for each divided region in the overlapping region and the non-overlapping region, a first enhancement coefficient and a second enhancement coefficient are respectively configured; according to the first enhancement coefficient and the second enhancement coefficient, the current display task of the binocular near-eye display device is executed. This technical solution can configure corresponding enhancement coefficients based on the overlapping region and the non-overlapping region of the binocular near-eye images, can realize more accurate enhancement processing of the binocular near-eye images, make the stereoscopic sense of the binocular near-eye image display stronger, and further improve the display quality of the binocular near-eye images.

[0137] It should be noted that the above sequence of the embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0138] Each embodiment in this specification is described in a progressive manner. The same and similar parts among the embodiments can be referred to each other. The key point of each embodiment is to illustrate the differences from other embodiments.

Claims

1. A method for enhancing image details of a binocular near-eye display device, characterized in that: The method comprises: Acquire a first image and a second image captured by a binocular near-eye display device in a current display task, wherein the first image and the second image are two images captured at different angles at the same time in the current display task; Dividing the first image and the second image into a plurality of regions, and determining overlapping regions and non-overlapping regions according to feature similarities between the regions divided into the first image and the second image; For each divided area in the overlapping area and the non-overlapping area, respectively configure a first enhancement coefficient and a second enhancement coefficient; Execute a current display task of the binocular near-eye display device according to the first enhancement coefficient and the second enhancement coefficient; The determining of overlapping areas and non-overlapping areas according to the feature similarity between the areas divided by the first image and the second image includes: Determine the grayscale value of the region and the shape similarity of each region between the first image and the second image according to the grayscale distribution characteristics of the pixel points of the regions divided by the first image and the second image; Determine an overlapping area and a non-overlapping area between the first image and the second image according to the area grayscale value and the shape similarity; The area divided in the overlapping area is the first area, and the area divided in the non-overlapping area is the second area; for each divided area in the overlapping area and the non-overlapping area, a first enhancement coefficient and a second enhancement coefficient are configured respectively, including: Performing mean processing on the grayscale values ​​of all pixels in each divided area to obtain an average grayscale value of each divided area; Obtaining a first enhancement coefficient of the current first region according to the number of regions of the first region, an average grayscale value of the current first region, and an average grayscale value of first regions adjacent to the current first region; A second enhancement coefficient of the current second region is obtained according to the average gray value of the current second region, the average gray value of the target first region, and the first enhancement coefficient, wherein the target first region is the first region closest to the current second region.

2. The image detail enhancement method of a binocular near-eye display device according to claim 1, characterized in that: Determining the grayscale value of the region according to the grayscale distribution characteristics of the pixels of the region divided by the first image and the second image includes: Obtaining the pixel grayscale, the number of valid pixels and the number of grayscale values ​​of each divided area on the first image and the second image; According to the formula , obtain the Image The gray value of the area ,in, For the Image The gray level of the pixels in the area, For the Image The number of effective pixels in an area, For the Image The number of gray values ​​in a region, is 1 or 2, is a natural number greater than 0, is the normalization function.

3. The image detail enhancement method of a binocular near-eye display device according to claim 1, characterized in that: Determining shape similarities of regions between the first image and the second image according to grayscale distribution characteristics of pixels of regions divided by the first image and the second image includes: Constructing an edge chain code of a corresponding area according to edge pixels of each divided area on the first image and the second image; Determine the angle difference between adjacent pixel lines in each area according to the edge chain code of each divided area, wherein the pixel line is a line connecting the geometric center of the edge chain code and the edge pixel point; Pixel sampling and screening are performed according to the angle difference between adjacent pixel lines in each area to obtain the target edge point of each area; Perform polar coordinate histogram transformation on all target edge points in each region to obtain the shape feature vector of the corresponding region; The shape similarity is obtained according to the similarity between the shape feature vector of the divided area on the first image and the shape feature vector of the divided area on the second image.

4. The image detail enhancement method of a binocular near-eye display device according to claim 3, characterized in that: The pixel sampling and screening is performed according to the angle difference between adjacent pixel lines in each area to obtain the target edge point of each area, including: When the angle difference between the current adjacent pixel lines is greater than a preset first threshold, determining that each edge pixel point corresponding to the current adjacent pixel line is a target edge point; When the angle difference between the current adjacent pixel lines is less than or equal to the first threshold, and the continuous cumulative number is greater than a preset second threshold, a preset number of edge pixel points are screened out from the current adjacent pixel lines as target edge points.

5. The image detail enhancement method of a binocular near-eye display device according to claim 3, characterized in that: The obtaining the shape similarity according to the similarity between the shape feature vector of the area divided on the first image and the shape feature vector of the area divided on the second image comprises: According to the formula , obtain the first image and the second image In the region The distribution similarity of target edge points ,in, The first image is divided into In the region The shape feature vector of the target edge points, The second image is divided into In the region The shape feature vector of the target edge points, cos represents the cosine similarity function, The first image is divided into in the area The target edge point is The number of pixels in a histogram region, The second image is divided into in the area The target edge point is The number of pixels in a histogram region, The number of histogram regions after the polar coordinate histogram is divided into regions; According to the formula , obtain the first image and the second image The shape similarity of the regions ,in, For the The number of polar histograms in the bin.

6. The method for enhancing image details of a binocular near-eye display device according to claim 1, characterized in that: The determining, according to the regional grayscale values ​​and the shape similarity, the overlapping region and the non-overlapping region between the first image and the second image comprises: Obtaining a regional grayscale difference value according to a difference in regional grayscale values ​​between any two regions of the first image and the second image; Obtaining a characteristic matching result of the position of the region according to the regional grayscale difference and shape similarity of the region divided by the first image and the second image; Determine an area corresponding to all characteristic matching results greater than a third threshold as an overlapping area between the first image and the second image; An image area of ​​the first image and the second image outside the overlapping area is determined as the non-overlapping area.

7. The image detail enhancement method of a binocular near-eye display device according to claim 1, characterized in that: The performing a current display task of the binocular near-eye display device according to the first enhancement coefficient and the second enhancement coefficient includes: Determine a first enhanced grayscale value of each pixel in the overlapping area according to the first enhancement coefficient and the grayscale value of each pixel in the overlapping area; Determine a second enhanced grayscale value of each pixel in the non-overlapping area according to the second enhancement coefficient and the grayscale value of each pixel in the non-overlapping area; According to the first enhanced grayscale value and the second enhanced grayscale value, the binocular near-eye display device is controlled to perform stereo matching display.

8. An image detail enhancement system for a binocular near-eye display device, characterized in that: The system comprises: A shooting terminal, used to obtain a first image and a second image captured by a binocular near-eye display device in a current display task, wherein the first image and the second image are two images captured at different angles at the same time in the current display task; a processing terminal connected to the shooting terminal, the processing terminal being used to divide the first image and the second image into a plurality of regions, and determine overlapping regions and non-overlapping regions according to feature similarities between the regions divided by the first image and the second image; and respectively configure a first enhancement coefficient and a second enhancement coefficient for each divided region within the overlapping region and the non-overlapping region; a display terminal connected to the processing terminal, the display terminal being used to execute a current display task of the binocular near-eye display device according to the first enhancement coefficient and the second enhancement coefficient; The determining of overlapping areas and non-overlapping areas according to the feature similarity between the areas divided by the first image and the second image includes: Determine the grayscale value of the region and the shape similarity of each region between the first image and the second image according to the grayscale distribution characteristics of the pixel points of the regions divided by the first image and the second image; Determine an overlapping area and a non-overlapping area between the first image and the second image according to the area grayscale value and the shape similarity; The area divided in the overlapping area is the first area, and the area divided in the non-overlapping area is the second area; for each divided area in the overlapping area and the non-overlapping area, a first enhancement coefficient and a second enhancement coefficient are configured respectively, including: Performing mean processing on the grayscale values ​​of all pixels in each divided area to obtain an average grayscale value of each divided area; Obtaining a first enhancement coefficient of the current first region according to the number of regions of the first region, an average grayscale value of the current first region, and an average grayscale value of first regions adjacent to the current first region; A second enhancement coefficient of the current second region is obtained according to the average gray value of the current second region, the average gray value of the target first region, and the first enhancement coefficient, wherein the target first region is the first region closest to the current second region.

Images