A method and system for generating phase difference between left and right images

Abstract

The application provides a phase difference generation method and system based on left and right images, wherein the method comprises: obtaining a region of interest of left and right reference texture maps, performing grid division to obtain left and right first-level grids, and performing secondary division to obtain left and right second-level subblocks; obtaining texture intensity values of each left and right second-level subblock from the texture maps of each left and right second-level subblock; then classifying each left and right second-level subblock; performing corresponding image preprocessing on each second-level subblock of different classifications to obtain each second-level subblock after preprocessing, and performing spatial position corresponding phase difference calculation on each left and right second-level subblock after preprocessing to obtain the final phase difference of each second-level subblock; and outputting a final phase difference matrix based on the final phase difference of each second-level subblock. The application can cover complex scenes such as sparse texture or dense texture, and ensure the accuracy of the phase difference calculation result.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a method and system for generating phase difference based on left and right images. Background Technology

[0002] Phase Detection Auto Focus (PDAF) technology, as one of the core technologies for achieving fast and accurate focusing in modern optical imaging systems, is widely used in digital cameras, smartphone cameras, and other fields. Its basic principle is to use specific phase-detection pixels (such as masked pixels, dual-core focusing pixels, etc.) in the image sensor to collect phase difference information between the left and right images, and then calculate the defocus distance between the lens and the image plane to drive the focusing motor for precise positioning. The accuracy of the phase difference calculation directly affects focusing speed and image quality, making it a crucial element of the PDAF system.

[0003] In existing technologies, phase difference calculation is mainly based on the traditional ZNCC (Zero-mean Normalized Cross-Correlation) algorithm, which compares and analyzes the left and right images of the original phase detection image output by the image sensor to obtain the phase difference. However, in practical applications, on the one hand, in complex scenarios such as sparse or dense textures, the algorithm is prone to phase difference estimation deviations due to insufficient image features or texture interference in the original phase detection image, and may even get stuck in local extrema, making it difficult to obtain the global optimal solution, thus reducing the calculation accuracy. On the other hand, the single large block strategy is used to perform local region calculations on the original phase detection image, but because the block size is generally large, in scenarios with uneven texture distribution, the single large block cannot effectively adapt to the characteristic differences of different regions, resulting in limited phase difference estimation accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a phase difference generation method and system based on left and right images, which solves the problem of low phase difference calculation accuracy in the prior art due to the inability to adapt to the characteristic differences of different regions in complex scenes and single large blocks.

[0005] The first aspect of this application provides a method for generating phase difference based on left and right images, the method comprising:

[0006] Obtain the left image and extract the reference texture from the left image to obtain the left reference texture map; obtain the right image and extract the reference texture from the right image to obtain the right reference texture map;

[0007] The region of interest (ROI) of the left reference texture map is obtained, and the ROI of the left reference texture map is meshed to obtain a left first-level mesh. The left first-level mesh is then further divided to obtain a left second-level sub-block. The region of interest (ROI) of the right reference texture map is obtained, and the ROI of the right reference texture map is meshed to obtain a right first-level mesh. The right first-level mesh is then further divided to obtain a right second-level sub-block. The relative position of the ROI of the left reference texture map to the left reference texture map is consistent with the relative position of the ROI of the right reference texture map to the right reference texture map.

[0008] Based on the left second-level sub-block, the texture strength value of each left second-level sub-block is calculated and obtained for the texture map of each left second-level sub-block; based on the right second-level sub-block, the texture strength value of each right second-level sub-block is calculated and obtained for the texture map of each right second-level sub-block.

[0009] The left second-level sub-blocks are classified based on the texture intensity values ​​of each left second-level sub-block; the right second-level sub-blocks are classified based on the texture intensity values ​​of each right second-level sub-block; corresponding image preprocessing is performed on each second-level sub-block of different classifications to obtain the preprocessed second-level sub-blocks; and the phase difference corresponding to the spatial position of each preprocessed left second-level sub-block and each right second-level sub-block is calculated to obtain the final phase difference of each second-level sub-block.

[0010] Based on the final phase difference of each second-level sub-block, the final phase difference matrix is ​​output.

[0011] In some embodiments, classifying each left second-level sub-block based on the texture intensity value of each left second-level sub-block; classifying each right second-level sub-block based on the texture intensity value of each right second-level sub-block; further includes:

[0012] Based on the texture intensity values ​​of each of the left second-level sub-blocks, each of the left second-level sub-blocks is divided into at least two categories; based on the texture intensity values ​​of each of the right second-level sub-blocks, each of the right second-level sub-blocks is divided into at least two categories.

[0013] In some implementations, each left second-level sub-block is divided into three categories: low-texture sub-block, medium-texture sub-block, and high-texture sub-block based on the texture intensity value of each left second-level sub-block, and each right second-level sub-block is divided into three categories: low-texture sub-block, medium-texture sub-block, and high-texture sub-block based on the texture intensity value of each right second-level sub-block.

[0014] The step of performing corresponding image preprocessing on each of the second-level sub-blocks of different categories, and calculating the phase difference corresponding to the spatial position of each of the preprocessed left and right second-level sub-blocks, includes:

[0015] The phase difference is calculated using the texture map obtained by extracting strong edges from the original phase detection image for each of the low-texture sub-blocks; the phase difference is calculated using the reference texture map for each of the medium-texture sub-blocks; and the phase difference is calculated using the original phase detection image for each of the high-texture sub-blocks.

[0016] In some implementations, each left second-level sub-block is divided into three categories: low-texture sub-block, medium-texture sub-block, and high-texture sub-block based on the texture intensity value of each left second-level sub-block, and the right second-level sub-block is divided into three categories: low-texture sub-block, medium-texture sub-block, and high-texture sub-block based on the texture intensity value of each right second-level sub-block, including: setting a first threshold and a second threshold, wherein the first threshold is less than the second threshold;

[0017] Each left and right second-level sub-block with a texture intensity value less than the first threshold is marked as a low-texture sub-block; each left and right second-level sub-block with a texture intensity value within the range between the first and second thresholds is marked as a medium-texture sub-block; each left and right second-level sub-block with a texture intensity value greater than the second threshold is marked as a high-texture sub-block, wherein the range includes two critical values: the first threshold and the second threshold.

[0018] In some implementations, the method further includes: based on the texture strength value and a preset third threshold,

[0019] Each left second-level sub-block with a texture strength value less than the third threshold is marked as an abnormal sub-block; each right second-level sub-block with a texture strength value less than the third threshold is marked as an abnormal sub-block, wherein the third threshold is less than the first threshold.

[0020] In some embodiments, after performing corresponding image preprocessing on each of the different categories of second-level sub-blocks to obtain the preprocessed second-level sub-blocks, and calculating the phase difference corresponding to the spatial position of each of the preprocessed left and right second-level sub-blocks to obtain the final phase difference of each second-level sub-block, the method further includes:

[0021] Calculate the phase difference confidence level of each second-level sub-block, and filter the final phase difference of each second-level sub-block based on a preset confidence level threshold.

[0022] In some embodiments, after performing corresponding image preprocessing on each of the different categories of second-level sub-blocks to obtain the preprocessed second-level sub-blocks, and calculating the phase difference corresponding to the spatial position of each of the preprocessed left and right second-level sub-blocks to obtain the final phase difference of each second-level sub-block, the method further includes:

[0023] The final phase difference of each second-level sub-block is filtered based on a preset phase difference threshold.

[0024] In some embodiments, after performing corresponding image preprocessing on each of the different categories of second-level sub-blocks to obtain the preprocessed second-level sub-blocks, and calculating the phase difference corresponding to the spatial position of each of the preprocessed left and right second-level sub-blocks to obtain the final phase difference of each second-level sub-block, the method further includes:

[0025] The phase difference of each second-level sub-block is numerically classified, and a binary spatial location mask matrix corresponding to each first-level grid is generated based on the results of the numerical classification.

[0026] For any first-level grid, the binary spatial location mask matrix corresponding to the first-level grid is multiplied with the expected texture strength value to obtain the weight matrix;

[0027] Based on the weight matrix, the phase difference matrix corresponding to each first-level grid is weighted to obtain the weighted phase difference matrix of each first-level grid, and the final phase difference matrix is ​​obtained based on the weighted phase difference matrix of each first-level grid.

[0028] In some embodiments, after calculating and obtaining the texture strength values ​​of each left second-level sub-block and each right second-level sub-block for the texture maps of each left second-level sub-block and each right second-level sub-block respectively, the method further includes:

[0029] The texture intensity values ​​of each of the left second-level sub-blocks are exponentially amplified; the texture intensity values ​​of each of the right second-level sub-blocks are exponentially amplified.

[0030] The left second-level sub-blocks are classified based on the texture intensity values ​​of each left second-level sub-block after exponential amplification; the right second-level sub-blocks are classified based on the texture intensity values ​​of each right second-level sub-block after exponential amplification.

[0031] A second aspect of this application provides a phase difference generation system based on left and right images, the system comprising:

[0032] The texture extraction module is used to acquire the left image, extract the reference texture from the left image to obtain a left reference texture map; acquire the right image, extract the reference texture from the right image to obtain a right reference texture map;

[0033] The mesh segmentation module is used to obtain the region of interest (ROI) of the left reference texture map, perform meshing on the ROI of the left reference texture map to obtain a left first-level mesh, and perform secondary subdivision on the left first-level mesh to obtain a left second-level sub-block; to obtain the ROI of the right reference texture map, perform meshing on the ROI of the right reference texture map to obtain a right first-level mesh, and perform secondary subdivision on the right first-level mesh to obtain a right second-level sub-block, wherein the relative position of the ROI of the left reference texture map on the left reference texture map is consistent with the relative position of the ROI of the right reference texture map on the right reference texture map;

[0034] The texture strength calculation module is used to calculate and obtain the texture strength value of each left second-level sub-block based on the texture map of each left second-level sub-block; and to calculate and obtain the texture strength value of each right second-level sub-block based on the texture map of each right second-level sub-block.

[0035] The texture differentiation module is used to classify each of the left second-level sub-blocks based on the texture intensity value of each of the left second-level sub-blocks; and to classify each of the right second-level sub-blocks based on the texture intensity value of each of the right second-level sub-blocks.

[0036] The texture phase difference calculation module is used to perform corresponding image preprocessing on each of the second-level sub-blocks of different categories to obtain the preprocessed second-level sub-blocks, and to calculate the phase difference corresponding to the spatial position of each of the preprocessed left second-level sub-blocks and each of the right second-level sub-blocks to obtain the final phase difference of each second-level sub-block;

[0037] The phase difference matrix generation module is used to output the final phase difference matrix based on the final phase difference of each second-level sub-block.

[0038] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0039] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0040] The fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0041] As described above, the phase difference generation method and system based on left and right images described in this application have the following beneficial effects:

[0042] This application divides the baseline texture map into a first-level grid and a second-level sub-block. Then, it classifies the texture regions of each second-level sub-block based on the texture strength value, performs corresponding image preprocessing on each second-level sub-block based on the classification, and calculates the phase difference of each preprocessed second-level sub-block to obtain the final phase difference. This can cover complex scenes such as sparse or dense textures and ensure the accuracy of the phase difference calculation results. Attached Figure Description

[0043] Figure 1 The flowchart shown is an exemplary embodiment of a method for generating phase difference based on left and right images.

[0044] Figure 2 The diagram shown is an exemplary embodiment of the original phase detection image after vertical texture extraction.

[0045] Figure 3 The flowchart shows a phase difference generation method based on left and right images, provided as another exemplary embodiment.

[0046] Figure 4 The flowchart shows a phase difference generation method based on left and right images, provided as another exemplary embodiment.

[0047] Figure 5 The flowchart shows a phase difference generation method based on left and right images, provided as another exemplary embodiment.

[0048] Figure 6 The flowchart shows a phase difference generation method based on left and right images, provided as another exemplary embodiment.

[0049] Figure 7 The diagram shows an architecture of a phase difference generation system based on left and right images, provided as another exemplary embodiment.

[0050] Figure 8 The diagram shows the structure of an electronic device. Detailed Implementation

[0051] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0052] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0053] In existing technologies, traditional ZNCC algorithms rely on pixel grayscale values ​​to calculate correlation. When signal strength is insufficient (meaning there is a lack of sufficient effective information in local image regions to calculate phase difference), it can easily lead to phase difference estimation errors, thus affecting focusing accuracy. In sparsely textured regions, such as solid-color walls or smooth object surfaces, the number of available feature points for phase detection pixels decreases, making it difficult for the ZNCC algorithm to find sufficiently reliable matching pairs. In this case, phase difference calculation is prone to getting trapped in local extrema, leading to misjudgment of the focusing direction or a decrease in focusing speed. In complex textured scenes, such as dense patterns or repetitive structures, multiple similar features may exist in local regions, causing "pseudo-matching" phenomena during ZNCC calculation. In addition, the complex light reflection in highly textured regions may introduce additional phase difference noise, further reducing calculation accuracy.

[0054] In current technologies, mainstream algorithms typically employ a single large block strategy for local region computation. However, the block size is generally large, making it difficult to balance global consistency with local detail differences. In scenarios with uneven texture distribution, a single large block cannot effectively adapt to the characteristics of different regions, resulting in limited accuracy in phase difference estimation.

[0055] In summary, although significant progress has been made in PD phase difference calculation technology, its accuracy in complex scenes remains a bottleneck. Existing ZNCC-based algorithms have failed to effectively address calculation errors caused by differences in lighting and texture, and the single large-block strategy is ill-suited to diverse shooting environments. Therefore, there is an urgent need for a PD phase difference algorithm that can dynamically optimize the calculation strategy based on real-time texture intensity to improve focusing accuracy in different scenarios.

[0056] The following embodiments of this application provide a method, system, device, medium, and program product for generating phase difference based on left and right images. This application divides the reference texture map extracted from the original phase detection image into grids and sub-blocks, classifies each sub-block based on the texture strength value, and calculates the phase difference of each classified sub-block. This achieves accurate calculation of the phase difference value of PD blocks in different scenarios, solving the problems in the prior art that fail to effectively solve the calculation error caused by differences in lighting and texture, and that the single large block strategy is difficult to adapt to diverse shooting environments.

[0057] The technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0058] An exemplary embodiment of this application provides a method for generating phase difference based on left and right images, such as... Figure 1 As shown, in this embodiment, the method includes steps 101 to 105, wherein:

[0059] Step 101: Obtain the left image and extract the reference texture from the left image to obtain the left reference texture map; obtain the right image and extract the reference texture from the right image to obtain the right reference texture map.

[0060] In some embodiments, the left image is captured by the left lens of the binocular camera and the right image is captured by the right lens; the left and right lenses of the binocular camera are at the same horizontal level and their optical axes are parallel. The left and right lenses are triggered synchronously during acquisition to ensure that the left and right images are scene images of the same moment and the same field of view.

[0061] In some embodiments, the left and right images may be obtained by taking consecutive pictures of the same object with a monocular camera.

[0062] In some embodiments, the left and right images may be images of the same object captured by a trinocular or more cameras.

[0063] In some embodiments, left and right images can be captured simultaneously by two cameras; the lenses of the two cameras are at the same horizontal level and their optical axes are parallel, ensuring that the left and right images are scene images of the same moment and the same field of view.

[0064] Specifically, extracting the baseline texture from the left image refers to extracting basic texture information that can characterize the image region, such as edges, details, and grayscale variation patterns, from the left image to form a left baseline texture map. Extracting the baseline texture from the right image refers to extracting basic texture information that can characterize the image region, such as edges, details, and grayscale variation patterns, from the right image to form a right baseline texture map. The left baseline texture map is used to characterize the basic texture information of the left image, and the right baseline texture map is used to characterize the basic texture information of the right image.

[0065] In some embodiments, the methods for extracting reference textures from the left and right images respectively include, but are not limited to, gradient operator-based edge texture extraction methods, such as: Sobel operator, Prewitt operator, Canny operator; texture statistical feature extraction methods based on gray-level co-occurrence matrix (GLCM); multi-scale texture extraction methods based on wavelet transform; and texture feature extraction methods based on local binary pattern (LBP).

[0066] Preferably, this embodiment uses the Sobel operator to extract the vertical reference texture from the left and right images respectively. In practical applications, those skilled in the art can adjust the coefficients of the Sobel operator's convolution kernel to control the texture extraction intensity according to actual needs, so as to make the phase difference calculation results more accurate.

[0067] It should be noted that in this embodiment, the Sobel operator with the same convolution kernel coefficients is used to extract the reference texture in the vertical direction for the left and right images respectively, so as to ensure that the texture features of the extracted left and right reference texture maps are the same, and to prevent the difference in texture features between the left and right reference texture maps from affecting the classification accuracy of subsequent blocks, and thus affecting the accuracy of the final phase difference calculation.

[0068] For example, in this embodiment, standard Sobel convolution kernel coefficients can be used to extract textures in the vertical direction for the left and right images respectively. If the Sobel operator's convolution kernel is a 3×3 convolution kernel, the vertical Sobel convolution kernel is used to process the left and right images respectively. The vertical convolution kernel can be represented as [-1,-2,-1;0,0,0;1,2,1].

[0069] This embodiment uses the Sobel operator to extract the vertical reference texture from the left and right images respectively, which can effectively extract features such as vertical lines and columnar structures, so as to obtain more accurate phase difference data by aligning the texture edges during subsequent phase difference calculation. Figure 2 The image shown is a schematic diagram of the texture map after extracting the reference texture in the vertical direction from the left image (or right image).

[0070] This embodiment uses a reference texture map for phase difference calculation, that is, it uses the image after edge extraction to calculate the phase difference. This not only preserves the texture structure of the left and right images, but also effectively filters out areas in the left and right images with weak local gray-level changes and gradients close to zero. This reduces the complexity of phase difference calculation and increases the accuracy of the phase difference calculation results. It solves the problem in the prior art that directly uses the left and right images for phase difference calculation, and performs the same calculation on areas in the left and right images with weak local gray-level changes and gradients close to zero, which increases the complexity of phase difference calculation and affects the accuracy of the phase difference calculation results.

[0071] The above-mentioned method of selecting a benchmark texture extraction method suitable for the scene, combined with parameter optimization, hardware parallel acceleration and multi-feature fusion strategy, can achieve high efficiency and high accuracy in extracting the texture features of the benchmark texture map, providing a high-quality texture benchmark for subsequent texture weight calculation and phase detection value selection.

[0072] Step 102: Obtain the region of interest (ROI) of the left reference texture map, perform meshing on the ROI of the left reference texture map to obtain the first-level left mesh, and perform secondary meshing on the first-level left mesh to obtain the second-level left sub-blocks; Obtain the ROI of the right reference texture map, perform meshing on the ROI of the right reference texture map to obtain the first-level right mesh, and perform secondary meshing on the first-level right mesh to obtain the second-level right sub-blocks; The relative position of the ROI of the left reference texture map on the left reference texture map is consistent with the relative position of the ROI of the right reference texture map on the right reference texture map.

[0073] In some embodiments, the region of interest (ROI) refers to a region in an image that possesses specific texture features and requires focused analysis or processing. Typically, the ROI is set as a region near the center of the texture map that contains specific texture features. The ROI is divided according to specific engineering requirements. For example, in a live-streaming sales scenario, when a salesperson in the live-streaming room is explaining a product, the area containing the product in the live-streaming frame is the key area for analysis; therefore, the area including the product is identified as the ROI. Generally, the ROI area is set to 40%-60% of the total area of ​​the reference texture map, but this can be adjusted according to the actual needs of the technical personnel. For example, the ROI of the left and right reference texture maps is divided into N×M left first-level grids and N×M right first-level grids, respectively. Each left grid cell and each right grid cell are then further divided to obtain I×J left second-level sub-blocks and I×J right second-level sub-blocks. The values ​​of N, M, I, and J can be the same or different, depending on specific engineering requirements. For example, N and M are both 5, and I and J are both 3. The relative position of the region of interest in the left reference texture map is consistent with the relative position of the region of interest in the right reference texture map. That is, the two regions of interest in the left and right images can be matched one-to-one, so that effective phase difference calculation can be performed subsequently.

[0074] In this embodiment, specifically, the region of interest (ROI) of the left reference texture map is obtained, and the ROI is evenly divided into a 5×5 grid (25 grids in total). This 5×5 grid is the first-level left grid, and the size of each first-level left grid is 384×216 pixels (1920 / 5=384, 1080 / 5=216). Generally, the texture features of the corners and edges of the reference texture map are weak and the noise is high. Therefore, this embodiment first obtains the ROI of the left reference texture map, and then divides the ROI into grids, which improves the detection accuracy of areas with strong texture features in the left reference texture map and avoids redundant calculations for all areas of the entire image. Each first-level left grid is further divided into 3×3 sub-blocks as second-level left sub-blocks, for a total of 9 second-level left sub-blocks, each with a size of 128×72 pixels (384 / 3=128, 216 / 3=72).

[0075] Similarly, the same method is applied to the right reference texture map to ensure that the pixel size of the block regions at corresponding spatial positions in the left and right reference texture maps is the same, thus preventing the difference in pixel size between the block regions of the left and right reference texture maps from affecting the accuracy of the final phase difference calculation.

[0076] This embodiment obtains the region of interest from the reference texture map and performs a two-level division of the texture map of the region of interest into a first-level grid and a second-level sub-block. This effectively eliminates small blocks with weak texture features or high noise in the grid, which serve as the input basis for subsequent phase difference calculation. This greatly improves the accuracy of phase detection calculation and avoids the problem of wasting computational resources in high-texture areas and generating incorrect phase differences in low-texture areas due to noise interference caused by global uniform processing due to the strength and uneven distribution of image texture.

[0077] Step 103: Based on the left second-level sub-block, calculate and obtain the texture strength value of each left second-level sub-block for the texture map of each left second-level sub-block; based on the right second-level sub-block, calculate and obtain the texture strength value of each right second-level sub-block for the texture map of each right second-level sub-block.

[0078] Specifically, this embodiment takes a single left second-level sub-block as an example. For the texture map of a single left second-level sub-block, the method for calculating and obtaining the texture strength value representing the texture features of the left second-level sub-block is as follows: calculate the local contrast and local standard deviation of the left second-level sub-block, and multiply the local contrast and local standard deviation of the left second-level sub-block to obtain the texture strength value representing the texture features of the left second-level sub-block, which is expressed by the formula:

[0079]

[0080] S represents the texture strength value of a single left second-level sub-block. C represents the local standard deviation of the second-level left sub-block, and C represents the local contrast of the second-level left sub-block.

[0081] The local standard deviation of the second-left sub-block is the standard deviation of all pixels in the texture map of that second-left sub-block, which can be calculated using the standard deviation function. For example, this can be achieved using Python code: `local std=np.std(img block)`. This represents the local standard deviation of the left second-level sub-block. `np.std()` is the function used to calculate the standard deviation, and `img block` contains all the pixels in the texture map of this left second-level sub-block. Calculating the local standard deviation of the left second-level sub-block is a classic method for measuring texture complexity. It is simple, effective, and computationally inexpensive, and can reflect the fluctuation of pixel values ​​within the local image region of each left second-level sub-block. For example, if a left second-level sub-block has high texture features, the corresponding pixel values ​​of that left second-level sub-block will vary significantly, resulting in a larger local standard deviation for that left second-level sub-block.

[0082] The local contrast of a single left second-level sub-block is calculated using the following formula:

[0083]

[0084] C represents the local contrast of a single left second-level sub-block, I i For all pixels covered by the texture map of the second-level left sub-block i, This represents the maximum pixel value of the texture map of the second-to-left sub-block i. This represents the minimum pixel value of the texture map of the second-to-left sub-block i. For smoothing terms, a minimum value is typically taken; for example, Pick .

[0085] For example, the local contrast data of a single left second-level sub-block can be obtained using the following Python code: local contrast=(np.max(img block)-np.min(img block)) / (np.max(img block)+np.min(img block)+1e-6), smoothing term This is to avoid a denominator of 0. For example, if the texture map of a certain second-left sub-block is completely black, then the maximum and minimum pixel values ​​of the texture map of that second-left sub-block are both 0. If there is no smoothing term... At this point, the denominator will be incorrect, and a smoothing term will be added. This avoids situations where the denominator is 0. The local contrast of the second-left sub-block reflects the brightness differences within the image block, which is particularly effective for edge textures. Second-left sub-blocks with higher local contrast usually correspond to texture maps with more pronounced edges or details, effectively distinguishing different texture regions.

[0086] This embodiment calculates the local standard deviation and local contrast of the pixels contained in the texture map of each left second-level sub-block, and multiplies the calculated local standard deviation and local contrast of each left second-level sub-block to obtain the texture strength value of each left second-level sub-block. This texture strength value can simultaneously consider the local changes and edge information of the texture map of each left second-level sub-block. It can be understood that the larger the local standard deviation and local contrast of the left second-level sub-block, the larger the texture strength value calculated for that left second-level sub-block. If the local standard deviation and local contrast of the left second-level sub-block are both high, this calculation method can increase the weight of the left second-level sub-block with stronger texture features in the subsequent phase difference calculation. Therefore, calculating the texture strength value can more intuitively reflect the richness of texture features in the texture map of each left second-level sub-block.

[0087] For example, if the pixel matrix of the texture map of a certain left second-level sub-block is: Then, the local contrast C of the corresponding left second-level sub-block is calculated using the above calculation method: C = (9-1) / (9+1+10).-6 )=0.8; Local standard deviation of the left second-level sub-block 2.58; therefore, the texture strength value S of the second-level left sub-block is 0.8 * 2.58. 2.06, which is the texture strength value. The texture strength values ​​of all left second-level sub-blocks are calculated using the method described above.

[0088] Similarly, in this embodiment, the texture strength value of a single right second-level sub-block is calculated in the same way as the texture strength value of a single left second-level sub-block. Assume the pixel matrix of the texture map of a certain right second-level sub-block is: The texture strength value of the second-level right sub-block is 1.73.

[0089] Step 104: Classify each left second-level sub-block based on the texture intensity value of each left second-level sub-block; classify each right second-level sub-block based on the texture intensity value of each right second-level sub-block; perform corresponding image preprocessing on each second-level sub-block of different classifications to obtain preprocessed second-level sub-blocks, and calculate the phase difference corresponding to the spatial position of each preprocessed left second-level sub-block and each right second-level sub-block to obtain the final phase difference of each second-level sub-block.

[0090] Specifically, based on the texture strength values ​​of the left second-level sub-blocks, the left second-level sub-blocks are classified, including: dividing the left second-level sub-blocks into at least two categories based on the texture strength values, so as to facilitate refined phase difference calculation for each classified left second-level sub-block.

[0091] More specifically, based on the calculated texture strength values ​​of all left second-level sub-blocks, each left second-level sub-block is divided into three categories: low-texture sub-blocks, medium-texture sub-blocks, and high-texture sub-blocks. This includes: setting a first threshold and a second threshold, wherein the first threshold is less than the second threshold; marking left second-level sub-blocks with texture strength values ​​less than the first threshold as low-texture sub-blocks; marking left second-level sub-blocks with texture strength values ​​within the range between the first and second thresholds as medium-texture sub-blocks, wherein the range includes both the first and second thresholds; and marking left second-level sub-blocks with texture strength values ​​greater than the second threshold as high-texture sub-blocks.

[0092] High-texture sub-blocks generally refer to left-hand second-level sub-blocks with drastic pixel value changes and obvious edges, corners, or complex patterns, such as object outlines, building corners, leaf stripes, and fingerprint stripes. Medium-texture sub-blocks generally refer to left-hand second-level sub-blocks with moderate pixel value changes, but not as drastic as high-texture sub-blocks, such as slowly changing shadows and rough but uniform sandpaper surfaces. Low-texture sub-blocks are left-hand second-level sub-blocks with small pixel value changes and lack significant grayscale changes and detailed information, such as out-of-focus backgrounds, watermarks with slight creases or low contrast, and uniform muscle tissue.

[0093] For example, the first threshold and the second threshold can be customized based on the needs of those skilled in the art.

[0094] For example, if the first threshold is set to 1 and the second threshold is set to 5, then left second-level sub-blocks with texture strength values ​​less than 1 are marked as low-texture sub-blocks, left second-level sub-blocks with texture strength values ​​in the range of 1 to 5 (inclusive) are marked as medium-texture sub-blocks, and left second-level sub-blocks with texture strength values ​​greater than 5 are marked as high-texture sub-blocks. That is, if the texture strength value of the left second-level sub-block calculated in the above embodiment is 2.06, which is in the range of 1 to 5, then the left second-level sub-block is marked as a medium-texture sub-block.

[0095] Similarly, the same processing is applied to the second-level right sub-block, and the processing procedure and effect are the same, so it will not be described in detail here.

[0096] Further, corresponding image preprocessing is performed on the second-level sub-blocks of different categories to obtain preprocessed second-level sub-blocks; then, the phase difference corresponding to the spatial position of the preprocessed left and right second-level sub-blocks is calculated to obtain the final phase difference of each second-level sub-block. The corresponding image preprocessing for the second-level sub-blocks of different categories includes:

[0097] The phase difference is calculated using the texture map obtained after strong edge extraction from the original phase detection image for each low-texture sub-block; the phase difference is calculated using the reference texture map for each medium-texture sub-block; and the phase difference is calculated using the original phase detection image for each high-texture sub-block. The original phase detection image is the original image acquired by the camera or video recording device, i.e., the left or right image mentioned in the above embodiments.

[0098] Specific methods for strong edge extraction of each low-texture sub-block using the original phase detection image include, but are not limited to, gradient operator-based edge texture extraction methods, such as the Sobel operator, Prewitt operator, and Canny operator; texture statistical feature extraction methods based on the gray-level co-occurrence matrix (GLCM); and multi-scale texture extraction methods based on wavelet transform. In some embodiments, the Sobel operator is used to extract strong edges in the vertical direction from the original phase detection image of the corresponding region of the low-texture sub-block. Furthermore, unlike the process of extracting a reference texture image from the original phase detection image, this scheme preprocesses the low-texture sub-block image to fully explore and utilize the effective texture features and detail information of the low-texture sub-block, thereby improving the phase difference calculation accuracy of the low-texture sub-block. Therefore, compared to extracting a reference texture image from the original phase detection image, the intensity of strong edge extraction based on the original phase detection image is higher than that of edge extraction based on the reference texture image. For example, the Sobel operator is used to extract strong edges from low-texture sub-blocks. Enhancing edge extraction strength can be achieved by amplifying the coefficients of the Sobel operator's convolution kernel. For instance, the coefficients of the vertical convolution kernel can be amplified by three times, i.e., the convolution kernel for vertical texture extraction in the original phase detection image is adjusted to [-3, -6, -3; 0, 0, 0; 3, 6, 3]. Using the Sobel operator for strong vertical edge extraction in the original phase detection image can effectively extract features such as vertical lines and columnar structures, more clearly delineating image boundaries, further improving the grayscale variations and detail information of low-texture sub-blocks, and enhancing the texture saliency of the low-texture sub-block edges.

[0099] For each low-texture sub-block, the enhanced texture map obtained by further strong edge extraction of the original phase detection image is used as the basis for subsequent phase difference calculation. This can improve the grayscale variation and detail information of the low-texture sub-block, significantly enhance the texture saliency of the low-texture sub-block edges, and avoid the changes in texture frequency distribution and uncontrollability of the effect that may be caused by directly performing strong edge extraction on the reference texture map.

[0100] The phase difference is calculated using a reference texture map for each medium texture sub-block to avoid distortion caused by over-processing when extracting strong edges of the medium texture sub-block, and to avoid noise interference affecting the accuracy of phase difference calculation when directly using the original phase detection image to calculate the phase difference.

[0101] The phase difference of each high-texture sub-block is calculated directly using the original phase detection image. The original phase detection image corresponding to the high-texture sub-block itself has obvious feature information. If the texture map is obtained by edge extraction, its texture information is further enhanced, but at the same time, some other feature information, such as brightness changes in smooth areas, will be weakened. In order to retain the significant feature information of the high-texture sub-block itself, this scheme directly uses the original phase detection image to calculate the phase difference of each high-texture sub-block. This allows the local structural information of the high-frequency texture region to be preserved when calculating the phase difference of the high-texture sub-block, which can improve the calculation accuracy of the phase difference of the high-texture sub-block.

[0102] This embodiment classifies the left and right second-level sub-blocks according to the texture strength values, achieving accurate division of different texture regions. It then performs corresponding image preprocessing on the second-level sub-blocks of different classifications and implements differentiated processing for different texture regions. This fully considers the differences in image features of the second-level sub-blocks in different texture regions and performs different image preprocessing on the second-level sub-blocks of different classifications based on these differences. It fully utilizes the effective feature information of the second-level sub-blocks in different texture regions, achieving refined phase difference calculation based on adaptive texture information. This greatly improves the accuracy and precision of phase difference calculation, outputting a high-accuracy phase difference even in scenarios with insufficient signal or complex textures. The technical solution of this application solves the shortcomings of existing technologies where applying the same processing to all textures within a large single block leads to low accuracy in phase difference calculation, and the problem that phase difference calculation is difficult to adapt to diverse shooting environments.

[0103] Furthermore, based on the texture strength value and the preset third threshold, each second-level sub-block is further divided into abnormal sub-blocks. Abnormal sub-blocks are second-level sub-blocks with texture strength values ​​less than the third threshold, where the third threshold is less than the first threshold, and abnormal sub-blocks do not participate in phase difference calculation.

[0104] Abnormal sub-blocks are the second-level sub-blocks on the left that do not require focused analysis or processing. For example, an abnormal sub-block might be a blank area of ​​paper, an air region in a CT image, or completely blurred leaf stripes. The third threshold can be customized based on the needs of those skilled in the art, and will not be described in detail here.

[0105] For example, if the third threshold is set to 0.3, then second-level sub-blocks with texture strength values ​​less than 0.3 are classified as abnormal sub-blocks. For instance, if the texture strength value of a second-level sub-block is 0.1, then this second-level sub-block is classified as an abnormal sub-block, and abnormal sub-blocks do not participate in phase difference calculation. This embodiment sets a third threshold to remove or delete sub-blocks with almost no texture features, preventing them from participating in phase difference calculation, saving system computing resources, preventing noisy sub-blocks from participating in subsequent phase difference calculations, and improving the accuracy of phase difference calculation.

[0106] For each left and right second-level sub-block after texture region classification and corresponding image preprocessing, the phase difference corresponding to the spatial location is calculated to obtain the final phase difference of each second-level sub-block. Specifically, the phase difference corresponding to the spatial location can be calculated using a zero-mean normalized cross-correlation algorithm.

[0107] Specifically, the phase difference is calculated using a zero-mean normalized cross-correlation algorithm for each individual left and right second-level sub-block corresponding to its spatial location, yielding the final phase difference for each second-level sub-block. The individual left and right second-level sub-blocks are obtained after the aforementioned texture region classification and corresponding image preprocessing; their participation in the phase difference calculation is based on the preprocessed image. The specific steps for calculating the phase difference using the zero-mean normalized cross-correlation algorithm are as follows:

[0108] Vectorization: Taking a single second-level sub-block as an example, all pixels contained in the single second-level sub-block are extracted row by row to form a one-dimensional vector of the single second-level sub-block. The same vectorization process is applied to both the single left and single right second-level sub-blocks corresponding to spatial positions. For example, if a left second-level sub-block and its corresponding right second-level sub-block are both 3×3 pixel matrices, the left second-level sub-block is first extracted from left to right, along with the first row of all pixels, then the second row, and so on, until all rows of pixels are extracted, forming a one-dimensional vector of the left second-level sub-block; similarly, the corresponding right second-level sub-block is extracted from left to right, along with the first row of all pixels, then the second row, and so on, until all rows of pixels are extracted, forming a one-dimensional vector of the right second-level sub-block.

[0109] For example, assuming the second-level left sub-block is classified as a medium-texture sub-block, the pixel matrix after image preprocessing corresponding to the medium-texture sub-block is: Then, the one-dimensional vector of the left second-level sub-block after vectorization is [1,2,3,4,5,6,7,8,9]; correspondingly, the right second-level sub-block should also be a medium-texture sub-block, and the pixel matrix after image preprocessing corresponding to the medium-texture sub-block is: The one-dimensional vector of the right second-level sub-block after vectorization is [2,3,4,5,6,7,8,9,10].

[0110] Zero-mean normalization: Based on the one-dimensional vector of the second-left sub-block described above, calculate the mean of the one-dimensional vector of the second-left sub-block, mean_left: ,in, This is a vector index, with values ​​from 1 to n. The one-dimensional vector of the second-level sub-block on the left There are n pixel values, where n is the vector length.

[0111] Based on the one-dimensional vector of the right second-level sub-block described above, calculate the mean of the one-dimensional vector of the right second-level sub-block: ,in, The one-dimensional vector of the right second-level sub-block Each pixel value.

[0112] Based on the mean of the one-dimensional vector of the left second-level sub-block, generate a zero-mean left vector F, F( i )= .

[0113] Based on the mean of the one-dimensional vector of the right second-level sub-block, generate a zero-mean right vector T: T( i )= .

[0114] For example, mean_left = (1+2+3+4+5+6+7+8+9) / 9 = 45 / 9 = 5.

[0115] mean_right=(2+3+4+5+6+7+8+9+10) / 9=54 / 9=6.

[0116] F=[1-5,2-5,3-5,4-5,5-5,6-5,7-5,8-5,9-5]=[-4,-3,-2,-1,0,1,2,3,4].

[0117] T=[2-6,3-6,4-6,5-6,6-6,7-6,8-6,9-6,10-6]=[-4,-3,-2,-1,0,1,2,3,4].

[0118] In this embodiment, zero-mean processing is performed on the left and right second-level sub-blocks respectively to eliminate the overall brightness shift between the left and right second-level sub-block vectors. For example, the left second-level sub-block is generally darker and the right second-level sub-block is generally brighter. This makes the mean of both vectors 0, retaining only the relative change features of pixel values, and avoiding the interference of brightness differences with subsequent similarity calculations.

[0119] Normalization: Calculate the norm_F of the left vector F after zero-mean normalization: .

[0120] Calculate the norm_T of the right vector T after zero-mean normalization: .

[0121] Generate the normalized left vector F_normF: .

[0122] Generate the normalized right vector T_normT: .

[0123] For example: .

[0124] .

[0125] (-0.516,-0.387,-0.258,-0.129,0,0.129,0.258,0.387,0.516).

[0126] (-0.516,-0.387,-0.258,-0.129,0,0.129,0.258,0.387,0.516).

[0127] This embodiment normalizes the zero-mean left vector F and the zero-mean right vector T, eliminating the contrast differences between the left and right second-level sub-block vectors. For example, the left second-level sub-block has a large pixel value variation, while the right second-level sub-block has a small pixel value variation. This makes the L2 norm of both the zero-mean left vector F and the zero-mean right vector T equal to 1, unifying the "scale" of the vectors and ensuring that subsequent cross-correlation calculations only reflect the similarity of the vector "shape".

[0128] Cross-correlation calculation: Based on the preset displacement, the cross-correlation value is calculated between the normalized left vector and the normalized right vector. The calculation formula is as follows:

[0129] ;

[0130] Where `lag` is the preset displacement, representing the preset offset direction and magnitude of the right second-level sub-block relative to the left second-level sub-block. `lag` takes an integer between -(n-1) and +(n-1), including the two critical values ​​of -(n-1) and +(n-1). `||` represents the absolute value, ensuring the cross-correlation value is non-negative. The summation range must satisfy the condition that the vector index does not exceed the bounds. and .

[0131] For each preset displacement lag, calculate its corresponding cross-correlation value. That is, within the range of -(n-1) to +(n-1), different values ​​of lag are taken, and the cross-correlation value of the normalized left vector and the normalized right vector is calculated under different lag values ​​to characterize the degree of closeness of the corresponding lag value to the actual displacement of the right second-level sub-block relative to the left second-level sub-block.

[0132] It should be noted that a positive lag indicates that the right second-level sub-block is offset to the right relative to the left second-level sub-block, a negative lag indicates that the right second-level sub-block is offset to the left relative to the left second-level sub-block, and lag=0 indicates that the right second-level sub-block has no offset, that is, the spatial position of the right second-level sub-block relative to the left second-level sub-block remains unchanged.

[0133] For example, when lag=0 (no offset): the cross-correlation calculation sums over the range i=1~9:

[0134] =∣F_normF[1]×T_normT[1]+F_normF[2]×T_normT[2]+...+F_normF[9]×T_normT[9]∣

[0135] Since F_normF = T_normT, therefore: Therefore, it can be concluded that the left second-level sub-block and the right second-level sub-block have the highest similarity at this time, that is, the lag value has no deviation, and the spatial position of the right second-level sub-block relative to the left second-level sub-block has no change.

[0136] When lag=+1, the right second-level sub-block is shifted 1 bit to the right: the cross-correlation calculation sums the range i=1~8; because after shifting 1 bit to the right, only the first 8 elements of this right second-level sub-block can match its corresponding left second-level sub-block: =∣F_normF[2]×T_normT[1]+F_normF[3]×T_normT[2]+...+F_normF[9]×T_normT[8]∣;

[0137] Substituting the numerical values, we get: F_normF[2]×T_normT[1]=(−3 / 60)×(−4 / 60)=12 / 60;

[0138] F_normF[3]×T_normT[2]=(−2 / 60)×(−3 / 60)=6 / 60;

[0139] …

[0140] F _ normF [9]× T _ normT [8] = (4 / 60) × (3 / 60) = 12 / 60;

[0141] Summing up, we get: (12+6+2+0+0+2+6+12) / 60=50 / 60≈0.833.

[0142] Therefore ≈0.833

[0143] When lag = -1, the right second-level sub-block is shifted 1 bit to the left: the cross-correlation calculation summation range is i = 1~8. Because after shifting 1 bit to the left, only the last 8 elements of this right second-level sub-block can match its corresponding left second-level sub-block: =∣F_normF[1]×T_normT[2]+F_normF[2]×T_normT[3]+...+F_normF[8]×T_normT[9]∣;

[0144] The calculation logic is the same as lag=+1, and the result is... ≈0.833.

[0145] In this embodiment, the similarity between F_normF and T_normT, i.e., the cross-correlation value between F_normF and T_normT, is calculated under different displacements, i.e., different lags. Different lag values ​​are taken sequentially within the range of -(n-1) to +(n-1), and the corresponding cross-correlation values ​​are calculated. This forms a cross-correlation sequence c with a sequence length of 2n-1. The higher the similarity, the closer the cross-correlation value is to 1, and the more likely the corresponding lag value is to be the true displacement of the left second-level sub-block and the right second-level sub-block.

[0146] Determine the position of the maximum cross-correlation value: In the cross-correlation sequence c, determine the position of the maximum cross-correlation value, denoted as peak_idx, and the corresponding lag is denoted as lag_initial.

[0147] The least squares method is used for fitting and optimization to calculate the optimal phase difference, which is then taken as the final phase difference. Based on the cross-correlation sequence c, with peak_idx as the center, two adjacent positions are selected, resulting in three points: peak_idx-1, peak_idx, and peak_idx+1. The lag values ​​x1, x2, and x3 of these three points, along with their corresponding cross-correlation values ​​y1=c[peak_idx-1], y2=c[peak_idx], and y3=c[peak_idx+1], are obtained respectively. For example, assuming the lag offsets corresponding to the position of the maximum cross-correlation value and its two adjacent positions are x2=0, x1=-1, and x3=+1, a quadratic function y=ax is fitted using the least squares method. 2 +bx+d, solve for the fitting parameters a, b, and d using the least squares method, and construct a system of equations:

[0148] y1=a(-1) 2 +b(-1)+d;

[0149] y2=a(0) 2 +b(0)+d;

[0150] y3 = a(+1) 2+b(+1)+d;

[0151] The parameters obtained by solving are: d=y2, a=(y1+y3−2y2) / 2, b=(y3−y1) / 2.

[0152] Calculate the vertex of the quadratic function, i.e. the optimal offset: we know that the x-coordinate of the vertex of the quadratic function is xopt=−b / (2a), then the optimal phase difference PD is: PD=lag_initial+xopt;

[0153] For example, in the cross-correlation sequence c, the maximum cross-correlation value peak_idx is determined, and the lag_initial corresponding to peak_idx is 0. If the value is 1, then select two adjacent positions of peak_idx, peak_idx-1 and peak_idx+1, with corresponding lag values ​​of -1 and 1 respectively, and the corresponding cross-correlation values... and All three points have a lag value of 0.833. Based on the positions of these three points, a quadratic function is fitted using the lag value as the independent variable and the cross-correlation value corresponding to the lag value as the dependent variable. That is, the quadratic function y=ax is fitted using the three points (-1, 0.833), (0, 1), and (1, 0.833). 2 Given +bx+d, we can calculate a=(0.833+0.833−2×1.0) / 2=(1.666−2) / 2=−0.334 / 2≈−0.167, b=(0.833−0.833) / 2=0.

[0154] The x-coordinate of the vertex of the quadratic function is: xopt=−b / (2a)=−0 / (2×(−0.167))=0, and the optimal phase difference PD is: PD=lag_initial+xopt=0+0=0. In this embodiment, based on the one-dimensional vector of length n corresponding to the left second-level sub-block and the right second-level sub-block, values ​​are sequentially taken within the preset lag range [-(n-1),+(n-1)] and the corresponding cross-correlation values ​​are calculated to form a cross-correlation sequence c of length 2n-1. The position of the lag with the largest cross-correlation value is determined based on the cross-correlation sequence c, and the optimal phase difference with sub-pixel accuracy is obtained by fitting and optimizing using the least squares method. The optimal phase difference is then used as the final phase difference PD of the second-level sub-block.

[0155] Step 105: Output the final phase difference matrix based on the phase difference of each second-level sub-block.

[0156] Specifically, following the steps of vectorization, zero-mean, normalization, cross-correlation calculation, and least squares fitting optimization, the phase difference is calculated for a single left second-level sub-block and its corresponding right second-level sub-block to obtain the optimal phase difference of a single second-level sub-block. The optimal phase difference is then used as the final phase difference of a single second-level sub-block. This method is applied to all second-level sub-blocks involved in the phase difference calculation to calculate the final phase difference, and the final phase difference matrix is ​​then integrated and output.

[0157] More specifically, assuming that the regions of interest in the left and right images in this embodiment are divided into 25 first-level grids, and each grid is further divided into 9 second-level sub-blocks, if all second-level sub-blocks participate in the phase difference calculation, then according to the above zero-mean normalized cross-correlation algorithm, the final phase difference of 25×9=225 second-level sub-blocks can be obtained. Then, the 225 final phase differences are finally integrated to output a phase difference matrix.

[0158] This embodiment divides the baseline texture map into a first-level grid and a second-level sub-block. Then, it classifies the texture regions of each second-level sub-block based on the texture strength value, performs corresponding image preprocessing on each second-level sub-block based on the classification, and calculates the phase difference of each preprocessed second-level sub-block to obtain the final phase difference. This can cover complex scenes such as sparse or dense textures and ensure the accuracy of the phase difference calculation results.

[0159] Another exemplary embodiment of this application also provides a method for generating phase difference based on left and right images, such as... Figure 3 As shown, the method includes:

[0160] Step 201: Obtain the left image and extract the reference texture from the left image to obtain the left reference texture map; obtain the right image and extract the reference texture from the right image to obtain the right reference texture map.

[0161] Step 202: Obtain the region of interest (ROI) of the left reference texture map, perform meshing on the ROI of the left reference texture map to obtain the left first-level mesh, and perform secondary meshing on the left first-level mesh to obtain the left second-level sub-blocks; Obtain the ROI of the right reference texture map, perform meshing on the ROI of the right reference texture map to obtain the right first-level mesh, and perform secondary meshing on the right first-level mesh to obtain the right second-level sub-blocks; The relative position of the ROI of the left reference texture map on the left reference texture map is consistent with the relative position of the ROI of the right reference texture map on the right reference texture map.

[0162] Step 203: Based on the left second-level sub-block, calculate and obtain the texture strength value of each left second-level sub-block for the texture map of each left second-level sub-block; based on the right second-level sub-block, calculate and obtain the texture strength value of each right second-level sub-block for the texture map of each right second-level sub-block.

[0163] Step 204: Classify each left second-level sub-block based on the texture intensity value of each left second-level sub-block; classify each right second-level sub-block based on the texture intensity value of each right second-level sub-block; perform corresponding image preprocessing on each second-level sub-block of different classifications to obtain preprocessed second-level sub-blocks, and calculate the phase difference corresponding to the spatial position of each preprocessed left second-level sub-block and each right second-level sub-block to obtain the final phase difference of each second-level sub-block.

[0164] In this embodiment, steps 201 to 204 are similar to steps 101 to 104 described above, and will not be repeated here.

[0165] In some embodiments, step 204 further includes filtering the final phase difference of each second-level sub-block based on a preset phase difference threshold.

[0166] For example, the phase difference threshold can be customized as needed. For instance, if the preset phase difference threshold is set to 5, then if the phase difference of each calculated second-level sub-block exceeds the preset phase difference threshold of 5 due to various interference reasons, it will be filtered out to exclude second-level sub-blocks with significantly larger phase differences, thus preventing the output of second-level sub-blocks with large phase differences from affecting the accuracy of the phase difference matrix of the final calculation result.

[0167] In this embodiment, filtering refers to excluding or deleting second-level sub-blocks whose phase difference is greater than the phase difference threshold.

[0168] Step 205: Calculate the confidence level of the phase difference of each second-level sub-block, and filter the phase difference of each second-level sub-block based on the preset confidence level threshold.

[0169] Specifically, the maximum value of the quadratic function fitted using the least squares method is used as the confidence level of the phase difference for each second-level sub-block. That is, the formula for calculating conf is conf = y(xopt) = a × xopt. 2 +b×xopt+d, where xopt is the x-coordinate of the vertex of the quadratic function, and a, b, and d are the least squares fitting parameters, where a<0.

[0170] For example, substituting the aforementioned example fitting parameters a≈-0.167, b=0, d=1, xopt=0, conf=(−0.167)×0 2 +0×0+1=1. Therefore, the confidence level of the phase difference calculated in the above embodiment is extremely high, and the matching result is reliable.

[0171] In this embodiment, the reliability of the phase difference is quantified by a confidence score, i.e., a confidence score conf. The closer conf is to 1, the more accurate the matching between the left second-level sub-block and the corresponding right second-level sub-block. The more reliable the final phase difference value of the second-level sub-block calculated by the aforementioned method is, and the higher the accuracy of the final phase difference matrix generated in subsequent steps. The lower conf is, such as <0.5, the more likely the matching between the left and right second-level sub-blocks may be affected by noise. The phase difference value needs to be removed or corrected to prevent noisy sub-blocks from participating in the phase difference calculation and affecting the accuracy of the final generated phase difference matrix.

[0172] In this embodiment, filtering the phase difference of each second-level sub-block based on a preset confidence threshold includes:

[0173] Specifically, each pair of second-level sub-blocks is assigned a confidence threshold. Each pair of second-level sub-blocks refers to a single left second-level sub-block and a single right second-level sub-block corresponding to its spatial position. The confidence threshold can be dynamically set according to the magnitude of the texture strength value of the left or right second-level sub-block in each pair of second-level sub-blocks.

[0174] Generally, the larger the texture strength value of the second-level sub-block, the higher the preset confidence threshold; the smaller the texture strength value of the second-level sub-block, the lower the preset confidence threshold. The larger the texture strength value, the stronger the texture intensity, and the smaller the texture strength value, the weaker the texture intensity.

[0175] The phase difference confidence level of each pair of second-level sub-blocks is calculated using the aforementioned method. Based on a preset confidence threshold, second-level sub-blocks with a phase difference confidence level less than the preset threshold are excluded, while those with a phase difference confidence level greater than or equal to the preset threshold are retained. For example, if the calculated confidence level of a pair of second-level sub-blocks is 0.92, and the confidence threshold for that pair is set to 0.8, then the final phase difference of that pair is retained as valid phase difference data. The final phase difference result of this pair of second-level sub-blocks is less affected by noise, ambiguity, and other factors, indicating high reliability of the phase difference. Conversely, if the confidence level is low, such as 0.3, it indicates that the phase difference result of this pair of second-level sub-blocks is more affected by noise, ambiguity, and other factors, indicating low reliability of the phase difference.

[0176] This embodiment filters the confidence of each second-level sub-block by setting a confidence threshold to avoid second-level sub-blocks with low reliability from participating in subsequent weighted calculations, which would waste computing resources or directly affect the final phase difference calculation accuracy.

[0177] Step 206: Based on the effective phase difference of each second-level sub-block, output the final phase difference matrix.

[0178] Specifically, each valid second-level sub-block refers to the second-level sub-blocks that have been filtered and retained. The phase differences of the valid second-level sub-blocks obtained based on the aforementioned method are integrated to output a final phase difference matrix. For example, in this embodiment, the regions of interest of the left and right images are each divided into 25 first-level grids, and each grid is further divided into 9 second-level sub-blocks. Then, according to the zero-mean normalized cross-correlation algorithm described above, the final phase differences of 25 × 9 = 225 second-level sub-blocks can be obtained (assuming that all second-level sub-blocks participate in the phase difference calculation). Then, phase difference filtering and / or confidence filtering are performed on all final phase differences. Assuming that 200 pairs of valid second-level sub-blocks are retained after phase difference filtering and / or confidence filtering, the corresponding 200 valid final phase differences are integrated to output a final phase difference matrix.

[0179] Another exemplary embodiment of this application also provides a method for generating phase difference based on left and right images, such as... Figure 4 As shown, the method includes:

[0180] Step 301: Obtain the left image and extract the reference texture from the left image to obtain the left reference texture map; obtain the right image and extract the reference texture from the right image to obtain the right reference texture map.

[0181] Step 302: Obtain the region of interest (ROI) of the left reference texture map, perform meshing on the ROI of the left reference texture map to obtain the left first-level mesh, and perform secondary meshing on the left first-level mesh to obtain the left second-level sub-blocks; obtain the ROI of the right reference texture map, perform meshing on the ROI of the right reference texture map to obtain the right first-level mesh, and perform secondary meshing on the right first-level mesh to obtain the right second-level sub-blocks; wherein the relative position of the ROI of the left reference texture map on the left reference texture map is consistent with the relative position of the ROI of the right reference texture map on the right reference texture map.

[0182] Step 303: Based on the left second-level sub-block, calculate and obtain the texture strength value of each left second-level sub-block for the texture map of each left second-level sub-block; based on the right second-level sub-block, calculate and obtain the texture strength value of each right second-level sub-block for the texture map of each right second-level sub-block.

[0183] Step 304: Classify each left second-level sub-block based on the texture intensity value of each left second-level sub-block; classify each right second-level sub-block based on the texture intensity value of each right second-level sub-block; perform corresponding image preprocessing on each second-level sub-block of different classifications to obtain preprocessed second-level sub-blocks, and calculate the phase difference corresponding to the spatial position of each preprocessed left second-level sub-block and each right second-level sub-block to obtain the final phase difference of each second-level sub-block.

[0184] Step 305: Calculate the confidence level of the phase difference of each second-level sub-block, and filter the phase difference of each second-level sub-block based on the preset confidence level threshold.

[0185] In this embodiment, steps 301 to 305 are similar to steps 201 to 205 described above, and will not be repeated here.

[0186] Step 306: Based on the effective final phase difference of each second-level sub-block, perform numerical classification on the final phase difference of each second-level sub-block, and generate a binary spatial location mask matrix corresponding to each first-level grid based on the result of the numerical classification.

[0187] Specifically, based on the phase difference of each second-level sub-block, the coordinate position of the second-level sub-block with a negative phase difference value in the phase difference matrix of its corresponding first-level grid is marked as 0, and the coordinate position of the second-level sub-block with a positive or zero phase difference value in the phase difference matrix of its corresponding first-level grid is marked as 1. The above steps are performed simultaneously on the left and right first-level grids.

[0188] For example: the phase difference matrix of a certain first-level grid obtained by the aforementioned calculation method is:

[0189] Following the aforementioned binary classification method, the coordinate positions of the second-level sub-blocks located in the first row and first column, the first row and third column, the second row and first column, and the second row and third column in the phase difference matrix of the first-level grid are marked as 1, and other positions are marked as 0. The resulting binary spatial position mask matrix is:

[0190] .

[0191] This embodiment uses a binary spatial location mask matrix to shield the interference from irrelevant regions, reducing the impact of noise on subsequent weighted calculations.

[0192] Step 307: For any first-level mesh, multiply the binarized spatial location mask matrix corresponding to the first-level mesh with the expected texture strength value to obtain the weight matrix.

[0193] The expected texture strength value is taken as the texture strength value of each left second-level sub-block or each right second-level sub-block corresponding to the first-level grid, or the expected texture strength value is taken as the average of the texture strength values ​​of each left second-level sub-block and each right second-level sub-block corresponding to the first-level grid.

[0194] Preferred, the expected texture strength value is the average of the texture strength values ​​of each left second-level sub-block and each right second-level sub-block corresponding to the first-level grid.

[0195] In this embodiment, for any first-level grid, the binarized spatial location mask matrix corresponding to the first-level grid is multiplied by the average of the texture strength values ​​of each left and right second-level sub-blocks corresponding to the first-level grid to obtain the weight matrix. This avoids using only the texture strength values ​​of each left second-level sub-block as weights, which would strengthen the influence of the left image texture region on the phase difference result and reduce the phase difference weight of the right image. Ultimately, the accuracy of the weighted phase difference depends on the texture strength values ​​of the left image texture region, resulting in a waste of texture feature information in the right image texture region. Similarly, it would also result in a waste of texture feature information in the left image texture region.

[0196] For example: Suppose an input first-level mesh, the matrix corresponding to the set of average texture strength values ​​of each left second-level sub-block and each right second-level sub-block is:

[0197] ;

[0198] The weight matrix obtained by multiplying the average texture strength values ​​of the left and right second-level sub-blocks at the corresponding spatial locations using the binarized spatial location mask matrix calculated above is as follows:

[0199] .

[0200] This embodiment obtains the weight matrix by averaging the texture strength values ​​of the left and right second-level sub-blocks using a binarized spatial location mask matrix. This fully considers the weight distribution of texture features in the left and right images, takes into account the reliability of both images, and suppresses one-sided errors.

[0201] In some embodiments, step 307 further includes: normalizing the weight matrix to obtain a normalized weight matrix.

[0202] For example, the non-zero values ​​corresponding to the spatial positions of the weight matrix obtained above are normalized proportionally to obtain a normalized weight matrix, wherein the sum of the non-zero values ​​corresponding to each coefficient of the normalized weight matrix is ​​1.

[0203] For example, the weight matrix above, after normalization, becomes:

[0204] .

[0205] This embodiment can limit the weight range and avoid interference from extreme values ​​by normalizing the weight matrix.

[0206] Step 308: Based on the weight matrix, perform weighted processing on the phase difference matrix corresponding to each first-level grid to obtain the weighted phase difference matrix of each first-level grid, and obtain the final phase difference matrix based on the weighted phase difference matrix of each first-level grid.

[0207] Specifically, the weight coefficients at each position of the weight matrix are multiplied by the phase difference of each second-level sub-block in the first-level grid at the corresponding spatial position. The multiplied value is used as the final phase difference matrix of a single first-level grid. The final phase difference matrices of all first-level grids are output to obtain the final phase difference matrix.

[0208] For example, based on the weight matrix described above: The phase difference matrix between the weight coefficients at each position of the above weight matrix and the first-level grid at the corresponding spatial position is: The product is multiplied, and the value of the product is used as the weighted phase difference matrix of the first-level grid. The weighted phase difference matrix of the first-level grid is calculated as follows: .

[0209] In some embodiments, step 308 further includes: weighting the phase difference matrix corresponding to each first-level grid based on the normalized weight matrix to obtain the weighted phase difference matrix of each first-level grid, and obtaining the final phase difference matrix based on the weighted phase difference of each first-level grid.

[0210] Specifically, the weight coefficients at each position of the normalized weight matrix are multiplied by the phase difference of the corresponding first-level grid. The multiplied value is used as the final phase difference matrix for a single first-level grid. The final phase difference matrices of all first-level grids are then output to obtain the final phase difference matrix.

[0211] For example, based on the weight matrix after the above normalization process: The phase difference matrix between the weight coefficients at each position of the normalized weight matrix and the first-level grid at the corresponding spatial position is then calculated: The product is multiplied, and the value of the product is used as the weighted phase difference matrix of the first-level grid. The weighted phase difference matrix of the first-level grid is calculated as follows: .

[0212] This embodiment divides the baseline texture map extracted from the original phase detection image into grids and sub-blocks, and then calculates the phase difference of each sub-block based on the texture strength value. This can cover complex scenes such as sparse or dense textures, ensuring the accuracy of the calculation results. Furthermore, by weighting the phase difference matrix corresponding to each first-level grid with a weight matrix, the weight of strong textures can be increased and the weight of weak textures can be decreased, thereby strengthening the contribution of the sub-blocks with smaller phase difference errors in the image and suppressing noise.

[0213] Another exemplary embodiment of this application also provides a method for generating phase difference based on left and right images, such as... Figure 5 As shown, the method includes:

[0214] Step 401: Obtain the left image and extract the reference texture from the left image to obtain the left reference texture map; obtain the right image and extract the reference texture from the right image to obtain the right reference texture map.

[0215] Step 402: Obtain the region of interest (ROI) of the left reference texture map, perform meshing on the ROI of the left reference texture map to obtain the left first-level mesh, and perform secondary meshing on the left first-level mesh to obtain the left second-level sub-blocks; obtain the ROI of the right reference texture map, perform meshing on the ROI of the right reference texture map to obtain the right first-level mesh, and perform secondary meshing on the right first-level mesh to obtain the right second-level sub-blocks; wherein the relative position of the ROI of the left reference texture map on the left reference texture map is consistent with the relative position of the ROI of the right reference texture map on the right reference texture map.

[0216] Step 403: Based on the left second-level sub-block, calculate and obtain the texture strength value of each left second-level sub-block for the texture map of each left second-level sub-block; based on the right second-level sub-block, calculate and obtain the texture strength value of each right second-level sub-block for the texture map of each right second-level sub-block.

[0217] In this embodiment, steps 401 to 403 are similar to steps 101 to 103 described above, and will not be repeated here.

[0218] Step 404: Exponentially amplify the texture intensity values ​​of each left second-level sub-block; exponentially amplify the texture intensity values ​​of each right second-level sub-block.

[0219] Specifically, the formula for exponentially amplifying the texture intensity values ​​of each left second-level sub-block is as follows: Where α is an adjustable coefficient and W represents the texture strength value. In this embodiment, the texture strength value W can be obtained by extracting the vertical gradient of each left second-level sub-block using the Sobel operator and calculating the non-zero mean of the vertical gradient. The texture strength values ​​are exponentially amplified and used for phase difference calculation and phase difference weighting calculation. The left second-level sub-blocks with high texture (i.e., those with larger corresponding W values) are significantly amplified, while the left second-level sub-blocks with low texture (i.e., those with smaller corresponding W values) are suppressed to near zero.

[0220] For example, if α∈[0.3, 0.7], when α=0.3, the left second-level sub-block W with low texture might be 0.1, i.e. For the left second-level sub-block with high texture, W may be 1, i.e. Therefore, it can be seen that the W value of the left second-level sub-block with high texture is significantly amplified; the W value of the left second-level sub-block with low texture is amplified to a value close to zero, and is suppressed by low values.

[0221] Similarly, the same processing is applied to the second-level right sub-block, and the processing procedure and effect are the same, so it will not be described in detail here.

[0222] This embodiment further enhances the distinguishability of texture strength values ​​through nonlinear amplification, thereby increasing the weight of high-texture second-level sub-blocks and suppressing interference from low-texture second-level sub-blocks during subsequent phase difference calculations.

[0223] Step 405: Classify each left second-level sub-block based on the texture intensity values ​​of each left second-level sub-block after exponential magnification; classify each right second-level sub-block based on the texture intensity values ​​of each right second-level sub-block after exponential magnification; perform corresponding image preprocessing on each second-level sub-block of different classifications to obtain preprocessed second-level sub-blocks, and calculate the phase difference corresponding to the spatial position of each preprocessed left second-level sub-block and each right second-level sub-block to obtain the final phase difference of each second-level sub-block.

[0224] Step 406: Output the final phase difference matrix based on the final phase difference of each second-level sub-block.

[0225] In this embodiment, steps 405 to 406 are similar to steps 104 to 105 described above, and will not be repeated here.

[0226] Another exemplary embodiment of this application provides a method for generating phase difference based on left and right images, such as... Figure 6 As shown, the method includes:

[0227] Step 501: Obtain the left image and extract the reference texture from the left image to obtain the left reference texture map; obtain the right image and extract the reference texture from the right image to obtain the right reference texture map.

[0228] Step 502: Obtain the region of interest (ROI) of the left reference texture map, perform meshing on the ROI of the left reference texture map to obtain the left first-level mesh, and perform secondary meshing on the left first-level mesh to obtain the left second-level sub-blocks; Obtain the ROI of the right reference texture map, perform meshing on the ROI of the right reference texture map to obtain the right first-level mesh, and perform secondary meshing on the right first-level mesh to obtain the right second-level sub-blocks; The relative position of the ROI of the left reference texture map on the left reference texture map is consistent with the relative position of the ROI of the right reference texture map on the right reference texture map.

[0229] In this embodiment, steps 501 to 502 are similar to steps 401 to 402 described above, and will not be repeated here.

[0230] Step 503: Perform flat region filtering on the reference texture map of each left second-level sub-block; perform flat region filtering on the reference texture map of each right second-level sub-block.

[0231] For example, this embodiment can use binarization preprocessing technology to filter the flat region of the reference texture map of each left second-level sub-block, which can effectively filter out the left second-level sub-blocks with extremely low texture in each left second-level sub-block, avoid the left second-level sub-blocks with extremely low texture participating in the phase difference calculation, and solve the problem of inaccurate phase difference that may be caused by the participation of the left second-level sub-blocks with extremely low texture in the phase difference calculation.

[0232] For example, common flat region filtering methods used in this embodiment include local variance / standard deviation filtering (by calculating the variance or standard deviation of each left second-level sub-block pixel, low variance regions are considered flat regions), Sobel operator (calculating the gradient magnitude of each left second-level sub-block, gradient magnitude close to zero is considered a flat region), Prewitt operator (calculating the gradient magnitude of each left second-level sub-block, gradient magnitude close to zero is considered a flat region), adaptive thresholding method (automatically calculating the optimal threshold to separate flat regions and texture regions, pixel value changes less than the binarization threshold are judged as flat regions), etc.

[0233] Similarly, the same processing is applied to the second-level right sub-block, and the processing procedure and effect are the same, so it will not be described in detail here.

[0234] Step 504: For the texture maps of each left second-level sub-block after flat region filtering, calculate and obtain the texture strength value of each left second-level sub-block; for the texture maps of each right second-level sub-block after flat region filtering, calculate and obtain the texture strength value of each right second-level sub-block.

[0235] Step 505: Exponentially amplify the texture intensity values ​​of each left second-level sub-block; exponentially amplify the texture intensity values ​​of each right second-level sub-block.

[0236] Step 506: Classify each left second-level sub-block based on the texture intensity values ​​of each left second-level sub-block after exponential magnification; classify each right second-level sub-block based on the texture intensity values ​​of each right second-level sub-block after exponential magnification; perform corresponding image preprocessing on each second-level sub-block of different classifications to obtain preprocessed second-level sub-blocks, and calculate the phase difference corresponding to the spatial position of each preprocessed left second-level sub-block and each right second-level sub-block to obtain the final phase difference of each second-level sub-block.

[0237] Step 507: Output the final phase difference matrix based on the final phase difference of each second-level sub-block.

[0238] In this embodiment, steps 504 to 507 are similar to steps 403 to 406 described above, and will not be repeated here.

[0239] The above embodiments are merely a few exemplary examples and do not represent all feasible solutions of this application. The steps in the above embodiments are not fixed, and those skilled in the art can arbitrarily change the steps according to their needs to form different embodiments and achieve better technical effects of this application.

[0240] The scope of protection of the phase difference generation method based on left and right images in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting or replacing steps in the prior art based on the principle of this application is included within the scope of protection of this application.

[0241] This application also provides a phase difference generation system based on left and right images. This system can implement the phase difference generation method based on left and right images of this application. However, the implementation device of the phase difference generation method based on left and right images of this application includes, but is not limited to, the structure of the phase difference generation system based on left and right images listed in this embodiment. All structural modifications and substitutions of the prior art made in accordance with the principles of this application are included within the protection scope of this application.

[0242] Another exemplary embodiment of this application also provides a second aspect of this application, which provides a phase difference generation system based on left and right images, such as... Figure 7 As shown, the system includes:

[0243] Texture extraction module 1 is used to acquire the left image and perform reference texture extraction on the left image to obtain a left reference texture map; acquire the right image and perform reference texture extraction on the right image to obtain a right reference texture map; mesh segmentation module 2 is used to acquire the region of interest (ROI) of the left reference texture map, perform mesh segmentation on the ROI of the left reference texture map to obtain a left first-level mesh, and perform secondary segmentation on the left first-level mesh to obtain a left second-level sub-block; acquire the ROI of the right reference texture map, perform mesh segmentation on the ROI of the right reference texture map to obtain a right first-level mesh, and perform secondary segmentation on the right first-level mesh to obtain a right second-level sub-block, wherein the relative position of the ROI of the left reference texture map on the left reference texture map is consistent with the relative position of the ROI of the right reference texture map on the right reference texture map; texture intensity calculation module 3 is used to calculate the intensity of the texture based on the left second-level sub-block. The system comprises: a texture differentiation module 4, a texture classification module 5, and a texture phase difference calculation module 6. The module preprocesses the images of each left second-level sub-block to obtain the preprocessed sub-blocks, and calculates the phase difference between the preprocessed left and right second-level sub-blocks based on their spatial positions. The system also includes a texture differentiation module 4 for classifying each left second-level sub-block based on its texture strength value, and a texture phase difference calculation module 5 for performing corresponding image preprocessing on each classified second-level sub-block to obtain the preprocessed sub-blocks. Finally, a phase difference matrix generation module 6 outputs the final phase difference matrix based on the final phase difference of each second-level sub-block.

[0244] This embodiment divides the baseline texture map into a first-level grid and a second-level sub-block. Then, it classifies the texture regions of each second-level sub-block based on the texture strength value, performs corresponding image preprocessing on each second-level sub-block based on the classification, and calculates the phase difference of each preprocessed second-level sub-block to obtain the final phase difference. This can cover complex scenes such as sparse or dense textures and ensure the accuracy of the phase difference calculation results.

[0245] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0246] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0247] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0248] In one exemplary embodiment, this embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a program executable on the processor, and when the program is executed by the processor, the electronic device performs the method described in any of the above embodiments.

[0249] Electronic devices can be, but are not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, smart cars, etc., while portable wearable devices can be smartwatches and smart bracelets, etc. It is understood that this method can also be applied to systems that include electronic devices and servers, and is implemented through the interaction between electronic devices and servers.

[0250] In one possible embodiment, such as Figure 8 As shown, the electronic device 20 further includes: an output interface 23 for outputting results; a communication interface 24 for transmitting communication signals; and an antenna 25 for transmitting or receiving signals.

[0251] It should be noted that the processor 21 in this embodiment can be an image processing chip or an integrated circuit chip, capable of processing image signals. In implementation, each step of the above method embodiment can be completed by the integrated logic circuitry in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices. It can implement or execute the methods, steps, and logic block diagrams disclosed in this embodiment. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this embodiment can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0252] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0253] In one exemplary embodiment, this embodiment also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0254] In one exemplary embodiment, this embodiment also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0255] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0256] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0257] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0258] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A phase difference generation method based on left and right images, characterized by, The method comprises: acquiring a left image, and performing fiducial texture extraction on the left image to obtain a left fiducial texture map; acquiring a right image, and performing fiducial texture extraction on the right image to obtain a right fiducial texture map; acquiring a region of interest of the left fiducial texture map, performing mesh division on the region of interest of the left fiducial texture map to obtain a left first-level mesh, and performing secondary division on the left first-level mesh to obtain a left second-level sub-block; acquiring a region of interest of the right fiducial texture map, performing mesh division on the region of interest of the right fiducial texture map to obtain a right first-level mesh, and performing secondary division on the right first-level mesh to obtain a right second-level sub-block, wherein the relative position of the region of interest of the left fiducial texture map in the left fiducial texture map is consistent with the relative position of the region of interest of the right fiducial texture map in the right fiducial texture map; based on the left second-level sub-block, calculating the texture strength value of each left second-level sub-block; based on the right second-level sub-block, calculating the texture strength value of each right second-level sub-block; based on the texture strength value of each left second-level sub-block, dividing each left second-level sub-block into at least two categories; based on the texture strength value of each right second-level sub-block, dividing each right second-level sub-block into at least two categories, wherein the dividing of each left second-level sub-block into at least two categories comprises dividing each left second-level sub-block into a low-texture sub-block and a high-texture sub-block based on the texture strength value of each left second-level sub-block, and the dividing of each right second-level sub-block into at least two categories comprises dividing each right second-level sub-block into a low-texture sub-block and a high-texture sub-block based on the texture strength value of each right second-level sub-block; performing corresponding image preprocessing on each second-level sub-block of different categories to obtain the preprocessed second-level sub-blocks, and performing phase difference calculation on the preprocessed left second-level sub-blocks and right second-level sub-blocks corresponding in space to obtain the final phase difference of each second-level sub-block, wherein the phase difference of each low-texture sub-block is calculated using a texture map obtained by performing strong edge extraction on an original phase detection image, and the phase difference of each high-texture sub-block is calculated using the original phase detection image, and the original phase detection image is the left image or the right image; based on the final phase difference of each second-level sub-block, outputting a final phase difference matrix.

2. The method of claim 1, wherein, based on the texture strength value of each left second-level sub-block, dividing each left second-level sub-block into a low-texture sub-block, a medium-texture sub-block, and a high-texture sub-block, and based on the texture strength value of each right second-level sub-block, dividing each right second-level sub-block into a low-texture sub-block, a medium-texture sub-block, and a high-texture sub-block; the performing of corresponding image preprocessing on each second-level sub-block of different categories, and the performing of phase difference calculation on the preprocessed left second-level sub-blocks and right second-level sub-blocks corresponding in space, comprise: The phase difference of each low-texture sub-block is calculated by using the original phase detection image after strong edge extraction, the phase difference of each medium-texture sub-block is calculated by using the reference texture image, and the phase difference of each high-texture sub-block is calculated by using the original phase detection image.

3. The method of claim 2, wherein, The left second-level sub-blocks and the right second-level sub-blocks are divided into three categories, i.e., low-texture sub-blocks, medium-texture sub-blocks and high-texture sub-blocks, based on the texture strength values of the left second-level sub-blocks and the right second-level sub-blocks, and the first threshold and the second threshold are set, wherein the first threshold is smaller than the second threshold; The left second-level sub-blocks and the right second-level sub-blocks with the texture strength values smaller than the first threshold are marked as low-texture sub-blocks, the left second-level sub-blocks and the right second-level sub-blocks with the texture strength values within the interval range between the first threshold and the second threshold are marked as medium-texture sub-blocks, and the left second-level sub-blocks and the right second-level sub-blocks with the texture strength values greater than the second threshold are marked as high-texture sub-blocks, wherein the interval range includes the first threshold and the second threshold as two critical values.

4. The method of claim 3, wherein, Further comprising: based on the texture strength value and a preset third threshold, the left second-level sub-blocks with the texture strength values smaller than the third threshold are marked as abnormal sub-blocks; the right second-level sub-blocks with the texture strength values smaller than the third threshold are marked as abnormal sub-blocks, wherein the third threshold is smaller than the first threshold.

5. The method of claim 1, wherein, After the image preprocessing of the second-level sub-blocks of different categories is performed respectively to obtain the preprocessed second-level sub-blocks, and the phase difference of the preprocessed left second-level sub-blocks and the preprocessed right second-level sub-blocks is calculated based on the spatial position correspondence to obtain the final phase difference of each second-level sub-block, the method further comprises: The phase difference confidence of each second-level sub-block is calculated, and the final phase difference of each second-level sub-block is filtered based on a preset confidence threshold.

6. The method of claim 1, wherein, After the image preprocessing of the second-level sub-blocks of different categories is performed respectively to obtain the preprocessed second-level sub-blocks, and the phase difference of the preprocessed left second-level sub-blocks and the preprocessed right second-level sub-blocks is calculated based on the spatial position correspondence to obtain the final phase difference of each second-level sub-block, the method further comprises: The final phase difference of each second-level sub-block is filtered based on a preset phase difference threshold.

7. The method according to any one of claims 1 to 6, characterized in that, After the image preprocessing of the second-level sub-blocks of different categories is performed respectively to obtain the preprocessed second-level sub-blocks, and the phase difference of the preprocessed left second-level sub-blocks and the preprocessed right second-level sub-blocks is calculated based on the spatial position correspondence to obtain the final phase difference of each second-level sub-block, the method further comprises: The phase difference of each second-level sub-block is classified by a numerical value, and a binary spatial position mask matrix corresponding to each first-level grid is generated based on the classification result of the numerical value; For any first-level grid, multiply the corresponding binary space position mask matrix of the first-level grid with the expected texture intensity value to obtain a weight matrix; Based on the weight matrix, weight the phase difference matrix corresponding to each first-level grid to obtain a weighted phase difference matrix of each first-level grid, and obtain a final phase difference matrix based on the weighted phase difference matrices of the first-level grids.

8. The method of claim 1, wherein, After the texture intensity values of each left second-level sub-block and the texture intensity values of each right second-level sub-block are obtained, the method further includes: Exponentially amplifying the texture intensity values of each left second-level sub-block; and exponentially amplifying the texture intensity values of each right second-level sub-block. Based on the exponentially amplified texture intensity values of each left second-level sub-block, classify each left second-level sub-block; and based on the exponentially amplified texture intensity values of each right second-level sub-block, classify each right second-level sub-block.

9. A phase difference generating system based on left and right images, characterized by, The system includes: a texture extraction module configured to obtain a left image, perform reference texture extraction on the left image to obtain a left reference texture image, obtain a right image, and perform reference texture extraction on the right image to obtain a right reference texture image; a grid partitioning module configured to obtain a region of interest of the left reference texture image, perform grid division on the region of interest of the left reference texture image to obtain a left first-level grid, perform secondary division on the left first-level grid to obtain a left second-level sub-block, obtain a region of interest of the right reference texture image, perform grid division on the region of interest of the right reference texture image to obtain a right first-level grid, and perform secondary division on the right first-level grid to obtain a right second-level sub-block, wherein the relative position of the region of interest of the left reference texture image in the left reference texture image is consistent with the relative position of the region of interest of the right reference texture image in the right reference texture image; a texture intensity calculation module configured to calculate texture intensity values of each left second-level sub-block based on the left second-level sub-blocks, and calculate texture intensity values of each right second-level sub-block based on the right second-level sub-blocks; a texture classification module configured to classify each left second-level sub-block into at least two categories based on the texture intensity values of each left second-level sub-block, and classify each right second-level sub-block into at least two categories based on the texture intensity values of each right second-level sub-block, wherein the classification of each left second-level sub-block into at least two categories includes classifying each left second-level sub-block into a low-texture sub-block and a high-texture sub-block based on the texture intensity values of each left second-level sub-block, and the classification of each right second-level sub-block into at least two categories includes classifying each right second-level sub-block into a low-texture sub-block and a high-texture sub-block based on the texture intensity values of each right second-level sub-block. a texture phase difference calculation module, configured to respectively perform corresponding image preprocessing on each of the second-level sub-blocks of different categories to obtain each of the preprocessed second-level sub-blocks, and perform phase difference calculation on each of the left second-level sub-blocks and each of the right second-level sub-blocks in a spatial position corresponding manner to obtain a final phase difference of each of the second-level sub-blocks, wherein the phase difference of each of the low-texture sub-blocks is calculated by using a texture map obtained by performing strong edge extraction on an original phase detection image, and the phase difference of each of the high-texture sub-blocks is calculated by using the original phase detection image, the original phase detection image being the left image or the right image; a phase difference matrix generation module, configured to output a final phase difference matrix based on the final phase difference of each of the second-level sub-blocks.

Images