Binocular vision stereo matching method and storage medium based on regional feature extraction

Abstract

A binocular vision stereo matching method and storage medium based on regional feature extraction, wherein the binocular vision stereo matching method includes: obtaining a first image and a second image under two viewpoints; setting a first pixel point in the first image, expanding other pixels distributed around the first pixel point with similar textures to form a first texture area, and obtaining a third image using the first texture area and the unexpanded area outside; setting a second pixel point in the second image, forming a second texture area by expanding similar textures and obtaining a fourth image; cost aggregation of any pixel point in the third image or the fourth image according to a preset cost function and a plurality of preset disparity values, so as to obtain the best disparity value of the pixel point from each disparity value. This method can overcome the disadvantage that the low texture content matching process between the two images is prone to errors, and greatly improve the performance of the binocular vision stereo matching algorithm.

Description

Technical Field

[0001] The present invention relates to the field of binocular stereo vision technology, and in particular to a binocular vision stereo matching method and a storage medium based on regional feature extraction. Background Art

[0002] As we all know, the light in the scene is collected by the human eye, a sophisticated imaging system, and sent to the brain containing hundreds of millions of neurons through the nerve center for parallel processing, obtaining real-time, high-definition, and accurate depth perception information. This greatly improves the human ability to adapt to the environment, and many complex actions can be completed: such as walking, sports, driving vehicles, and conducting scientific experiments.

[0003] Computer vision is a discipline that uses computers to simulate the human visual system, with the goal of restoring a 3D image based on two acquired flat images. Currently, the level of computer stereo vision is still far from that of human binocular vision, so research on it is still a very active field.

[0004] Binocular Stereo Vision is an important form of computer vision. It is based on the principle of parallax and uses imaging equipment to obtain two images of the object being measured from different positions. By calculating the position deviation between corresponding points in the images, the three-dimensional geometric information of the object is obtained. It can be seen that it processes the real world by simulating the human visual system. The research on stereo vision matching can greatly enhance the computer or robot's perception of the environment, so that the robot can better adapt to the environment and be more intelligent, so as to better serve people. After years of technological development, binocular stereo vision has been applied in the fields of robot vision, aerial mapping, reverse engineering, military application, medical imaging and industrial inspection.

[0005] Currently, binocular stereo vision fuses the images obtained by two imaging devices and observes the difference between them, so that the computer can obtain accurate depth information, establish the correspondence between features, and match the image points of the same spatial physical point in different images. This difference is usually called disparity. However, the most important but very difficult problem in binocular stereo vision is the stereo matching problem, that is, finding matching corresponding points from images with different viewpoints.

[0006] However, some images in binocular stereo vision often contain a large amount of low-texture content, such as the sky, distant mountains, grasslands and other areas. Each pixel in these areas is not much different from other pixels. Traditional methods can only rely on local color texture information to extract regional features. Whether it is the color plus gradient processing form, the local area cost aggregation processing form, or the more complex matching processing method that has a certain inhibitory effect on light changes, it cannot solve this problem, which brings some adverse effects to the practical application of binocular vision stereo matching. Summary of the invention

[0007] The main technical problem solved by the present invention is how to improve the accuracy of stereo matching of pixels in the process of using the existing binocular vision technology. To solve the above technical problem, the present application discloses a binocular vision stereo matching method based on regional feature extraction and a storage medium.

[0008] According to the first aspect, an embodiment provides a binocular vision stereo matching method based on regional feature extraction, including: acquiring a first image and a second image under two viewpoints; setting a first pixel point in the first image, and starting from the first pixel point, expanding other pixel points distributed around with similar texture to form a first texture area, and using the first texture area and the unexpanded area outside to obtain a third image; setting a second pixel point in the second image, and starting from the second pixel point, expanding other pixel points distributed around with similar texture to form a second texture area, and using the second texture area and the unexpanded area outside to obtain a fourth image; performing cost aggregation on any pixel point in the third image or the fourth image according to a preset cost function and a plurality of preset disparity values ​​to obtain the best disparity value of the pixel point from each of the disparity values.

[0009] A first pixel point is set in the first image, and other pixels distributed around the first pixel point are extended with similar textures to form a first texture area starting from the first pixel point, and a third image is obtained by using the first texture area and the unextended area outside. The method includes: determining low-texture content in the first image and setting a first pixel point therein, and taking the first pixel point as the center point to establish a first limited area; within the scope of the first limited area, starting from the first pixel point, the continuous pixels distributed longitudinally and transversely are judged with similar textures in sequence to obtain each pixel point with similar texture to the first pixel point; forming a first texture area by using each pixel point with similar texture to the first pixel point, and marking each pixel point in the first texture area as a first value; taking the area outside the first texture area and within the first limited area as the unextended area, and marking each pixel point in the unextended area as a second value; and generating a third image according to each pixel point marked as the first value and each pixel point marked as the second value in the first image.

[0010] The method comprises: starting from the first pixel point, sequentially expanding the continuous pixel points distributed along the longitudinal direction and the horizontal direction, and determining that the textures are similar to each other with respect to the first pixel point within the scope of the first limited area; and determining that the grayscale difference between a pixel point and the last expanded pixel point is less than a preset threshold value, confirming that the pixel point is a pixel point with a similar texture to the first pixel point, until determining that the next pixel point in the longitudinal direction is a pixel point with a non-similar texture; and starting from each pixel point with a similar texture obtained in the longitudinal direction, sequentially expanding the continuous pixel points distributed along the horizontal direction, and determining that the grayscale difference between a pixel point and the last expanded pixel point is less than the threshold value, confirming that the pixel point is a pixel point with a similar texture to the first pixel point, until determining that the next pixel point in the horizontal direction is a pixel point with a non-similar texture.

[0011] The method of setting a second pixel point in the second image, extending other pixel points distributed around the second pixel point with similar textures to form a second texture area, and obtaining a fourth image using the second texture area and the unextended area outside the second texture area includes: setting a second pixel point in the second image according to the position of the first pixel point in the first image, and using the second pixel point as the center point to establish a second limited area; within the scope of the second limited area, starting from the second pixel point, successively judging the similar textures of continuous pixel points distributed longitudinally and horizontally to obtain each pixel point with similar textures to the second pixel point; forming a second texture area using each pixel point with similar textures to the second pixel point, and marking each pixel point in the second texture area as a first value; using the area outside the second texture area and within the second limited area as the unextended area, and marking each pixel point in the unextended area as a second value; and generating a fourth image based on each pixel point marked as the first value and each pixel point marked as the second value in the second image.

[0012] In the third image and the fourth image, each pixel point marked as the first value has the same grayscale value as its original pixel point, and each pixel point marked as the second value has a completely black grayscale value.

[0013] After acquiring the first image and the second image under two viewpoints, and before setting the first pixel point in the first image, it also includes: taking the edge pixel points of the first image as the axis of symmetry, and performing mirror extension processing on the first image along the axis of symmetry, so as to extend it to a preset height and a preset width around the first image; taking the edge pixel points of the second image as the axis of symmetry, and performing mirror extension processing on the second image along the axis of symmetry, so as to extend it to a preset height and a preset width around the second image; the preset height is greater than or equal to half the height value of the first limited area, and the preset width is greater than or equal to half the width value of the first limited area; the second limited area is the same size as the first limited area.

[0014] The method of cost aggregating any pixel point in the third image or the fourth image according to a preset cost function and a preset plurality of disparity values ​​to obtain the best disparity value of the pixel point from each of the disparity values ​​includes: obtaining any pixel point from the third image or the fourth image and setting it as the third pixel point; cost aggregating the third pixel point according to a preset cost function and a preset plurality of disparity values ​​to obtain a cost aggregation function corresponding to each disparity value; and obtaining the best disparity value of the third pixel point from each disparity value using the cost aggregation function corresponding to each disparity value.

[0015] The cost function is a cost function corresponding to color, gradient, rank or NCC; the disparity value is represented by d and has a value range of {0, 1, ..., d max}, where d max Indicates the maximum allowed value of the disparity value.

[0016] The method of using the cost aggregation function corresponding to each disparity value to obtain the best disparity value of the third pixel from each disparity value includes: calculating the cost aggregation function corresponding to each disparity value and expressing it as C(y, x, d), obtaining the disparity value input when satisfying the minimum function value, and taking the disparity value as the best disparity value; wherein y and x are the ordinate value and abscissa value of the third pixel point, respectively.

[0017] According to the second aspect, an embodiment provides a computer-readable storage medium, including a program, wherein the program can be executed by a processor to implement the binocular vision stereo matching method described in the first aspect.

[0018] The beneficial effects of this application are:

[0019] According to the binocular vision stereo matching method and storage medium based on regional feature extraction of the above-mentioned embodiment, the binocular vision stereo matching method includes: acquiring a first image and a second image under two viewpoints; setting a first pixel point in the first image, and starting from the first pixel point, expanding other pixel points distributed around with similar texture to form a first texture area, and obtaining a third image using the first texture area and the unexpanded area outside; setting a second pixel point in the second image, and starting from the second pixel point, expanding other pixel points distributed around with similar texture to form a second texture area, and obtaining a fourth image using the second texture area and the unexpanded area outside; cost aggregation is performed on any pixel point in the third image or the fourth image according to a preset cost function and a plurality of preset disparity values ​​to obtain the best disparity value of the pixel point from each disparity value. On the one hand, since the first image and the second image under two viewpoints are respectively extended with similar textures, the third image and the fourth image containing specific texture areas are respectively obtained, which can improve the correlation coefficient between the third image and the fourth image, overcome the disadvantage that the low-texture content matching process between the two images is prone to errors, and greatly improve the performance of the binocular vision stereo matching algorithm based on the traditional method; on the other hand, since the similar textures of other pixel points distributed around the first image and the second image are extended starting from the set pixel point, the specific texture area containing structural information can be obtained through this extension method, so the third image and the fourth image required for binocular vision stereo matching can be easily formed based on the specific texture area, so that the structural information, color information and texture information contained in the image are used to participate in the stereo matching process, and a more accurate cost aggregation calculation result is obtained, thereby improving the accuracy of stereo matching; on the third hand, since the first image and the second image are Restricted areas are established respectively, and each pixel point with similar texture to the set pixel point is obtained by judging similar texture within the range of the restricted area, so that the range of pixel expansion can be standardized in the first image and the second image respectively, and a specific texture area with higher similarity can be obtained, thereby improving the correlation coefficient between the images and the matching cost of the subsequent stereo matching process; in the fourth aspect, before the similar texture of the first image and the second image is extended, the image is also mirrored along the axis of symmetry, so that the expandable range within the first image and the second image is increased, so that even the similar texture extension of the surrounding pixels using the edge pixels of the original image can be easily realized; in the fifth aspect, since the third pixel point obtained in the third image or the fourth image is cost aggregated according to the preset cost function and the preset multiple disparity values, the cost aggregation function with higher robustness can be obtained by cost aggregation, which is conducive to obtaining the best disparity value with higher accuracy from each disparity value. In the sixth aspect, the technical solution of the present application can effectively solve the problem of mismatching during stereo matching, which is conducive to accurately finding matching corresponding points in different viewpoint images and improving the accuracy of stereo matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flow chart of the binocular vision stereo matching method based on regional feature extraction in this application;

[0021] Figure 2 A flow chart for expanding a first texture region in a first image and obtaining a third image;

[0022] Figure 3 A flow chart for expanding a second texture region in a second image and obtaining a fourth image;

[0023] Figure 4 Flowchart for obtaining the best disparity value through cost aggregation;

[0024] Figure 5 A flowchart of performing image mirroring extension processing in a first image and a second image respectively;

[0025] Figure 6 Schematic diagram of the structure of the stereo matching device in this application;

[0026] Figure 7 is a schematic image of a first image;

[0027] Figure 8 is a schematic image of the first image after the mirror image extension process;

[0028] Fig. 9 is a schematic diagram of a first limited area in a first image;

[0029] Fig.10 Schematic images of the first texture area and the unexpanded area in the third image. DETAILED DESCRIPTION

[0030] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0031] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0032] The serial numbers of the components in this document, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0033] In stereo matching of binocular vision, a key issue is to find matching points in the left and right images to obtain the horizontal position difference of the corresponding pixels in the two images, also known as disparity, so that the depth of the pixel can be further calculated. Pixels that are not at the same depth may have the same color, texture, gradient, etc., so this often leads to mismatches during stereo matching, which further leads to large errors in disparity calculation, greatly affecting the application of binocular vision in depth measurement. In order to overcome this, in the existing stereo matching methods of binocular images, the pixel points in the surrounding area of ​​the pixel point are generally used to estimate the pixel point, such as the problem of finding matching blocks in motion search of video coding and the problem of directly calculating the SAD (sum of absolute difference) matching. However, in these methods, the weighting of the pixel matching cost can only be calculated using the above-mentioned features such as color, texture and gradient that are not directly related to disparity, so these methods are still relatively non-robust.

[0034] In order to improve the robustness of the matching cost and the accuracy of pixel matching, the present application expands the similar textures of the first image and the second image under two viewpoints respectively, and obtains the third image and the fourth image containing specific texture areas respectively. Not only does it improve the correlation coefficient between the third image and the fourth image with the help of the structure of the texture area, but it also uses the structural information, color information and texture information contained in the image to participate in the stereo matching process. This can effectively solve the problem of low texture mismatching during stereo matching, which is conducive to accurately finding matching corresponding points in different viewpoint images and improving the accuracy of stereo matching.

[0035] The technical solution of the present application will be described in detail below with reference to embodiments.

[0036] Embodiment 1

[0037] Please refer to Figure 1The present application also discloses a binocular vision stereo matching method based on regional feature extraction, which mainly includes steps S100-S400, which are described below respectively.

[0038] Step S100: acquiring a first image and a second image at two viewpoints.

[0039] In one embodiment, the target object is imaged by a binocular camera. Since the binocular camera forms two image capturing viewpoints, a frame of image is obtained at each of the two image capturing viewpoints, thereby obtaining two left and right images, namely, a first image and a second image. The first image and the second image have substantially the same content, and there is a difference in parallax between the two images.

[0040] In addition, the first image and the second image here can also come from a certain data set, such as the KITTI data set (autonomous driving test set), which is currently the world's largest computer vision algorithm evaluation data set in the autonomous driving scenario. It is used to evaluate the performance of computer vision technologies such as stereo, optical flow, visual odometry, 3D object detection and 3D tracking in a vehicle environment. The KITTI data set contains real image data collected in scenes such as urban areas, rural areas and highways. Each image contains up to 15 vehicles and 30 pedestrians, as well as various degrees of occlusion and truncation. It has rich image content and is very suitable for training stereo matching networks for binocular vision.

[0041] Step S200, setting a first pixel point in the first image, expanding other pixel points distributed around the first pixel point with similar textures to form a first texture area, and obtaining a third image using the first texture area and the unexpanded area outside.

[0042] It should be noted that the first pixel point set in the first image can be a pixel point within a specific texture area (such as low-texture content such as the sky, grass, mountains, rivers, streets, etc.), and the first pixel point can be selected in advance by the user on the first image at will, and it is preferred to select a pixel point with a representative color in the area of ​​low-texture content.

[0043] It can be understood that after selecting the first pixel point in a low-texture content, other pixel points in the area can be found by similar texture expansion, so that the other pixel points meet the requirement of having a similar texture with the first pixel point, thereby forming a first texture area using the first pixel point and the other pixel points found.

[0044] Step S300, setting a second pixel point in the second image, expanding other pixel points distributed around the second pixel point with similar textures to form a second texture area, and obtaining a fourth image using the second texture area and the unexpanded area outside.

[0045] Similarly, the second pixel point set in the second image may also be a pixel point in a specific texture area (such as low-texture content such as the sky, grass, mountains, rivers, streets, etc.). Since the image content between the first image and the second image is basically the same and there is only a difference in parallax, the setting position of the second pixel point in the second image should be at a setting position corresponding to the first pixel point in the first image, so as to facilitate the expansion of similar textures for each pixel point in the same type of low-texture content in the second image.

[0046] It can be understood that after selecting a second pixel point within the same low-texture content, other pixel points in the area can be found through similar texture expansion, so that the other pixel points meet the requirement of having a similar texture with the second pixel point, thereby forming a second texture area using the second pixel point and the other pixel points found.

[0047] Step S400 , cost aggregation is performed on any pixel point in the third image or the fourth image according to a preset cost function and a plurality of preset disparity values, so as to obtain an optimal disparity value of the pixel point from various disparity values.

[0048] It should be noted that after cost aggregation is performed on any pixel point in the third image or the fourth image according to the preset cost function and the preset multiple disparity values, the cost aggregation function corresponding to each disparity value can be obtained, such as the cost aggregation function {C(y,x,d)|d=0,1,...,d max}, (y, x) represents the coordinate position of the pixel point, and d represents the disparity value. Then, the existing matching cost weighted algorithm can be used to process these cost aggregation functions, such as the winner-take-all algorithm WTA (Winner-take-all), which only needs to select the point with the best superposition matching cost within a certain range (the minimum value of SAD and SSD, or the maximum value of NCC) as the corresponding matching point.

[0049] In this embodiment, the above step S200 mainly involves the process of forming the first texture area in the first image and obtaining the third image. Figure 2 It can be seen that this step specifically includes steps S210-S250, which are described as follows respectively.

[0050] Step S210 , determining low-texture content in a first image and setting a first pixel point therein, and using the first pixel point as a center point to establish a first limited area.

[0051] For example Figure 7 For a first image composed of streets, cars, grass, distant mountains, sky, and buildings, if the image content that the user wants to match is the sky, then a first pixel point p can be set in the low-texture content represented by the sky in the first image, and a first limited area can be established with the first pixel point p as the center point. The first limited area can be as follows: Figure 8 As shown in the mark P.

[0052] It should be noted that the first limited area can be set to any shape, such as a rectangle, a circle, a triangle, etc. In this embodiment, the first limited area is preferably set to a rectangle, so as to form a rectangular strip around the first pixel point, and the rectangular strip should cover enough low-texture content. In some cases, the first limited area may also include some image content other than low-texture.

[0053] For example, Figure 8 In the figure, the first limited area P is a rectangular strip with a height of 500 pixels and a width of 200 pixels. The rectangular strip contains not only low-texture content represented by the sky, but also high-texture content represented by buildings, numbers and traffic signs.

[0054] Step S220, starting from the first pixel point, sequentially judge the similar textures of the continuous pixel points distributed longitudinally and transversely in the first limited area to obtain the pixel points having the similar textures to the first pixel point.

[0055] In a specific embodiment, starting from the first pixel point in the first limited area, the continuous pixel points distributed along the longitudinal direction are expanded in sequence, and when it is determined that the grayscale difference between a pixel point and the last expanded pixel point is less than a preset threshold, the pixel point is confirmed to be a pixel point with a similar texture to the first pixel point, until the next pixel point in the longitudinal direction is confirmed to be a pixel point with a non-similar texture. Then, starting from each pixel point with a similar texture obtained in the longitudinal direction, the continuous pixel points distributed along the transverse direction are expanded in sequence, and when it is determined that the grayscale value between a pixel point and the last expanded pixel point is less than a threshold, the pixel point is confirmed to be a pixel point with a similar texture to the first pixel point, until the next pixel point in the transverse direction is confirmed to be a pixel point with a non-similar texture.

[0056] For example Fig. 9 , the figure is for Figure 8For the extraction of the first limited area P, for the convenience of explanation, an xy coordinate system is established at the upper left corner of the first limited area P. In this coordinate system, any pixel point can be represented by (y, x), and the coordinates of the origin O are (0, 0); in addition, the first pixel point p in the first limited area P is located at the center of the area. If the height of the first limited area P is set to 2*L+1 and the width is set to 2*R+1, then the coordinates of the first pixel point p are (y0, x0) = (L+1, R+1).

[0057] Here, the first pixel point p can be used as the starting point of expansion, and the continuous pixel points in the longitudinal distribution and the transverse distribution can be expanded in sequence.

[0058] The first expansion step: distribute the low texture content vertically and expand it upwards. The following formula (1) can be used to determine whether the grayscale difference between a pixel and the last expanded pixel is less than a preset threshold.

[0059] max(|r(j,x0)-r(j+1,x0)|,|g(j,x0)-g(j+1,x0)|,|b(j,x0)-b(j+1,x0)|)<δ(1)

[0060] Among them, max() is the maximum value function, r() is the gray value of the R channel in the RGB color channel, g() is the gray value of the G channel in the RGB color channel, b() is the gray value of the B channel in the RGB color channel, and || is the absolute value operation; δ is the preset threshold, j is the vertical coordinate value and satisfies j=L, L-1, L-2, …, 1. At this time, the coordinate (j, x0) represents a pixel point distributed vertically upward, and the coordinate (j+1, x0) represents the previous expandable pixel point distributed vertically. For a certain pixel point (j, x0), if formula (1) is satisfied, it can be confirmed that the pixel point (j, x0) is a pixel point with similar texture to the first pixel point (y0, x0); if formula (1) is not satisfied, it can be confirmed that the pixel point (j, x0) is a pixel point with non-similar texture to the first pixel point (y0, x0); when it is confirmed that a certain pixel point (j, x0) is a pixel point with non-similar texture, the expansion of the continuous pixel points distributed vertically upward is terminated.

[0061] The second expansion step: distribute the low texture content vertically and expand it downwards. The following formula (2) can be used to determine whether the grayscale difference between a pixel and the last expanded pixel is less than a preset threshold.

[0062] max(|r(j,x0)-r(j-1,x0)|,|g(j,x0)-g(j-1,x0)|,|b(j,x0)-b(j-1,x0)|)<δ(2)

[0063] Wherein, δ is a preset threshold, j is a vertical coordinate value and satisfies j=L+2, L+3, …, 2L+1. At this time, the coordinate (j, x0) represents a pixel point distributed vertically downward, and the coordinate (j-1, x0) represents the previous expandable pixel point distributed vertically. For a certain pixel point (j, x0), if formula (2) is satisfied, it can be confirmed that the pixel point (j, x0) is a pixel point with a similar texture to the first pixel point (y0, x0); if formula (2) is not satisfied, it can be confirmed that the pixel point (j, x0) is a pixel point with a non-similar texture to the first pixel point (y0, x0); when it is confirmed that a certain pixel point (j, x0) is a pixel point with a non-similar texture, the expansion of the continuous pixel points distributed vertically downward is terminated.

[0064] The third expandable step: distribute the low texture content horizontally and expand it to the left. The following formula (3) can be used to determine whether the grayscale difference between a pixel point and the last expanded pixel point is less than a preset threshold.

[0065] max(|r(jˊ,i)-r(jˊ,i+1)|,|g(jˊ,i)-g(jˊ,i+1)|,|b(jˊ,i)-b(jˊ,i+1)|)<δ(3)

[0066] Wherein, δ is a preset threshold, i is a horizontal coordinate value and satisfies i=R, R-1, …, 1, jˊ is each pixel point with similar texture obtained in the vertical direction, for example, jˊ can be a pixel point (j, x0) with similar texture to the first pixel point (y0, x0) confirmed in the above process. At this time, the coordinate (jˊ, i) represents a pixel point distributed horizontally to the left, and the coordinate (jˊ, i+1) represents the previous expandable pixel point distributed horizontally. For a certain pixel point (jˊ,i), if formula (3) is satisfied, it can be confirmed that the pixel point (jˊ,i) is a pixel point with similar texture to the first pixel point (y0,x0); if formula (3) is not satisfied, it can be confirmed that the pixel point (jˊ,i) is a pixel point with non-similar texture to the first pixel point (y0,x0); when it is confirmed that a certain pixel point (jˊ,i) is a pixel point with non-similar texture, the expansion of the continuous pixels distributed to the left and along the horizontal direction is terminated.

[0067] Fourth expansion step: the low texture content is expanded horizontally and to the right. The following formula (4) can be used to determine whether the grayscale difference between a pixel point and the last expanded pixel point is less than a preset threshold.

[0068] max(|r(jˊ,i)-r(jˊ,i-1)|,|g(jˊ,i)-g(jˊ,i-1)|,|b(jˊ,i)-b(jˊ,i-1)|)<δ(4)

[0069] Wherein, δ is a preset threshold, i is a horizontal coordinate value and satisfies i=R+2, R+3, …, 2R+1, jˊ is each pixel point with similar texture obtained in the vertical direction, for example, jˊ can be a pixel point (j, x0) with similar texture to the first pixel point (y0, x0) confirmed in the above process. At this time, the coordinate (jˊ, i) represents a pixel point distributed horizontally to the right, and the coordinate (jˊ, i-1) represents the previous expandable pixel point distributed horizontally. For a certain pixel point (jˊ,i), if formula (4) is satisfied, it can be confirmed that the pixel point (jˊ,i) is a pixel point with similar texture to the first pixel point (y0,x0); if formula (4) is not satisfied, it can be confirmed that the pixel point (jˊ,i) is a pixel point with non-similar texture to the first pixel point (y0,x0); when it is confirmed that a certain pixel point (jˊ,i) is a pixel point with non-similar texture, the expansion of the continuous pixel points distributed to the right and along the horizontal direction is terminated.

[0070] Furthermore, a fifth step may be included. In the low-texture expansion process along the horizontal distribution, if a certain pixel point (jˊ, i) of the horizontal distribution is a pixel point with a similar texture to the first pixel point (y0, x0), then the upward or downward expansion along the vertical distribution may be continued from the pixel point (jˊ, i). The upward or downward expansion process along the vertical distribution may refer to the first and second steps mentioned above and will not be described here.

[0071] Step S230: forming a first texture region using each pixel point having a texture similar to the first pixel point, and marking each pixel point in the first texture region as a first value (eg, 1).

[0072] In step S220, each pixel point having a texture similar to the first pixel point in the first limited area can be found, and the first texture area is formed by using these pixel points. Fig.10 Is Fig. 9 The result obtained by pixel expansion is Fig.10 The area D1 in contains pixels with similar textures, so the area D1 is the first texture area.

[0073] For example, when the above formula (1), (2), (3) or (4) satisfies the limiting condition of the threshold δ, the pixel point (j, x0) or pixel point (jˊ, i) involved in the calculation of the formula is a pixel point with similar texture. At this time, these pixels can be marked so that

[0074] mask(j,x0)=1; mask(jˊ,i)=1;

[0075] Among them, mask is a mask function commonly used in image processing.

[0076] It should be noted that, since the pixel point marked with the first value and the first pixel point have similar textures, the first pixel (y0, x0) can obtain the same marking result, that is, mask (y0, x0) = 1.

[0077] Step S240: The area outside the first texture area and within the first limited area is regarded as a non-expanded area, and each pixel point in the non-expanded area is marked as a second value (such as 0).

[0078] For example, when the above formula (1), (2), (3) or (4) does not meet the limiting condition of the threshold δ, the pixel point (j, x0) or pixel point (jˊ, i) involved in the calculation of the formula is a pixel point that does not have a similar texture. At this time, these pixels can be marked so that

[0079] mask(j,x0)=0; mask(jˊ,i)=0.

[0080] It should be noted that, for example Fig.10 The area D2 in is the unexpanded area outside the first texture area D1, and each pixel in the unexpanded area can be marked as a second value (such as 0).

[0081] Step S250: generating a third image according to each pixel point marked as the first value and each pixel point marked as the second value in the first image.

[0082] For example, Fig.10 In the figure, each pixel in the first texture area D1 is marked as a first value, and each pixel in the unextended area D2 is marked as a second value, so the third image can be generated using the first texture area D1 and the second texture area D3.

[0083] It should be noted that in the third image, each pixel point marked as the first value in the first texture area D1 has the same grayscale value as its original pixel point, that is, the color of these pixels remains unchanged; however, each pixel point marked as the second value has a completely black grayscale value, that is, the grayscale values ​​of these pixels on the color channel are all 0 and appear to be completely black. For the pixel point (j, x0) marked as the second value, r(j, x0) = g(j, x0) = b(j, x0) = 0 is satisfied, and for the pixel point (jˊ, i) marked as the second value, r(jˊ, i) = g(jˊ, i) = b(jˊ, i) = 0 is satisfied.

[0084] In this embodiment, the above step S300 mainly involves the process of forming the second texture area in the second image and obtaining the fourth image. Figure 3 It can be seen that this step specifically includes steps S310-S350, which are described as follows respectively.

[0085] Step S310: setting a second pixel point in the second image corresponding to the position of the first pixel point in the first image, and using the second pixel point as the center point to establish a second limited area.

[0086] It should be noted that, since the image contents of the first image and the second image are substantially the same and only differ in disparity, the setting position of the second pixel in the second image should correspond to the setting position of the first pixel in the first image, so as to facilitate the expansion of similar textures for each pixel in the same type of low-texture content in the second image. If any pixel in the first image is represented by coordinates (y, x), the corresponding pixel in the second image is represented by coordinates (y, xd), where d represents disparity.

[0087] The process of using the second pixel point as the center point to establish the second defined area can be specifically referred to the above step S210. The shape and size of the second defined area can be the same as the first defined area, for example, the height of the second defined area is set to 2*L+1, the width is set to 2*R+1, and the coordinates of the second pixel point are (yˊ0, xˊ0) = (L+1, R+1).

[0088] Step S320, starting from the second pixel point, sequentially judge the similar textures of the continuous pixel points distributed longitudinally and transversely in the second limited area to obtain the pixel points having the similar textures to the second pixel point.

[0089] Regarding the process of obtaining each pixel point having a texture similar to the second pixel point within the second limited range, specific reference may be made to the above-mentioned step S220, which will not be described in detail here.

[0090] Step S330: forming a second texture region using each pixel point having a texture similar to the second pixel point, and marking each pixel point in the second texture region as a first value (eg, 1).

[0091] Regarding the process of forming the second texture area and marking each pixel therein as the first value, specific reference may be made to the above-mentioned step S230, which will not be described in detail here.

[0092] Step S340: The area outside the second texture area and within the second limited area is regarded as a non-expanded area, and each pixel point in the non-expanded area is marked as a second value (such as 0).

[0093] Regarding the process of obtaining the non-expanded area and marking each pixel therein as the second value, specific reference may be made to the above-mentioned step S240, which will not be described in detail here.

[0094] Step S350: Generate a fourth image based on each pixel point marked as the first value and each pixel point marked as the second value in the second image.

[0095] The process of generating the fourth image may be specifically referred to the above-mentioned step S210, but it should be noted that, in the fourth image, each pixel point marked as the first value has the same grayscale value as its original pixel point, that is, the color of these pixels remains unchanged; each pixel point marked as the second value has a grayscale value of all black, that is, the grayscale value of these pixels on the color channel is 0 and appears to be all black.

[0096] In this embodiment, the above step S400 mainly involves the process of performing cost aggregation on any pixel point in the third image or the fourth image to obtain the optimal disparity value. Figure 4 It can be seen that this step specifically includes steps S410-S430, which are described as follows respectively.

[0097] Step S410: Obtain any pixel from the third image or the fourth image and set it as the third pixel.

[0098] It should be noted that both the third image and the fourth image have texture areas and unexpanded areas. Since users are mainly concerned about the stereo matching of low-texture content in the images, it is preferred to select pixel points from the first texture area of ​​the third image or the second texture area of ​​the fourth image, and set the selected pixel points as the third pixel points.

[0099] Step S420 , cost aggregation is performed on the third pixel point according to a preset cost function and a plurality of preset disparity values ​​to obtain a cost aggregation function corresponding to each disparity value.

[0100] In this embodiment, the cost function is a cost function corresponding to color, gradient, rank or NCC; the disparity value can be represented by d and the value range is {0,1,...,d max}, where d max Indicates the maximum allowed disparity value.

[0101] In one embodiment, for each cost function, the function value of each disparity value at the third pixel point under the cost function is calculated, and the function values ​​of each disparity value at the pixel point are aggregated to obtain a cost aggregation function corresponding to the cost function.

[0102] It should be noted that the cost function in the present application includes but is not limited to the cost function corresponding to color, gradient, rank, NCC or mutual-information; among them, the cost function for color can refer to the technical document "Reference on cost function based on color [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994, Vol. 16 (9), pp. 920-932 Cross Ref", the cost function for gradient can refer to the technical document "Yang Xin, an image matching algorithm based on gradient operator [J]. Journal of Electronics, 1999 (10): 30-33"; the cost function for rank can refer to the technical document "A constraint to improve the reliability of stereo matching using the rank transform: Acoustics, Speech, and Signal Processing, 1999. on 1999 IEEE International Conference, 1999 [C]", for the cost function of NCC, please refer to the technical document "Blog article, Image processing based on NCC template matching recognition, the query URL is https: / / blog.csdn.net / jia20003 / article / details / 48852549, which believes that NCC is an algorithm based on statistical calculation of the correlation between two groups of sample data". Since all the listed cost functions belong to the prior art, they are not described one by one here. In addition, those skilled in the art should understand that with the development of technology, some other types of cost functions may appear in the future. These cost functions that will appear in the future can still be applied to the technical solution disclosed in this embodiment, and do not constitute a limitation to the technical solution of this embodiment.

[0103] Step S430 , obtaining the best disparity value of the third pixel from each disparity value using the cost aggregation function corresponding to each disparity value.

[0104] In a specific embodiment, the cost aggregation function corresponding to each disparity value is calculated and expressed as C(y,x,d), and the disparity value input when the minimum function value is satisfied is obtained, and the disparity value is used as the optimal disparity value; wherein y and x are the ordinate value and the abscissa value of the third pixel point, respectively. If the optimal disparity value is expressed as d*, then the maximum value formula d*=arg max C(y,x,d) can be used to calculate the optimal disparity value d* of the third pixel point (y,x).

[0105] Here, the existing method can be used to obtain the minimum matching cost of the pixel point and the corresponding disparity value, and the method to be developed in the future can also be used to obtain the minimum matching cost of the pixel point and the corresponding disparity value, which is not restricted here.

[0106] For example, for regional algorithms, after completing the superposition of matching costs, disparity can be easily obtained. It is only necessary to select the point with the best superposition matching cost within a certain range (the minimum value of SAD and SSD, or the maximum value of NCC) as the corresponding matching point, such as the Winner-take-all algorithm WTA. For global algorithms, the original matching cost is directly processed. Generally, an energy evaluation function is given first, and then the minimum energy value is obtained through different optimization algorithms. At the same time, the minimum matching cost of each pixel point and the disparity value corresponding to the minimum matching cost can be calculated.

[0107] For another example, for most stereo matching algorithms, the calculated disparities are discrete specific integer values, which can meet the accuracy requirements of general applications. However, in some situations where higher accuracy is required, such as accurate 3D reconstruction, it is necessary to use some measures to refine the disparity after the initial disparity is obtained, such as matching cost curve fitting, image filtering, image segmentation, etc., so that some more accurate matching costs and corresponding disparity values ​​can be obtained.

[0108] In a specific embodiment, the minimum matching cost and the corresponding disparity value of any pixel can be obtained by the extreme value verification method. According to the existing cost aggregation processing method, the cost aggregation function of multiple disparity values ​​is obtained, and the minimum value (i.e., matching cost) corresponding to each disparity value under the cost aggregation function is calculated. These matching costs are sorted in ascending order to obtain the minimum matching cost and the corresponding disparity value; another method is that after obtaining the matching cost corresponding to each disparity value through the cost aggregation function, the disparity value corresponding to each matching cost is checked for consistency between the left and right images, and the minimum matching cost and its corresponding disparity value are selected from the results that pass the verification. If the minimum matching cost of a pixel is represented by c, then the confidence of this pixel can be expressed as exp(-c). In addition, the disparity value of each pixel in the image obtained by the traditional method at the minimum matching cost may have some errors, resulting in the inaccuracy of the disparity value of individual pixels. Therefore, it is necessary to further weight some pixels with low disparity value accuracy based on the disparity value of each pixel to obtain a better disparity value estimation result.

[0109] Regarding the process of obtaining the cost aggregation function corresponding to each disparity value and obtaining the best disparity value of the third pixel, specific reference may be made to previously applied patent documents.

[0110] For example, in patent document CN2019101607989, a binocular vision stereo matching method and system based on matching cost weighting is disclosed. When implementing the above steps S410-S430, the third image and the fourth image involved in the present application can be used as the first image and the second image in the patent document respectively, so as to use the binocular vision stereo matching method disclosed in the patent document to calculate the optimal disparity value of the third pixel point.

[0111] For example, in patent document CN2019101614342, a binocular vision stereo matching method based on weighted voting and its system are disclosed. When implementing the above steps S410-S430, the third image and the fourth image involved in this application can be used as images under two viewpoints in the patent document, respectively, so as to use the binocular vision stereo matching method disclosed in the patent document to calculate the optimal disparity value of the third pixel point.

[0112] Those skilled in the art can understand that when applying the binocular vision stereo matching method disclosed in this embodiment, the following technical advantages can be achieved: (1) the first image and the second image under two viewpoints are respectively expanded with similar textures to obtain a third image and a fourth image containing a specific texture area, respectively. This can improve the correlation coefficient between the third image and the fourth image, overcome the shortcoming that the low texture content matching process between the two images is prone to errors, and greatly improve the performance of the binocular vision stereo matching algorithm based on the traditional method; (2) starting from the set pixel point, the other pixel points distributed around the first image and the second image are expanded with similar textures. Through this expansion method, a specific texture area containing structural information can be obtained. In this way, the third image and the fourth image required for binocular vision stereo matching can be easily formed based on the specific texture area, thereby utilizing the image The structural information, color information and texture information contained therein participate in the stereo matching process together to obtain a more accurate cost aggregation calculation result, thereby improving the accuracy of stereo matching; (3) a limited area is established in the first image and the second image respectively, and each pixel point with a similar texture to the set pixel point is obtained by judging the similar texture within the limited area. In this way, the pixel extension range can be standardized in the first image and the second image respectively to obtain a specific texture area with higher similarity, thereby improving the correlation coefficient between the images and the matching cost of subsequent stereo matching processing; (4) because the third pixel point obtained in the third image or the fourth image is cost aggregated according to the preset cost function and the preset multiple disparity values, the cost aggregation function with higher robustness can be obtained by cost aggregation, which is conducive to obtaining the best disparity value with higher accuracy from each disparity value.

[0113] Embodiment 2

[0114] The technical solution provided in this embodiment is an improvement on the binocular vision stereo matching method in Example 1. After acquiring the first image and the second image from two viewpoints and before setting the first pixel point in the first image, it also includes a process of performing image mirroring and extension processing on the first image and the second image respectively.

[0115] For details, please refer to Figure 5 , between step S100 and step S200, the first image and the second image are respectively subjected to image mirroring extension processing, which is represented by steps S510-S520.

[0116] Step S510, after the above step S100 is completed, this step can be directly entered, taking each edge pixel point of the first image as the axis of symmetry, and performing mirror extension processing on the first image along the axis of symmetry to extend it to the four sides of the first image to a preset height and a preset width.

[0117] For example, for Figure 7The first image shown in FIG. 1 can be obtained by performing mirror image extension processing on the first image. Figure 8 The image shown in Figure 8 In the figure, the dotted lines a, b, c, and d are all symmetry axes set at the edge pixel points of the first image. The dotted line a can realize the mirror extension processing along the symmetry axis, extending upward to a preset height; the dotted line b can realize the mirror extension processing along the symmetry axis, extending downward to a preset height; the dotted line c can realize the mirror extension processing along the symmetry axis, extending to the left to a preset width; the dotted line d can realize the mirror extension processing along the symmetry axis, extending to the right to a preset height.

[0118] It should be noted that the preset height is greater than or equal to the first limited area (for example Figure 8 The preset width is greater than or equal to the half-width value of the first limited area (such as the data R in the above step S220). By mirror-extending the first image, even if the first pixel point is set at the edge of the original first image, it can be ensured that the first limited area is always within the image range of the extended first image, and the situation where the first limited area cannot be established will not occur.

[0119] Step S520: Using each edge pixel point of the second image as an axis of symmetry, the second image is mirror-extended along the axis of symmetry to extend the second image to a preset height and a preset width around the second image.

[0120] Regarding the process of mirror-extending the second image along the axis of symmetry, specific reference may be made to the above step S510. It should be noted that in order to ensure the consistency of the mirror-extending processing of the first image and the second image, the size of the second limited area and the first limited area may also be required to be consistent. Then, the height of the second image extended to the surroundings is greater than or equal to half the height value of the second limited area (such as the value L in the above step S220), and the extended width is greater than or equal to half the width value of the second limited area (such as the data R in the above step S220).

[0121] See also Figure 5 , after step S520 is completed, step S200 can be directly entered.

[0122] Those skilled in the art will appreciate that, when applying the binocular vision stereo matching method disclosed in this embodiment, the following technical advantages can be achieved: (1) before the first image and the second image are extended with similar textures, the images are also extended along the axis of symmetry, so that the expandable range within the first image and the second image is increased, so that even the similar texture extension of the surrounding pixels using the edge pixels of the original image can be easily achieved; (2) The technical solution of the present application can effectively solve the problem of mismatching during stereo matching, which is conducive to accurately finding matching corresponding points in different viewpoint images, thereby improving the accuracy of stereo matching.

[0123] Those skilled in the art can also understand that the texture areas in the third image and the fourth image have the boundary shape information of the flat low-texture content, which is convenient for calculating the similarity of the two texture areas (i.e., NCC). According to the expansion result, the color in the texture area is the color and texture in the original expanded area, while the unexpanded area is completely black. In the left image of binocular vision, a first texture area represented by a pixel (y, x) will be expanded, and in the right image of binocular vision, a second texture area represented by a corresponding pixel (y, xd) will also be expanded. If the pixels in the left and right images cannot match, the shape of the expanded texture area is also different. In this way, when the NCC is used to calculate the correlation coefficient, a larger matching error (i.e., a smaller correlation coefficient NCC) will be obtained; conversely, in the corresponding matching, when the shape of the expanded texture area is the same, a relatively small matching error (i.e., a relatively large correlation coefficient NCC) can be obtained, thereby overcoming the disadvantage that low-texture content matching is prone to errors, thereby greatly improving the performance of the stereo matching algorithm of binocular vision based on traditional methods.

[0124] Embodiment 3

[0125] Based on the binocular vision stereo matching method in embodiment 1 or embodiment 2, this embodiment also provides an image stereo matching method, please refer to Figure 3 The image visual stereo matching method includes steps S210-S220, which are described below respectively.

[0126] Step S210, acquiring images from at least two viewpoints. In a specific embodiment, multiple cameras may be used to capture images of the stereo matching object, so that images from multiple viewpoints may be obtained.

[0127] Step S220, expand the similar textures of each image through the binocular vision stereo matching method disclosed in Example 1 or the embodiment, and perform stereo matching on each pixel point in an image after expansion to obtain the best disparity value of each pixel point.

[0128] Those skilled in the art can understand that the binocular vision stereo matching method in Example 1 or Example 2 obtains the optimal disparity value of a pixel point in the image, and the matching corresponding point in another image can be found based on the optimal disparity value. Then, the optimal disparity value of all pixel points in the image can be continued to be calculated according to the method, so that one-to-one stereo matching of pixels between two or more images can be achieved, thereby achieving the effect of image stereo matching.

[0129] Embodiment 4:

[0130] Please refer to Figure 6 Based on the binocular vision stereo matching method disclosed in Example 1 or Implementation 2, or the image stereo matching method in Example 3, the present application also discloses a stereo matching device 6, which may include a signal-connected memory 61 and a processor 62, which are described below respectively.

[0131] The memory 61 is used to store programs.

[0132] The processor 62 is used to implement the binocular vision stereo matching method disclosed in the first or second embodiment, or the image stereo matching method disclosed in the third embodiment, by executing the program stored in the memory 61 .

[0133] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0134] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A binocular vision stereo matching method based on regional feature extraction, characterized in that: include: Acquire a first image and a second image at two viewpoints; A first pixel point is set in the first image, and other pixel points distributed around the first pixel point are expanded with similar textures to form a first texture area, and a third image is obtained by using the first texture area and the unexpanded area outside the first texture area; A second pixel point is set in the second image, and other pixel points distributed around the second pixel point are expanded with similar textures to form a second texture area, and a fourth image is obtained by using the second texture area and the unexpanded area outside the second texture area; Performing cost aggregation on any pixel point in the third image or the fourth image according to a preset cost function and a plurality of preset disparity values, so as to obtain an optimal disparity value of the pixel point from the respective disparity values; After acquiring the first image and the second image at two viewpoints and before setting the first pixel point in the first image, the method further includes: Taking each edge pixel point of the first image as a symmetry axis, performing a mirror image extension process on the first image along the symmetry axis, so as to extend the first image to a preset height and a preset width around the first image; Each edge pixel point of the second image is used as a symmetry axis, and the second image is mirror-extended along the symmetry axis to extend to a preset height and a preset width around the second image.

2. The binocular vision stereo matching method as claimed in claim 1, characterized in that: A first pixel point is set in the first image, and other pixel points distributed around the first pixel point are extended with similar textures to form a first texture area, and a third image is obtained by using the first texture area and an unextended area outside the first texture area, including: Determine low texture content in the first image and set a first pixel point therein, and use the first pixel point as a center point to establish a first limited area; Within the scope of the first limited area, starting from the first pixel point, sequentially judging the similar textures of the continuous pixel points distributed longitudinally and transversely, and obtaining the pixel points having the similar textures to the first pixel point; Using each pixel point having a texture similar to that of the first pixel point to form a first texture region, and marking each pixel point in the first texture region as a first value; The area outside the first texture area and within the first limited area is regarded as a non-expanded area, and each pixel point in the non-expanded area is marked as a second value; A third image is generated according to each pixel point marked as the first value and each pixel point marked as the second value in the first image.

3. The binocular vision stereo matching method as claimed in claim 2, characterized in that: The method of determining similar textures of continuous pixel points distributed longitudinally and transversely within the first limited area starting from the first pixel point in sequence to obtain pixel points having similar textures to the first pixel point includes: Starting from the first pixel point, the continuous pixel points distributed along the longitudinal direction are sequentially expanded, and when it is determined that the grayscale difference between a pixel point and the last expanded pixel point is less than a preset threshold, the pixel point is confirmed to be a pixel point with a similar texture to the first pixel point, until the next pixel point in the longitudinal direction is confirmed to be a pixel point with a non-similar texture; Starting from each pixel point with similar texture obtained in the vertical direction, the continuous pixel points distributed along the horizontal direction are expanded in sequence. When it is determined that the grayscale value between a pixel point and the last expanded pixel point is less than the threshold, the pixel point is confirmed to be a pixel point with similar texture to the first pixel point, until the next pixel point in the horizontal direction is confirmed to be a pixel point with non-similar texture.

4. The binocular vision stereo matching method as claimed in claim 2, characterized in that: The method of setting a second pixel point in the second image, extending other pixel points distributed around the second pixel point with similar textures to form a second texture area, and obtaining a fourth image using the second texture area and an unextended area outside the second texture area includes: According to the position of the first pixel point in the first image, a second pixel point is correspondingly set in the second image, and the second pixel point is used as the center point to establish a second limited area; Within the scope of the second limited area, starting from the second pixel point, sequentially judging the similar textures of the continuous pixel points distributed longitudinally and transversely, and obtaining each pixel point having a similar texture to the second pixel point; forming a second texture region by using each pixel point having a texture similar to the second pixel point, and marking each pixel point in the second texture region as a first value; The area outside the second texture area and within the second limited area is regarded as a non-expanded area, and each pixel point in the non-expanded area is marked as a second value; A fourth image is generated according to each pixel point marked as the first value and each pixel point marked as the second value in the second image.

5. The binocular vision stereo matching method as described in claim 4, characterized in that: In the third image and the fourth image, each pixel point marked as the first value has the same grayscale value as its original pixel point, and each pixel point marked as the second value has a completely black grayscale value.

6. The binocular vision stereo matching method as described in claim 4, characterized in that: The preset height is greater than or equal to half the height of the first limited area, and the preset width is greater than or equal to half the width of the first limited area; the second limited area is consistent in size with the first limited area.

7. The binocular vision stereo matching method according to any one of claims 1 to 6, characterized in that: The cost aggregation is performed on any pixel point in the third image or the fourth image according to a preset cost function and a plurality of preset disparity values ​​to obtain an optimal disparity value of the pixel point from each of the disparity values, including: Acquire any pixel point from the third image or the fourth image and set it as a third pixel point; Performing cost aggregation on the third pixel point according to a preset cost function and a plurality of preset disparity values ​​to obtain a cost aggregation function corresponding to each disparity value; The best disparity value of the third pixel is obtained from each disparity value using a cost aggregation function corresponding to each disparity value.

8. The binocular vision stereo matching method as claimed in claim 7, characterized in that: The cost function is a cost function corresponding to color, gradient, rank or NCC; the disparity value is represented by d and has a value range of {0, 1, ..., d max }, where d max Indicates the maximum allowed value of the disparity value.

9. The binocular vision stereo matching method as claimed in claim 8, characterized in that: The step of obtaining the best disparity value of the third pixel from each disparity value by using the cost aggregation function corresponding to each disparity value includes: The cost aggregation function corresponding to each disparity value is calculated and expressed as C(y, x, d), and the disparity value input when the minimum function value is satisfied is obtained, and the disparity value is used as the optimal disparity value; wherein y and x are the ordinate value and abscissa value of the third pixel point, respectively.

10. A computer-readable storage medium, characterized in that: The invention comprises a program which can be executed by a processor to implement the binocular vision stereo matching method as described in any one of claims 1 to 9.

Images