Stereo matching method, device and storage medium based on slope cost aggregation
By adopting multiple rounds of loop iteration operations based on inclined cost aggregation in the stereo matching method, the problems of large computing resources and poor real-time performance in the prior art are solved, and efficient and accurate matching results and good real-time performance are achieved.
Patent Information
- Application Number
- CN202210338837.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-01
- Publication Date
- 2025-05-06
- Estimated Expiration
- 2042-04-01
AI Technical Summary
The existing stereo matching methods consume a lot of computing resources and have poor real-time performance, making it difficult to achieve efficient image processing in applications such as autonomous driving and robot control.
A three-dimensional matching method based on inclined cost aggregation is adopted, through multiple rounds of loop iteration operations, multiple inclined surfaces are constructed according to the target inclined parameters and disparity map, and lookup operations and adaptive aggregation are performed to reduce calculation and memory costs.
This method can efficiently aggregate the cost of matching in local 3D space, improve matching accuracy, and reduce computing and memory consumption, achieving good real-time and fast computing speed.
Smart Images
Figure CN114708219B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a method, a computer device and a storage medium for stereo matching based on slope cost aggregation. Background Art
[0002] Stereo matching of images is required in technologies such as autonomous driving, robot control, and augmented reality. Stereo matching is the process of one-to-one correspondence between a pair of stereo images at the pixel level. Existing stereo matching algorithms often follow a four-step process of matching cost calculation, matching cost aggregation, disparity regression, and disparity refinement. In the stereo matching process, the matching cost usually exhibits ambiguous properties in areas such as reflection, low texture, and thin structure of the image. At this time, the matching cost is ambiguous, and matching cost aggregation can alleviate the ambiguity of the matching cost. Therefore, matching cost aggregation is particularly important for accurate disparity estimation.
[0003] Traditional stereo matching methods can be divided into three types: local methods, global methods, and semi-global methods. Local algorithms usually aggregate matching costs in local areas, while global algorithms minimize the global energy function over the entire image to construct disparity estimates. Semi-global algorithms aggregate matching costs in different directions over the entire image, which greatly reduces the computational complexity compared to global algorithms, and has higher accuracy than local algorithms. In recent years, deep learning methods have been proposed to solve the stereo matching problem. Early methods extracted deep abstract features of images by utilizing the powerful feature extraction capabilities of convolutional neural networks, but still required traditional cost aggregation methods for subsequent processing. Currently, stereo matching methods using deep learning are generally divided into two types. The 3D cost volume obtained by dot multiplication of the left and right feature maps is aggregated using the 2D CNN method; the 4D cost volume obtained by concatenating the left and right feature maps is aggregated using the 3D CNN method. At present, the method of using 3D CNN for matching cost aggregation has become the mainstream of stereo matching. Although 3D CNN can produce accurate disparity maps, its high computational and video memory consumption limits their further application. Summary of the invention
[0004] In view of at least one technical problem that the current stereo matching technology consumes more computing resources and has poor real-time performance, the object of the present invention is to provide a stereo matching method, a computer device and a storage medium based on slope cost aggregation.
[0005] On the one hand, an embodiment of the present invention further includes a method for stereo matching based on slope cost aggregation, comprising the following steps:
[0006] Acquire a first image and a second image; the first image and the second image may form a stereo pair image;
[0007] Performing feature map matrix multiplication processing on the first image and the second image to obtain a first cost volume;
[0008] Performing initial disparity estimation on the first cost volume to obtain an initial disparity map;
[0009] Perform multiple rounds of loop iteration operations; in each round of the loop iteration operation, construct multiple slopes of this round according to the target slope parameters and the target disparity map, perform a lookup operation in the neighborhood centered on the pixel in the first cost volume according to each slope, trace back to obtain the traced cost volume of this round, perform adaptive aggregation on the traced cost volume, obtain the context feature map of this round, input the target disparity map, the traced cost volume of this round and the context feature map of this round to the gate activation unit, and the gate activation unit outputs the disparity map of this round, the slope parameters of this round and the upmask map of this round;
[0010] Wherein, for the first round of the cyclic iteration operation, the target slope parameter is the initially set slope parameter, and the target disparity map is the initial disparity map; for each round of the cyclic iteration operation other than the first round of the cyclic iteration operation, the target slope parameter is the slope parameter obtained by the previous round of the cyclic iteration operation, and the target disparity map is the disparity map obtained by the previous round of the cyclic iteration operation;
[0011] The disparity map and upmask map obtained by the last round of loop iteration operation are processed to obtain the original resolution disparity map.
[0012] Furthermore, the stereo matching method based on slope cost aggregation further includes:
[0013] Before performing feature map matrix multiplication processing on the first image and the second image, epipolar line correction processing is performed on the first image and the second image.
[0014] Furthermore, performing feature map matrix multiplication processing on the first image and the second image to obtain a first cost volume includes:
[0015] Performing multiple feature map extraction operations on the first image and the second image in sequence to obtain a first feature map corresponding to the first image and a second feature map corresponding to the second image;
[0016] Perform matrix multiplication on the first feature map and the second feature map to obtain the first cost volume.
[0017] Further, the performing a plurality of feature map extraction operations on the first image and the second image in sequence to obtain a first feature map corresponding to the first image and a second feature map corresponding to the second image includes:
[0018] Perform a convolution with kernel=7 and stride=2 on the first image to obtain a feature map featuremap101; perform a convolution with kernel=7 and stride=2 on the second image to obtain a feature map featuremap102;
[0019] After performing a convolution with stride=1 on feature map 101, feature map 201 is obtained through a residual connection operation; after performing a convolution with stride=1 on feature map 102, feature map 202 is obtained through a residual connection operation;
[0020] After performing a convolution with stride=2 on feature map 101, feature map 301 is obtained through a residual connection operation; after performing a convolution with stride=2 on feature map 202, feature map 302 is obtained through a residual connection operation;
[0021] After performing a convolution with stride=2 on feature map 301, feature map 401 is obtained through a residual connection operation; after performing a convolution with stride=2 on feature map 302, feature map 402 is obtained through a residual connection operation;
[0022] A convolution with stride=1 is performed on the feature map 401 to obtain the first feature map 501; a convolution with stride=1 is performed on the feature map 402 to obtain the second feature map 502.
[0023] Furthermore, the step of constructing multiple slopes of the current round according to the target slope parameters and the target disparity map includes:
[0024] For the pixel p in the first cost volume, construct a slope set {f i}; where the slope f i The expression for f i =a p (xx p )+bp (yy p )+d p +i,a p and b p is the target slope parameter corresponding to pixel p, x p and p The coordinates of pixel p, x and y are the slope f i The coordinates of the point on p is the target disparity map corresponding to pixel p, i and slope f i The serial number information is related.
[0025] Furthermore, the initially set slope parameters are all zero.
[0026] Furthermore, the disparity map and upmask map obtained by the last round of iterative operation are processed to obtain the disparity map with the original resolution, including:
[0027] Perform dimension transformation on the disparity map obtained from the last round of loop iteration, and perform unfold operation on the upmask map obtained from the last round of loop iteration;
[0028] The disparity map after the dimension transformation is multiplied and weighted-added with the upmask map after the unfold operation to obtain the original resolution disparity map.
[0029] Furthermore, the method for stereo matching based on slope cost aggregation further includes the following steps:
[0030] One-dimensional pooling down-sampling is performed on the last dimension of the first cost volume to obtain a second cost volume and a third cost volume.
[0031] On the other hand, an embodiment of the present invention also includes a computer device, including a memory and a processor, wherein the memory is used to store at least one program, and the processor is used to load the at least one program to execute the method of stereo matching based on slope cost aggregation in the embodiment.
[0032] On the other hand, an embodiment of the present invention further includes a storage medium storing a program executable by a processor, wherein the program executable by the processor is used to execute the method of stereo matching based on slope cost aggregation in the embodiment when executed by the processor.
[0033] The beneficial effects of the present invention are as follows: the method for stereo matching based on slope cost aggregation in the embodiment constructs multiple slopes of this round according to the target slope parameters and the target disparity map in each round of cyclic iteration operation, and performs a lookup operation according to the relationship between each slope and the first cost body. The parameters of the slope can be updated in the next round of cyclic iteration operation, that is, the slope is learnable, and can aggregate the matching cost from the local 3D space, reducing the calculation and memory costs, and having higher matching accuracy. Moreover, since the slope aggregation reduces the high consumption cost, the method in the present embodiment has good real-time performance and can achieve a very fast computing speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 It is a flowchart of a stereo matching method based on slope cost aggregation in an embodiment;
[0035] Figure 2 A schematic diagram of obtaining the first characteristic graph and the second characteristic graph in the embodiment;
[0036] Figure 3 It is a schematic diagram of each round of iterative operation in the embodiment;
[0037] Figure 4 It is a comparison diagram of the principles of the stereo matching method based on slope cost aggregation in the embodiment and the prior art. DETAILED DESCRIPTION
[0038] In this embodiment, refer to Figure 1 , the stereo matching method based on slope cost aggregation includes the following steps:
[0039] S1. Acquire a first image and a second image;
[0040] S2. Perform feature map matrix multiplication on the first image and the second image to obtain a first cost volume;
[0041] S3. Perform an initial disparity estimation on the first cost volume to obtain an initial disparity map;
[0042] S4. Perform multiple rounds of loop iteration operations; in each round of loop iteration operations, multiple slopes of this round are constructed according to the target slope parameters and the target disparity map, and according to each slope, a lookup operation is performed in the neighborhood centered on the pixel in the first cost volume, and the tracing cost volume of this round is traced back, and adaptive aggregation is performed on the tracing cost volume to obtain the context feature map of this round, and the target disparity map, the tracing cost volume of this round, and the context feature map of this round are input into the gate activation unit, and the gate activation unit outputs the disparity map of this round, the slope parameters of this round, and the upmask map of this round;
[0043] Among them, for the first round of cyclic iteration operation, the target slope parameter is the initially set slope parameter, and the target disparity map is the initial disparity map; for each round of cyclic iteration operation except the first round of cyclic iteration operation, the target slope parameter is the slope parameter obtained by the previous round of cyclic iteration operation, and the target disparity map is the disparity map obtained by the previous round of cyclic iteration operation;
[0044] S5. Process the disparity map and upmask map obtained by the last round of loop iteration operation to obtain the original resolution disparity map.
[0045] In this embodiment, steps S1-S5 may be executed by a computer.
[0046] In step S1, a pair of horizontally placed cameras installed on the vehicle can be used to capture a first image and a second image at the same time. When viewed from the camera in the shooting direction, the first image can be captured by the camera on the left, and the second image can be captured by the camera on the right, so that the first image and the second image form a stereo pair image.
[0047] In step S2, the first cost volume C1 is obtained by performing feature map matrix multiplication on the first image and the second image. Specifically, before executing step S2, the first image and the second image are first subjected to epipolar line correction, that is, the first image and the second image that are subjected to epipolar line correction may be processed in step S2.
[0048] When executing step S2, the following steps may be specifically performed:
[0049] S201. Perform multiple feature map extraction operations on the first image and the second image respectively, to obtain a first feature map corresponding to the first image and a second feature map corresponding to the second image;
[0050] S202. Perform matrix multiplication on the first feature map and the second feature map to obtain a first cost volume.
[0051] When executing step S201, refer to Figure 2 , perform the following steps:
[0052] Perform a convolution with kernel=7 and stride=2 on the first image to obtain feature map feature map101; perform a convolution with kernel=7 and stride=2 on the second image to obtain feature map feature map102;
[0053] After performing a convolution with stride=1 on feature map 101, feature map 201 is obtained through a residual connection operation; after performing a convolution with stride=1 on feature map 102, feature map 202 is obtained through a residual connection operation;
[0054] After performing a convolution with stride=2 on feature map 101, feature map 301 is obtained through a residual connection operation; after performing a convolution with stride=2 on feature map 202, feature map 302 is obtained through a residual connection operation;
[0055] After performing a convolution with stride=2 on feature map 301, feature map 401 is obtained through a residual connection operation; after performing a convolution with stride=2 on feature map 302, feature map 402 is obtained through a residual connection operation;
[0056] A convolution with stride=1 is performed on the feature map feature map401 to obtain a first feature map feature map501; a convolution with stride=1 is performed on the feature map feature map402 to obtain a second feature map feature map502.
[0057] When the first image is processed as Figure 2 After multiple convolution and residual connection operations as shown, the obtained feature map feature map 501 is used as the first feature map corresponding to the first image; Figure 2 After the multiple convolution and residual connection operations shown, the obtained feature map feature map 502 is used as the second feature map corresponding to the second image.
[0058] In step S202, the first feature map 501 and the second feature map 502 are matrix multiplied to obtain a first cost volume C1. Specifically, before performing the matrix multiplication, the dimension of the first feature map 501 can be transformed into [b, h, w, c], and the dimension of the second feature map 502 can be transformed into [b, h, c, w], so that the dimension of the first cost volume C1 obtained is [b, h, w, w].
[0059] In step S3, a softmax operation is performed on the first cost volume C1, so as to achieve an initial disparity estimation for the first cost volume C1 and obtain an initial disparity map D1.
[0060] In step S4, multiple rounds of loop iteration operations are performed. In each round of loop iteration operations, multiple slopes of this round are constructed according to the target slope parameters and the target disparity map. According to each slope, a lookup operation is performed in the neighborhood centered on the pixel in the first cost volume to trace back to obtain the tracing cost volume of this round, and adaptive aggregation is performed on the tracing cost volume to obtain the context feature map of this round. The target disparity map, the tracing cost volume of this round, and the context feature map of this round are input to the gate activation unit, and the gate activation unit outputs the disparity map of this round, the slope parameters of this round, and the upmask map of this round.
[0061] Specifically, refer to Figure 3 For the first round of cyclic iteration operation, the "target slope parameter" is the initially set slope parameter. Specifically, the value of each specific parameter in the initially set slope parameter can be 0, and the "target disparity map" is the initial disparity map D1. For each round of cyclic iteration operation other than the first round of cyclic iteration operation (for example, the mth round of cyclic iteration operation, m=2, 3, 4...), the target slope parameter is the slope parameter obtained by the previous round of cyclic iteration operation (the m-1th round of cyclic iteration operation), and the target disparity map is the disparity map obtained by the previous round of cyclic iteration operation (the m-1th round of cyclic iteration operation).
[0062] Taking the mth round of loop iteration operation as an example, in the mth round of loop iteration operation, for the pixel p in the first cost volume C1, the slope parameter output by the m-1th round of loop iteration operation can be and (If m = 1, that is and is the initial slope parameter, then you can set ), the disparity map related to pixel p output by the m-1th round of iterative operation and the coordinate x of pixel p p and p , according to the equation Construct the slope set {f i}, where i is the slope f i Specifically, i can be a positive integer used to represent the sequence number. When i takes multiple different values, a series of slopes can be obtained.
[0063] In the mth round of loop iteration, a series of slopes {f i}, a lookup operation is performed on the neighborhood (specifically, the size can be 3×3) centered on the pixel p in the first cost volume C1, and the result obtained by tracing is called the tracing cost volume. In the mth round of loop iteration operation, adaptive aggregation is performed on the tracing cost volume obtained in this round. Specifically, the feature map generated by the context module can be used to learn the parameters of the 3*3 aggregation filter, and then these filters are used as a guide to aggregate the cost volume, thereby obtaining the context feature map of this round.
[0064] In the mth round of loop iteration, the target disparity map (the disparity map obtained by the m-1th round of loop iteration), the current round of tracing cost body and the current round of context feature map are input to the GRU update module. Specifically, the core of the GRU update module can be the gate activation unit. The GRU update module processes and outputs the disparity map of this round, the slope parameters of this round and the upmask map of this round, thereby completing the mth round of loop iteration.
[0065] In this embodiment, the total number of loop iteration operations to be performed can be pre-set, for example, the total number of rounds is set to q. After executing the qth round of loop iteration operation, the next round of loop iteration operation may not be executed, that is, the qth round of loop iteration operation is the last round of loop iteration operation.
[0066] In step S5, the disparity map and upmask map obtained by the last round of loop iteration operation are obtained. The disparity map and upmask map obtained by the last round of loop iteration operation are processed as follows:
[0067] S501. Performing a dimensional transformation on the disparity map obtained by the last round of loop iteration operation, and performing an unfold operation on the upmask map obtained by the last round of loop iteration operation;
[0068] S502. Multiply and weightedly add the dimensionally transformed disparity map and the upmask map that has undergone the unfolding operation to obtain a disparity map of original resolution.
[0069] For example, in step S501, the disparity map obtained by the last round of loop iteration operation is dimensionalized, so that the dimension of the disparity map becomes [b, 9, 8, 8, h, w], and the upmask map obtained by the last round of loop iteration operation is unfolded, so that the dimension of the upmask map becomes [9, 1, 1, h, w]. In step S502, the disparity map after dimensionalization transformation and the upmask map are multiplied and weighted added to obtain the upsampled disparity map D up , that is, the original resolution disparity map to be obtained in step S5.
[0070] In this embodiment, the last dimension of the first cost volume C1 can also be subjected to pooling downsampling processing, so as to obtain the second cost volume C2 and the third cost volume C3. The first cost volume C1, the second cost volume C2 and the third cost volume C3 can form a matching cost volume pyramid, which can provide a dot product similarity relationship of large displacement and small displacement at the same time.
[0071] In this embodiment, the principle of the stereo matching method based on slope cost aggregation is: Figure 4 , where (a) represents the principle of the 2D CNN method in the prior art, and (b) represents the principle of the 3D CNN method in the prior art. The traditional stereo matching method shows that the tilted support window in the 3D space helps to aggregate the matching cost from the information area (i.e., the surface of the object); inspired by this idea, the stereo matching method based on slope cost aggregation in this embodiment is as follows Figure 4 As shown in part (c), in each round of loop iteration operation, multiple slopes of this round are constructed according to the target slope parameters and the target disparity map, and a lookup operation is performed according to the relationship between each slope and the first cost body. The parameters of the slope can be updated in the next round of loop iteration operation. That is, the slope is learnable and can aggregate the matching cost from the local 3D space, reducing the calculation and memory costs and having higher matching accuracy. Moreover, since the slope aggregation reduces the high consumption cost, the method in this embodiment has good real-time performance and can achieve a very fast computing speed.
[0072] In this embodiment, the original resolution disparity map finally obtained by the stereo matching method based on slope cost aggregation includes the disparity value corresponding to each pixel in the reference image (one of the first image or the second image, generally the reference image refers to the first image taken by the camera on the seat), and the disparity value is the pixel-level difference between the positions of corresponding points of a certain point in the three-dimensional scene in the left and right images, and includes depth of field information, thereby being able to provide support for fields that require depth of field information, such as autonomous driving.
[0073] In this embodiment, the camera for capturing the first image and the second image may be an RGB camera, which has the advantages of low price, rich information content, and small size, and is conducive to realizing technologies such as autonomous driving at low cost.
[0074] A computer program for executing the method for stereo matching based on slope cost aggregation in the present embodiment can be written and written into a computer device or a storage medium. When the computer program is read out and executed, the method for stereo matching based on slope cost aggregation in the present embodiment is executed, thereby achieving the same technical effect as the method for stereo matching based on slope cost aggregation in the embodiment.
[0075] It should be noted that, unless otherwise specified, when a feature is referred to as "fixed" or "connected" to another feature, it may be directly fixed or connected to another feature, or it may be indirectly fixed or connected to another feature. In addition, the descriptions of up, down, left, right, etc. used in this disclosure are only relative to the relative positional relationship of the components of the present disclosure in the accompanying drawings. The singular forms of "a", "said" and "the" used in this disclosure are also intended to include the plural forms, unless the context clearly indicates other meanings. In addition, unless otherwise defined, all technical and scientific terms used in this embodiment have the same meaning as those generally understood by technicians in this technical field. The terms used in the specification of this embodiment are only for describing specific embodiments, not for limiting the present invention. The term "and / or" used in this embodiment includes any combination of one or more related listed items.
[0076] It should be understood that, although the term first, second, third etc. may be adopted to describe various elements in the present disclosure, these elements should not be limited to these terms. These terms are only used to distinguish the same type of elements from each other. For example, without departing from the scope of the present disclosure, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element. The use of any and all examples or exemplary language ("for example", "such as" etc.) provided by the present embodiment is only intended to better illustrate embodiments of the present invention, and unless otherwise required, the scope of the present invention will not be limited.
[0077] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in an assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed ASIC for this purpose.
[0078] In addition, the operations of the process described in this embodiment may be performed in any suitable order, unless otherwise indicated in this embodiment or otherwise clearly contradicted by the context. The process described in this embodiment (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as a code (e.g., executable instructions, one or more computer programs, or one or more applications) executed jointly on one or more processors, by hardware or a combination thereof. The computer program includes a plurality of instructions that may be executed by one or more processors.
[0079] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or a portion thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described in this embodiment includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.
[0080] The computer program can be applied to input data to perform the functions described in the present embodiment, thereby converting the input data to generate output data stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on the display.
[0081] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation. As long as the technical effect of the present invention is achieved by the same means, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of protection of the present invention. Within the scope of protection of the present invention, its technical scheme and / or implementation method may have various modifications and changes.
Claims
1. A stereo matching method based on slope cost aggregation, characterized in that: The method for stereo matching based on slope cost aggregation comprises the following steps: Acquire a first image and a second image; the first image and the second image form a stereo pair image; Performing feature map matrix multiplication processing on the first image and the second image to obtain a first cost volume; Performing initial disparity estimation on the first cost volume to obtain an initial disparity map; Perform multiple rounds of loop iteration operations; in each round of the loop iteration operation, construct multiple slopes of this round according to the target slope parameters and the target disparity map, perform a lookup operation in the neighborhood centered on the pixel in the first cost volume according to each slope, trace back to obtain the traced cost volume of this round, perform adaptive aggregation on the traced cost volume, obtain the context feature map of this round, input the target disparity map, the traced cost volume of this round and the context feature map of this round to the gate activation unit, and the gate activation unit outputs the disparity map of this round, the slope parameters of this round and the upmask map of this round; Wherein, for the first round of the cyclic iteration operation, the target slope parameter is the initially set slope parameter, and the target disparity map is the initial disparity map; for each round of the cyclic iteration operation other than the first round of the cyclic iteration operation, the target slope parameter is the slope parameter obtained by the previous round of the cyclic iteration operation, and the target disparity map is the disparity map obtained by the previous round of the cyclic iteration operation; The disparity map and upmask map obtained by the last round of loop iteration operation are processed to obtain the original resolution disparity map.
2. The method for stereo matching based on slope cost aggregation according to claim 1, characterized in that: The method for stereo matching based on slope cost aggregation further includes: Before performing feature map matrix multiplication processing on the first image and the second image, epipolar line correction processing is performed on the first image and the second image.
3. The method for stereo matching based on slope cost aggregation according to claim 1 or 2, characterized in that: The performing feature map matrix multiplication processing on the first image and the second image to obtain a first cost volume includes: Performing multiple feature map extraction operations on the first image and the second image in sequence to obtain a first feature map corresponding to the first image and a second feature map corresponding to the second image; Perform matrix multiplication on the first feature map and the second feature map to obtain the first cost volume.
4. The method for stereo matching based on slope cost aggregation according to claim 3, characterized in that: The step of performing multiple feature map extraction operations on the first image and the second image in sequence to obtain a first feature map corresponding to the first image and a second feature map corresponding to the second image includes: Perform a convolution with kernel=7 and stride=2 on the first image to obtain a feature map feature map101; perform a convolution with kernel=7 and stride=2 on the second image to obtain a feature map feature map102; After performing a convolution with stride=1 on feature map 101, feature map 201 is obtained through a residual connection operation; after performing a convolution with stride=1 on feature map 102, feature map 202 is obtained through a residual connection operation; After performing a convolution with stride=2 on feature map 101, feature map 301 is obtained through a residual connection operation; after performing a convolution with stride=2 on feature map 202, feature map 302 is obtained through a residual connection operation; After performing a convolution with stride=2 on feature map 301, feature map 401 is obtained through a residual connection operation; after performing a convolution with stride=2 on feature map 302, feature map 402 is obtained through a residual connection operation; A convolution with stride=1 is performed on the feature map feature map401 to obtain the first feature map feature map501; a convolution with stride=1 is performed on the feature map feature map402 to obtain the second feature map feature map502.
5. The method for stereo matching based on slope cost aggregation according to claim 1, characterized in that: The method of constructing multiple slopes of this round according to the target slope parameters and the target disparity map includes: For pixels in the first cost volume , construct a slope set ; Among them, the inclined plane The expression is , and Pixel The corresponding target slope parameters, and Pixel The coordinates of and For inclined surface The coordinates of the points on Pixel The corresponding target disparity map, with bevel The serial number information is related.
6. The method for stereo matching based on slope cost aggregation according to claim 5, characterized in that: The initially set slope parameters are all zero.
7. The method for stereo matching based on slope cost aggregation according to claim 1, characterized in that: The disparity map and upmask map obtained by the last round of iterative operation are processed to obtain the disparity map of the original resolution, including: Perform dimension transformation on the disparity map obtained from the last round of loop iteration, and perform unfold operation on the upmask map obtained from the last round of loop iteration; The disparity map after the dimension transformation is multiplied and weighted-added with the upmask map after the unfold operation to obtain the original resolution disparity map.
8. The method for stereo matching based on slope cost aggregation according to claim 1, characterized in that: The method for stereo matching based on slope cost aggregation also includes the following steps: One-dimensional pooling down-sampling is performed on the last dimension of the first cost volume to obtain a second cost volume and a third cost volume.
9. A computer device, characterized in that: The invention comprises a memory and a processor, wherein the memory is used to store at least one program, and the processor is used to load the at least one program to execute the method for stereo matching based on slope cost aggregation according to any one of claims 1 to 8.
10. A storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to execute the slope cost aggregation-based stereo matching method described in any one of claims 1 to 8 when executed by the processor.