Real-time binocular stereo matching method and automotive suspension control system based on same

Abstract

The invention provides a real-time binocular stereo matching method and an automobile suspension control system based on the method. Comprising the following steps: acquiring multi-scale features of left and right views based on a feature extraction network, and representing the multi-scale features by a plurality of three-dimensional tensors, so as to construct three-fusion cost volumes (texture volume, correlation volume and mutual volume) based on a low-level feature map with 1 / 4 resolution for realizing cost regression; and then realizing cost regression through hourglass-shaped stacked 3D convolution in combination with feature maps of other levels, and finally performing parallax regression under the 1 / 4 resolution and the original resolution and performing supervised learning to obtain a parallax map under the original resolution. Based on the obtained disparity map and a parameter matrix of the binocular camera, a depth map of the road surface can be calculated, depth differences of front and back frames are compared, the change of the undulation degree of the road surface can be calculated by utilizing a geometrical relationship, and then the change is used as input to carry out hierarchical control on the suspension, so that the driving comfort is ensured.

Description

Technical Field

[0001] The present invention relates to the field of automobile technology, and in particular to a real-time binocular stereo matching method and an automobile suspension control system based on the method. Background Art

[0002] In the field of intelligent driving, binocular stereo matching is an important part of environmental perception technology and is of great significance for automobile suspension control. Specifically, it uses a pair of calibrated binocular cameras to collect environmental images and simulate the visual effect of the human eye, so as to accurately locate the pixel points corresponding to the left view in the right view. In the suspension control task, accurately predicting the undulation of the road surface is of great guiding significance for active suspension control.

[0003] In recent years, many researchers have devoted themselves to the study of binocular stereo matching, aiming to find efficient and accurate matching methods. The current stereo matching methods mainly include four modules: feature extraction with shared weights, cost volume construction, cost volume aggregation, and disparity regression. Among them, the cost volume construction is to encode the feature map into a 4D tensor containing the similarity information of the left and right views, providing a rough representation for matching pixel pairs, so the richness and accuracy of the cost volume information constructed in this process basically determines the final stereo matching performance.

[0004] However, the existing construction strategy only considers the correlation between the left and right views, ignoring the similarity information ambiguity caused by strong light reflection and ill-posed regions formed by large-scale repetition of some areas, which greatly limits the model's ability to accurately model complex areas. In addition, the existing strategy only considers the matching cost when the pixels in the left and right feature maps correspond one-to-one, ignoring two key issues: first, when the left and right feature maps are used as disparity probability distributions, the similarity information between their overall distributions is extremely important for cost volume construction; second, due to different shooting angles, a single pixel in the left view may correspond to multiple pixels in the right view.

[0005] At the same time, the automobile suspension system is crucial to the vehicle's driving performance and ride comfort. It not only affects the vehicle's stability and handling under different road conditions, but also affects the passengers' comfort experience. High-quality suspension control methods can effectively absorb road impacts, reduce the amplitude of vibration transmitted to the seat, and improve driving safety and comfort. In addition, with the development of automobile technology, the intelligentization and automation of the suspension system have become important trends in the future. Adjusting the suspension posture according to real-time road conditions to achieve optimal driving performance is an important research direction at present.

[0006] Although the existing intelligent suspension control technology has made some progress, it still has some shortcomings. On-board cameras have been widely used in various types of vehicles, but most of the current control strategies do not make full use of advanced environmental perception technology. They only combine body sensors to perform active or semi-active control indirectly based on the changes in the vehicle's posture.

[0007] Through searching patent documents, it was found that the invention patent with application number CN202410355410.1 discloses a binocular depth estimation method based on automatic network compression. This method obtains the left view and the right view and their corresponding disparity maps, and constructs a binocular depth estimation model including NASCNet and NASSNet. The model training adopts a progressive search strategy. First, the network with the largest structure is trained, and then the convolution kernel size, model depth and width are automatically searched in turn to better fit the data. This method has limitations in the optimization of the cost volume construction strategy, resulting in limited matching accuracy.

[0008] In summary, in view of the above-mentioned problems of the prior art, studying a real-time binocular stereo matching method and a vehicle suspension control system based on the method has become a key task that needs to be solved urgently. Summary of the invention

[0009] In view of the defects in the prior art, the object of the present invention is to provide a real-time binocular stereo matching method and a vehicle suspension control system based on the method.

[0010] A real-time binocular stereo matching method based on three-fusion cost volumes provided by the present invention comprises the following steps:

[0011] S1: By processing the original binocular image, a multi-level feature map in the form of a three-dimensional tensor of different scales is obtained to obtain semantic information and texture information at different levels;

[0012] S2: Based on the original binocular image and the low-level feature map of the multi-level feature map, a three-fusion cost volume is constructed to construct the similarity and correlation of the left and right views in the depth direction;

[0013] S3: Based on the three-fusion cost volume and other level feature maps of the multi-level feature map, downsampling-upsampling is performed layer by layer based on 3D convolution in cascade according to the tensor size to achieve aggregation of the cost volume and obtain the aggregated cost volume;

[0014] S4: Input the aggregated cost volume into the dual-scale disparity regression module, use the arg-softmax function, and output the disparity maps at 1 / 4 resolution and original resolution respectively;

[0015] S5: Calculate the depth map of the road surface using the binocular camera parameters and the disparity map. Compare the depth changes in the depth map under consecutive frames and calculate the change in road surface undulation according to the geometric relationship as the input signal for subsequent suspension control.

[0016] Preferably, step S1 includes the following sub-steps:

[0017] Step S1.1, regularizing the original binocular image and expanding it into a four-dimensional tensor to obtain a processed image;

[0018] Step S1.2, the processed image is passed through a lightweight feature extraction network, and the initial multi-scale feature map {I0, I1, I2, I3} is obtained by progressively using convolution, pooling and activation layers;

[0019] Step S1.3, based on the initial multi-scale feature map, with the help of the feature pyramid structure, the following operations are performed in sequence: ① k-1 After upsampling and I k Splice and aggregate to get f' k ; ② for f' k Use 1×1 convolution to further encode and get the new I at the current level k ; ③F k =I k .Thus, a multi-level feature map {F0, F1, F2, F3} is obtained.

[0020] Preferably, in step S1.1, the regularization factors for the original binocular image are: μ R，G，B =[0.485, 0.456, 0.406], σ R，G，B =[0.229, 0.224, 0.225], where R, G, B represent the three channels of the image, μ represents the mean, and σ represents the variance.

[0021] Preferably, in step S1.3, the multi-level feature maps {F0, F1, F2, F3} correspond to feature information at 1 / 4, 1 / 8, 1 / 16 and 1 / 32 resolutions respectively, and the size of their feature dimensions is the same as the maximum parallax value d that can be captured by the binocular camera. max The feature maps with 1 / 4 resolution are low-level feature maps, and the feature maps with the remaining resolution are feature maps of other levels.

[0022] Preferably, step S2 includes the following sub-steps:

[0023] Step S2.1, based on the original binocular image {L0, R0}, use the shared weight module to extract the texture information at the original resolution, and then design a layer-by-layer cascaded and cross-layer connected convolutional neural network to encode the texture information at the original resolution respectively, and extract the texture information at the low resolution {L T , R T}, {L T , R T} and low-level feature maps {F 0L , F 0R} Aggregation, use 3D convolution to calculate the texture volume V t, the calculation formula is as follows:

[0024] {L1,R1}=Norm(Conv{L0,R0})

[0025] T=L0◎R0

[0026]

[0027] L2=L1◎T,R2=R1◎T

[0028] L4=L1◎L2,R4=R1◎R2

[0029] L5=(L4T)☉(L4L1)

[0030] R5=(R4T)☉(R4R1)

[0031] L T =L1☉L2☉L3☉L5

[0032] R T =R1☉R2☉R3☉R5

[0033]

[0034] Among them, {L 1～5 , R 1～5} and T are the intermediate outputs of the network, Norm represents the normalization operation, Conv represents the two-dimensional convolution, Conv3d represents the three-dimensional convolution, ⊙ represents the Hadamard product of the matrix, Represents the concatenation operation of tensors, ◎ represents the composite operation of addition-convolution-regularization;

[0035] Step S2.2, based on the low-level feature map {F 0L , F 0R}, first use cosine similarity to roughly calculate the correlation tensor C between the two, and then use the deformable attention mechanism to calculate the correlation tensor C for each pixel position p in C. q The feature vector q of the image is adaptively searched and matched with other related pixels, and the feature information of each related pixel is weighted and aggregated to obtain a new feature vector q'. This operation is repeated for each position to obtain a new correlation tensor C', and the correlation volume V is calculated using a three-dimensional convolution. c , the calculation formula is as follows:

[0036] Δp 1,2，...,K =Linear(q),W=Conv(C)

[0037] A 1,2,...,K =SoftMax(Linear(q))

[0038]

[0039] gather all q'→C'

[0040] V c =Conv3d(C')

[0041] Among them, Linear represents the linear layer, W is the intermediate layer output obtained after the correlation tensor C is further learned by two-dimensional convolution, and Δp 1，2，..，K Represents the bias of adaptive learning, A 1，2，..，K Represents the aggregate weight after adaptive learning;

[0042] Step S2.3, based on the low-level feature map {F 0L , F 0R}, and use the SoftMax function to project it into the probability space to obtain the corresponding probability distribution tensor {P L , P R}, and then based on the definition of mutual information, the network structure is designed to calculate the similarity tensor M at the disparity probability distribution level, and further use three-dimensional convolution to calculate the mutual volume. The specific formula is as follows:

[0043]

[0044] V t =Conν3d(M)

[0045] Step S2.4, use Hadamard multiplication to aggregate the texture volume, correlation volume and mutual volume to calculate the three-fusion cost volume: V TFV =V t ⊙V c ⊙V m .

[0046] Preferably, in step S2.4, the resolutions of the texture volume, the correlation volume, the mutual volume and the three-fusion cost volume are all 1 / 4 of the resolution of the original binocular image.

[0047] Preferably, step S3 comprises: using cascaded multi-layer three-dimensional convolution to merge the three fused cost volumes V without changing the 1 / 4 resolution. TFV Gradually perform weighted aggregation with the feature maps {F1, F2, F3} of the remaining three levels of the multi-level feature map to obtain the aggregated cost volume V low .

[0048] Preferably, step S4 includes the following sub-steps:

[0049] Step S4.1, V lowThe disparity probability values ​​of each position in are sorted, and the disparity result predicted at the corresponding position is obtained by weighted summation of the two disparity values ​​(d1, d2) with the largest probability. The calculation formula is as follows:

[0050] p1,p2=SoftMax(V top-2 [x,y])

[0051] d low [x,y=p1d1+p2d2

[0052] Among them, V top-2 After sorting, only the cost volumes with the top two probabilities are taken, and p1 and p2 are the corresponding probability values;

[0053] Step S4.2, V low Use nearest neighbor interpolation to restore to the original resolution and get the complete cost volume V full , for V full All disparity values ​​at each position in The weighted sum is used to obtain the disparity result predicted at the corresponding position. The calculation formula is as follows:

[0054]

[0055] Preferably, step S5 includes the following sub-steps:

[0056] Step S5.1, based on the baseline distance b, focal length f of the binocular camera and the parallax value at a specific position at time t1 Calculate the depth value at a specific location Then generate the depth map at time t1;

[0057] Step S5.2: At time t2, the depth value at the same position is calculated using the same method. Then, based on the principle of similar triangles, if the height of the camera above the ground at time t1 is h1, the change in road surface undulation Δz can be obtained, and the calculation formula is as follows:

[0058]

[0059] The present invention also provides a vehicle suspension control system, based on the above-mentioned real-time binocular stereo matching method based on three-fusion cost volume, comprising:

[0060] Binocular vision module: used to execute the above-mentioned real-time binocular stereo matching method based on the three-fusion cost volume to obtain the change in road surface undulation; divide the range of the change in road surface undulation into multiple intervals, each interval corresponds to a preset suspension posture, and the control system adjusts the suspension posture in real time according to the actual situation.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. The present invention proposes a lightweight three-fusion volume construction strategy. The volume constructed by this strategy contains multi-scale texture information, pixel-level correlation information and probability distribution level similarity information. It can effectively integrate the information that is conducive to matching in the left and right views while maintaining the low computational complexity of the model, thereby improving the model's ability to model three-dimensional features.

[0063] 2. The present invention proposes a binocular stereo matching method based on the three-fusion cost volume. Based on this method, the disparity map and depth map can be accurately obtained, thereby providing accurate prior results for the input of road height during suspension control.

[0064] 3. The present invention proposes a vehicle suspension control system based on road depth perception, which can adaptively adjust the suspension posture based on changes in road undulation, and can significantly enhance the working stability and real-time performance of the vehicle suspension system. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0066] Figure 1 is a flow chart of a real-time binocular stereo matching method based on three-fusion cost volumes in an embodiment of the present invention;

[0067] Figure 2 A schematic diagram of the TCMNet model framework and its main components according to an embodiment of the present invention;

[0068] Figure 3 is a schematic diagram of a preset suspension posture in an embodiment of the present invention;

[0069] Figure 4 is a visualization result diagram of the matching result in the embodiment of the present invention;

[0070] Figure 5 It is a performance-computational complexity evaluation index diagram of the method of the present invention and other 6 methods in the embodiment of the present invention. DETAILED DESCRIPTION

[0071] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0072] The present invention provides a real-time binocular stereo matching method and a vehicle suspension control system based on the method, which aims to extract multi-scale texture information, use a deformable attention mechanism to adaptively extract pixel-level correlation, and further calculate the similarity of the probability distribution level, so as to construct an efficient and information-rich three-fusion cost volume, which can maintain good stereo matching performance while maintaining low computational complexity. On this basis, the hierarchical control of the vehicle suspension is further realized to ensure driving comfort.

[0073] Embodiment 1:

[0074] Figure 1 It is a flow chart of a real-time binocular stereo matching method based on three-fusion cost volumes in an embodiment of the present invention.

[0075] like Figure 1 As shown, this embodiment provides a real-time binocular stereo matching method based on three fusion cost volumes, including the following steps:

[0076] Step S1, by processing the original binocular image, a multi-level feature map in the form of three-dimensional tensors of different scales is obtained to obtain semantic information and texture information at different levels.

[0077] Specifically, step S1 includes the following sub-steps:

[0078] Step S1.1, regularize the original binocular image and expand it into a four-dimensional tensor to obtain a processed image.

[0079] In this embodiment, the regularization factors for the original binocular image are: μ R，G，B =[0.485, 0.456, 0.406], σ R，G，B =[0.229, 0.224, 0.225], where R, G, B represent the three channels of the image, μ represents the mean, and σ represents the variance.

[0080] Step S1.2, the processed image is passed through a lightweight feature extraction network, and the initial multi-scale feature map {I0, I1, I2, L3} is obtained by progressively using convolution, pooling and activation layers.

[0081] Step S1.3, based on the initial multi-scale feature map, with the help of the feature pyramid structure, the following operations are performed in sequence: ① k-1 After upsampling and I k Splice and polymerize to obtain F' k ② For F' k Use 1×1 convolution to further encode and get the new I at the current level k ; ③F k =Ik .Thus, a multi-level feature map {F0, F1, F2, F3} is obtained.

[0082] In this embodiment, the multi-level feature maps {F0, F1, F2, F3} correspond to the feature information at 1 / 4, 1 / 8, 1 / 16 and 1 / 32 resolutions respectively, and the size of their feature dimensions is the same as the maximum disparity value d that can be captured by the binocular camera. max The feature maps with 1 / 4 resolution are low-level feature maps, and the feature maps with the remaining resolution are feature maps of other levels.

[0083] Step S2, constructing a three-fusion cost volume based on the original binocular image and the low-level feature map of the multi-level feature map, which is used to construct the similarity and correlation between the left and right views in the depth direction.

[0084] Specifically, step S2 includes the following sub-steps:

[0085] Step S2.1, based on the original binocular image {L0, R0}, use the shared weight (convolution-regularization) module to extract the texture information at the original resolution, and then design a layer-by-layer cascaded and cross-layer connected convolutional neural network to encode the texture information at the original resolution respectively, and extract the texture information at the low resolution {L T , R T}.

[0086] {L T , R T} and low-level feature maps {F 0L , F 0R} Aggregation, use 3D convolution to calculate the texture volume V t , the calculation formula is as follows:

[0087] {L1, R1} = Norm(Conv{L0, R0})

[0088] T=L0◎R0

[0089]

[0090] L2=L1◎T,R2=R1◎T

[0091] L4=L1◎L2,R4=R1◎R2

[0092] L5=(L4T)☉(L4L1)

[0093] R5=(R4T)☉(R4R1)

[0094] L T =L1☉L2☉L3☉L5

[0095] RT =R1☉R2☉R3☉R5

[0096]

[0097] Among them, {L 1-5 ,R 1-5} and T are the intermediate outputs of the network, Norm represents the normalization operation, Conv represents the two-dimensional convolution, Conv3d represents the three-dimensional convolution, ⊙ represents the Hadamard product of the matrix, Represents the concatenation operation of tensors, and ◎ represents the composite operation of addition-convolution-regularization.

[0098] Step S2.2, based on the low-level feature map {F 0L , F 0R}, first use cosine similarity to roughly calculate the correlation tensor C between the two. Then use the deformable attention mechanism to calculate the correlation tensor C of each pixel at the position p in C. q The feature vector q of the image is adaptively searched and matched with other related pixels, and the feature information of each related pixel is weighted and aggregated to obtain a new feature vector q'. This operation is repeated for each position to obtain a new correlation tensor C', and the correlation volume V is calculated using a three-dimensional convolution. c , the calculation formula is as follows:

[0099] Δp 1,2，...，K =Linear(q),W=Conv(C)

[0100] A 1,2,...,K =SoftMax(Linear(q))

[0101]

[0102] gather oll q'→C'

[0103] V c =Conν3d(C')

[0104] Among them, Linear represents the linear layer, W is the intermediate layer output obtained after the correlation tensor C is further learned by two-dimensional convolution, and Δp 1，2，..，K Represents the bias of adaptive learning, A 1，2，..，K Represents the aggregate weight after adaptive learning.

[0105] Step S2.3, based on the low-level feature map {F 0L , F 0R}, and use the SoftMax function to project it into the probability space to obtain the corresponding probability distribution tensor {P L , P R}, and then based on the definition of mutual information, the network structure is designed to calculate the similarity tensor M at the disparity probability distribution level, and further use three-dimensional convolution to calculate the mutual volume. The specific formula is as follows:

[0106]

[0107] V t =Conv3d(M)

[0108] Step S2.4, use Hadamard multiplication to aggregate the texture volume, correlation volume and mutual volume to calculate the three-fusion cost volume: V TFV =V t ⊙V c ⊙V m .

[0109] In this embodiment, the resolutions of the texture volume, the correlation volume, the mutual volume and the three-fusion cost volume are all 1 / 4 of the resolution of the original binocular image.

[0110] Step S3, based on the three-fusion cost volume and other level feature maps of the multi-level feature map, downsampling-upsampling is performed based on 3D convolution layer by layer in cascade according to the tensor size to achieve aggregation of the cost volume and obtain the aggregated cost volume.

[0111] Specifically, step S3 includes: using cascaded multi-layer 3D convolution, without changing the 1 / 4 resolution, to convert the three fused cost volumes V TFV Gradually perform weighted aggregation with the feature maps {F1, F2, F3} of the remaining three levels of the multi-level feature map to obtain the aggregated cost volume V low .

[0112] In step S4, the aggregated cost volume is input into a dual-scale disparity regression module, and the arg-softmax function is used to output disparity maps at 1 / 4 resolution and original resolution, respectively.

[0113] Specifically, step S4 includes the following sub-steps:

[0114] Step S4.1, V low The disparity probability values ​​of each position in are sorted, and the disparity result predicted at the corresponding position is obtained by weighted summation of the two disparity values ​​(d1, d2) with the largest probability. The calculation formula is as follows:

[0115] p1,p2=SoftMax(V top-2 [x, y])

[0116] d low [x,y]=p1d1+p2d2

[0117] Among them, V top-2 After sorting, only the cost volumes with the top two probabilities are taken, and p1 and p2 are the corresponding probability values;

[0118] Step S4.2, V low Use nearest neighbor interpolation to restore to the original resolution and get the complete cost volume V full . For V full All disparity values ​​at each position in The weighted sum is used to obtain the disparity result predicted at the corresponding position. The calculation formula is as follows:

[0119]

[0120] Step S5, using the binocular camera parameters and the disparity map, calculate the depth map of the road surface, and by comparing the depth changes in the depth map under consecutive frames, calculate the change in road surface undulation according to the geometric relationship, which is used as the input signal for subsequent suspension control.

[0121] Specifically, step S5 includes the following sub-steps:

[0122] Step S5.1, based on the baseline distance b, focal length f of the binocular camera and the parallax value at a specific position at time t1 Calculate the depth value at a specific location Then, a depth map at time t1 is generated.

[0123] Step S5.2: At time t2, the depth value at the same position is calculated using the same method. Then, based on the principle of similar triangles, if the height of the camera above the ground at time t1 is h1, the change in road surface undulation Δz can be obtained, and the calculation formula is as follows:

[0124]

[0125] Embodiment 2:

[0126] This embodiment provides a vehicle suspension control system, which adopts a real-time binocular stereo matching method based on three-fusion cost volume in the above embodiment 1, including:

[0127] Binocular vision module: a real-time binocular stereo matching method based on three-fusion cost volume used in the above-mentioned embodiment 1 to obtain the change in road surface undulation; the range of the change in road surface undulation is divided into multiple intervals, each interval corresponds to a preset suspension posture, and the control system adjusts the suspension posture in real time according to the actual situation to ensure driving comfort.

[0128] Figure 3 It is a schematic diagram of the preset suspension posture in an embodiment of the present invention.

[0129] like Figure 3 As shown, during the actual vehicle driving process, the change in road undulation is monitored in real time. When the change is in a certain range, the suspension posture is immediately adjusted to the state preset in the range, so that the suspension can respond in time according to the dynamic changes in road conditions to ensure the comfort and stability of vehicle driving.

[0130] The present invention realizes a binocular stereo matching method with low matching error. The method shows excellent matching effect in the test, and the predicted results are highly consistent with the actual results. Figure 4 This method not only maintains low endpoint error (EPE), first-order error (D1) and error beyond n pixels (>npx), but also has extremely low computational complexity. Test results and performance parameter comparisons are shown in Figure 5 In view of the key role of binocular stereo matching in environmental perception tasks and subsequent suspension active control safety, the present invention has important practical value, especially in improving the safety and efficiency of intelligent driving systems and vehicle driving comfort.

[0131] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0132] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A real-time binocular stereo matching method based on three-fusion cost volume, characterized in that: The following steps are involved: S1: By processing the original binocular image, a multi-level feature map in the form of a three-dimensional tensor of different scales is obtained to obtain semantic information and texture information at different levels; S2: constructing a three-fusion cost volume based on the original binocular image and the low-level feature map of the multi-level feature map, which is used to construct the similarity and correlation of the left and right views in the depth direction; S3: based on the three-fusion cost volume and other level feature maps of the multi-level feature map, downsampling-upsampling is performed layer by layer based on 3D convolution in cascade according to the tensor size to achieve aggregation of the cost volume and obtain an aggregated cost volume; S4: inputting the aggregated cost volume into a dual-scale disparity regression module, using an arg-softmax function, and outputting disparity maps at 1 / 4 resolution and original resolution respectively; S5: Calculate the depth map of the road surface using the binocular camera parameters and the disparity map, and calculate the change in road surface undulation based on the geometric relationship by comparing the depth changes in the depth map under consecutive frames, which serves as the input signal for subsequent suspension control.

2. A real-time binocular stereo matching method based on three-fusion cost volumes according to claim 1, characterized in that: The step S1 includes the following sub-steps: Step S1.1, regularizing the original binocular image and expanding it into a four-dimensional tensor to obtain a processed image; Step S1.2, passing the processed image through a lightweight feature extraction network, and obtaining an initial multi-scale feature map {I0, I1, I2, I3} by progressively using convolution, pooling and activation layers; Step S1.3, based on the initial multi-scale feature map, with the help of the structure of the feature pyramid, the following operations are performed in sequence: ① k-1 After upsampling and I k Splice and polymerize to obtain F' k ② for F' k Use 1×1 convolution to further encode and get the new I at the current level k ; ③F k =I k .Thus, a multi-level feature map {F0, F1, F2, F3} is obtained.

3. A real-time binocular stereo matching method based on three-fusion cost volumes according to claim 2, characterized in that: In step S1.1, the regularization factors for the original binocular image are: μ R，G，B =[0.485, 0.456, 0.406], σ R，G，B =[0.229, 0.224, 0.225], where R, G, B represent the three channels of the image, μ represents the mean, and σ represents the variance.

4. The real-time binocular stereo matching method based on three-fusion cost volumes according to claim 2, characterized in that: In step S1.3, the multi-level feature maps {F0, F1, F2, F3} correspond to feature information at 1 / 4, 1 / 8, 1 / 16 and 1 / 32 resolutions respectively, and The size of its feature dimension is related to the maximum parallax value d that the binocular camera can capture. max The feature maps with 1 / 4 resolution are low-level feature maps, and the feature maps with the remaining resolution are feature maps of other levels.

5. The real-time binocular stereo matching method based on three-fusion cost volumes according to claim 4, characterized in that: The step S2 includes the following sub-steps: Step S2.1, based on the original binocular image {L0, R0}, use the shared weight module to extract the texture information at the original resolution, and then design a layer-by-layer cascaded and cross-layer connected convolutional neural network to encode the texture information at the original resolution respectively, and extract the texture information at the low resolution {L T , R T }, {L T , R T } and the low-level feature map {F 0L , F 0R } Aggregation, use 3D convolution to calculate the texture volume V t , the calculation formula is as follows: {L1,R1}=Norm(Conv{L0,R0}) T=L0◎R L2=L1◎T,R2=R1◎T L4=L1◎L2,R4=R1◎R2 L5=(L4T)☉(L4L1) R5=(R4T)☉(R4R1) L T =L1☉L2☉L3☉L5 R T =R1☉R2☉R3☉R5 Among them, {L 1～5 , R 1～5 } and T are the intermediate outputs of the network, Norm represents the normalization operation, Conv represents the two-dimensional convolution, Conv3d represents the three-dimensional convolution, ⊙ represents the Hadamard product of the matrix, Represents the concatenation operation of tensors, ◎ represents the composite operation of addition-convolution-regularization; Step S2.2, based on the low-level feature map {F 0L , F 0R }, first use cosine similarity to roughly calculate the correlation tensor C between the two, and then use the deformable attention mechanism to calculate the correlation tensor C for each pixel position p in C. q The feature vector q of the image is adaptively searched and matched with other related pixels, and the feature information of each related pixel is weighted and aggregated to obtain a new feature vector q'. This operation is repeated for each position to obtain a new correlation tensor C', and the correlation volume V is calculated using a three-dimensional convolution. c , the calculation formula is as follows: Δp 1,2,...,K =Linear(q),W=Conv(C) A 1,2,...,K =SoftMax(Linear(q)) gather allq'→C' V c =Conv3d(C') Among them, Linear represents the linear layer, w is the intermediate layer output obtained after the correlation tensor C is further learned by two-dimensional convolution, and Δp 1，2，..，K Represents the bias of adaptive learning, A 1，2，..，K Represents the aggregate weight after adaptive learning; Step S2.3, based on the low-level feature map {F 0L , F 0R }, and use the SoftMax function to project it into the probability space to obtain the corresponding probability distribution tensor {P L , P R }, and then based on the definition of mutual information, the network structure is designed to calculate the similarity tensor M at the disparity probability distribution level, and further use three-dimensional convolution to calculate the mutual volume. The specific formula is as follows: V t =Conv3d(M) Step S2.4, using Hadamard multiplication to aggregate the texture volume, the correlation volume and the mutual volume, and calculate the three-fusion cost volume: V TFV =V t ⊙V c ⊙V m .

6. The real-time binocular stereo matching method based on three-fusion cost volumes according to claim 5, characterized in that: In step S2.4, the resolutions of the texture volume, the correlation volume, the mutual volume and the three-fusion cost volume are all 1 / 4 of the resolution of the original binocular image.

7. The real-time binocular stereo matching method based on three-fusion cost volumes according to claim 1, characterized in that: The step S3 comprises: using cascaded multi-layer three-dimensional convolutions to transform the three fused cost volumes V without changing the 1 / 4 resolution. TFV Gradually weighted aggregate with the feature maps {F1, F2, F3} of the remaining three levels of the multi-level feature map to obtain the aggregated cost volume V low .

8. The real-time binocular stereo matching method based on three-fusion cost volumes according to claim 1, characterized in that: The step S4 includes the following sub-steps: Step S4.1, V low The disparity probability values ​​of each position in are sorted, and the disparity result predicted at the corresponding position is obtained by weighted summation of the two disparity values ​​(d1, d2) with the largest probability. The calculation formula is as follows: p1,p2=SoftMax(V top-2 [x,y]) d tow [x,y]=p1d1+p2d2 Among them, V top-2 After sorting, only the cost volumes with the top two probabilities are taken, and p1 and p2 are the corresponding probability values; Step S4.2, V low Use nearest neighbor interpolation to restore to the original resolution and get the complete cost volume V full , for V full All disparity values ​​at each position in The weighted sum is used to obtain the disparity result predicted at the corresponding position. The calculation formula is as follows:

9. The real-time binocular stereo matching method based on three-fusion cost volumes according to claim 1, characterized in that: The step S5 comprises the following sub-steps: Step S5.1, based on the baseline distance b, focal length f of the binocular camera and the parallax value at a specific position at time t1 Calculate the depth value of the specific position Then generate the depth map at time t1; Step S5.2: At time t2, the depth value at the same position is calculated using the same method. Then, based on the principle of similar triangles, if the height of the camera above the ground at time t1 is h1, the change in road surface undulation Δz can be obtained, and the calculation formula is as follows:

10. A vehicle suspension control system, based on a real-time binocular stereo matching method based on a three-fusion cost volume as claimed in any one of claims 1 to 9, characterized in that: include: Binocular vision module: used to execute a real-time binocular stereo matching method based on a three-fusion cost volume as described in any one of claims 1 to 9 to obtain a change in road surface undulation; divide the range of the change in road surface undulation into multiple intervals, each interval corresponding to a preset suspension posture, and the control system adjusts the suspension posture in real time according to actual conditions.

Images