A vehicle scene depth estimation method and system based on multi-scale attention and light three-dimensional convolution
By employing multi-scale attention and lightweight 3D convolution, the problems of high computational cost and low detection accuracy in vehicle depth estimation are solved, achieving efficient and real-time vehicle front depth recognition and improving driving safety in complex environments.
Patent Information
- Application Number
- CN202610770130.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-01
- Publication Date
- 2026-07-28
- Estimated Expiration
- 2046-06-01
AI Technical Summary
Existing deep learning-based vehicle depth estimation methods have high computational overhead, slow inference speed, and difficulty in meeting real-time requirements. Furthermore, they fail to extract sufficient features in complex environments, resulting in low detection accuracy.
A vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution is adopted. Multi-scale features are obtained through a lightweight feature extraction network, and information interaction and fusion are performed using a multi-scale spatial attention network to construct a 4D cost volume. The cost volume is then regularized and aggregated through a lightweight 3D convolutional network. Finally, multi-round residual learning and iterative updates are performed by combining convolutional gated recurrent units to output scene disparity estimation results.
While ensuring computational efficiency and real-time performance, it achieves high-precision recognition of the depth of field in front of the vehicle, improves driving safety in complex environments, and balances the algorithm's prediction speed and accuracy.
Smart Images
Figure CN122312732B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of stereo matching, and in particular to a method and system for estimating vehicle scene depth based on multi-scale attention and lightweight 3D convolution. Background Technology
[0002] During vehicle operation, accurate 3D perception of the environment ahead is crucial for obstacle avoidance. Existing in-vehicle depth estimation methods often employ deep learning-based stereo matching algorithms. The core process typically includes feature extraction, cost volume construction, and cost aggregation using a 3D convolutional neural network. The advantage of this approach lies in its ability to leverage the powerful feature representation capabilities of convolutional neural networks to generate denser depth maps, resulting in better matching performance compared to traditional hand-crafted feature algorithms in typical scenarios.
[0003] However, in practical applications, traditional deep learning-based depth detection methods have their own shortcomings: First, the computational cost is too high. Standard 3D convolution operations are complex and consume a lot of memory, resulting in slow inference speed, which makes it difficult to meet the stringent real-time requirements of vehicles traveling at high speeds. Second, they lack effective perception of complex regions. When faced with weak textures, reflections, or fine structures, they are prone to insufficient feature extraction, which leads to low detection accuracy and makes it difficult to achieve a balance between accuracy and speed with limited computing resources. Summary of the Invention
[0004] The purpose of this application is to provide a vehicle scene depth estimation method and system based on multi-scale attention and lightweight 3D convolution, which can achieve high-precision recognition of the depth of field in front of the vehicle while ensuring computational efficiency and real-time performance, and can balance the prediction speed and accuracy of the algorithm, thereby improving the safety of driving in complex environments.
[0005] To achieve the above objectives, this application provides the following solution:
[0006] In a first aspect, this application provides a vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution. This method is applied to a processing device for a target vehicle. The method includes: acquiring a binocular view of the scene in front of the target vehicle; the binocular view includes a left view and a right view; constructing a lightweight feature extraction network based on a lightweight bottleneck structure, and using the lightweight feature extraction network to extract features from the binocular view according to different spatial resolutions to obtain multi-scale features; the multi-scale features include 1 / 4 scale features, 1 / 8 scale features, 1 / 16 scale features, and 1 / 32 scale features; and inputting the 1 / 32 scale features into the multi-scale space. An attention network is used to perform cross-spatial information interaction and fusion to obtain multi-scale contextual features. A group correlation method is used to construct a 4D cost volume within different disparity ranges corresponding to the 1 / 4 scale features. The 4D cost volume is used to capture geometric matching information under different disparities. A lightweight 3D convolutional network is constructed based on 3D depth separable convolution, and the 4D cost volume is regularly aggregated and optimized using the lightweight 3D convolutional network according to the multi-scale features to obtain an aggregated 4D cost volume. Disparity regression calculation is performed on the aggregated 4D cost volume to obtain an initial disparity map. Based on the multi-scale contextual features and the initial disparity map, a hierarchical residual refinement module based on convolutional gated recurrent units is used to perform multiple rounds of residual learning and iterative updates to obtain the scene disparity estimation results.
[0007] Secondly, this application also provides a computer system, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution as described in the first aspect.
[0008] According to the specific embodiments provided in this application, the following technical effects are disclosed:
[0009] This application acquires binocular views; extracts multi-scale features using a lightweight feature extraction network; extracts multi-scale contextual features using a multi-scale spatial attention network; constructs a 4D cost volume on the 1 / 4 scale features using a group correlation method, and regularizes it using a lightweight 3D convolutional network; performs disparity regression on the aggregated 4D cost volume to obtain an initial disparity map. Based on the multi-scale contextual features and the initial disparity map, a hierarchical residual refinement module based on convolutional gated recurrent units is used for multi-round residual learning and iterative updates to output scene disparity estimation results. This application solves the problem of low matching accuracy in weak texture and occluded areas when estimating the depth of field in front of a vehicle in traditional technologies. While ensuring computational efficiency and real-time performance, it achieves high-precision recognition of the depth of field in front of a vehicle, balancing the algorithm's prediction speed and accuracy, and improving the safety of vehicles driving in complex environments. Attached Figure Description
[0010] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0011] Figure 1 A flowchart illustrating the vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution provided in this application embodiment. Figure 1 .
[0012] Figure 2 A flowchart illustrating the vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution provided in this application embodiment. Figure 2 .
[0013] Figure 3 This is a schematic diagram of the structure of a multi-scale spatial attention network provided in an embodiment of this application.
[0014] Figure 4 This is a schematic diagram of the structure of a three-dimensional depth-separable convolution module provided in an embodiment of this application.
[0015] Figure 5 This is an internal structure diagram of a computer system provided in an embodiment of this application. Detailed Implementation
[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0017] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0018] Example 1, as Figures 1-2 As shown, this embodiment provides a vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution. The vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution is applied to the processing device of the target vehicle (the processing device includes at least a local processor or a cloud processor for vehicle driving). The method includes the following steps.
[0019] S1. Obtain a binocular view of the scene in front of the target vehicle; the binocular view includes a left view and a right view.
[0020] Optionally, the binocular view is acquired via a binocular camera.
[0021] In practical applications, it mainly refers to scene depth. By obtaining scene depth information of the scene in front of the target vehicle, the relative distance parameters between each target object in the scene ahead and the vehicle can be determined, which helps the vehicle driver or autonomous driving system to achieve real-time identification and early warning avoidance of potential environmental hazards.
[0022] S2. A lightweight feature extraction network is constructed based on a lightweight bottleneck structure, and the lightweight feature extraction network is used to extract features from the binocular view according to different spatial resolutions to obtain multi-scale features; the multi-scale features include: 1 / 4 scale features, 1 / 8 scale features, 1 / 16 scale features and 1 / 32 scale features.
[0023] In practical applications, a U-Net-style encoder-decoder structure is used to extract local features from image pairs. The encoder structure can use a pre-trained MobileNet V2 as a lightweight backbone to obtain features at multiple scales. In the decoder stage, the spatial resolution of the feature maps is gradually restored through three upsampling modules, and the feature maps of the corresponding layers in the encoding stage are fused with the feature maps of the current layer through skip connections. Finally, feature maps with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original resolution are obtained. Then, the 1 / 4 resolution feature map is used to construct the 4D cost volume, and the other 1 / 8, 1 / 16, and 1 / 32 scale features can be used as contextual guidance for subsequent cost aggregation.
[0024] S3. Input the 1 / 32 scale features into a multi-scale spatial attention network to perform cross-spatial information interaction and fusion to obtain multi-scale contextual features.
[0025] Furthermore, step S3 specifically includes the following steps.
[0026] S31. Input the 1 / 32 scale features into the multi-scale spatial attention network.
[0027] S32. Capture global contextual information through multi-scale pooling and spatial weight mapping.
[0028] S33. Fuse global contextual information with local features to obtain multi-scale contextual features.
[0029] Furthermore, the multi-scale spatial attention network is a network structure with group processing, parallel branches, and cross-space learning; the lightweight feature extraction network is used to recalibrate the channel weights in each parallel branch by encoding global information, and is also used to further aggregate the output features of two parallel branches through cross-dimensional interaction to capture pixel-level pairwise relationships (improving the performance of feature extraction).
[0030] In practical applications, such as Figure 3 As shown, the input features are divided into several subgroups along the channel dimension. Each group is processed using two parallel branches. One branch performs average pooling on the features at both the height and width, respectively, to obtain global information in the vertical and horizontal directions. After dimensionality transformation, the resulting subgroups are concatenated vertically and then separated again after a 1×1 convolution. Activation functions and dimensionality transformations are then used to obtain attention weights in the horizontal and vertical directions. Each group of features is multiplied by these two weights for weighted summation, enhancing spatial information interaction. After grouping and normalization, the final feature set is obtained. The formula is as follows.
[0031] .
[0032] In the formula, For grouping normalization, For grouping features, Attention weights representing the vertical direction, with dimension 1. , Represents the horizontal weight, with dimension 1. , Represents element-wise multiplication.
[0033] The other branch extracts local spatial features through 3×3 convolution. In the cross-spatial learning stage, the features of the two branches are processed by global average pooling and softmax functions respectively, and then cross-multiplied with the original features of the other branch. The results are summed and the dimensions are reconstructed to generate the attention map M. The calculation process is as follows.
[0034] .
[0035] .
[0036] In the formula, The matrix cross product is used; the sigmoid function is used for normalization to obtain attention weights, the original features are multiplied by the weights to enhance important regions, and the final output features are obtained after dimensionality reconstruction. .
[0037] .
[0038] S4. A 4D cost volume is constructed in different disparity ranges corresponding to 1 / 4 scale features using a group correlation method; the 4D cost volume is used to capture geometric matching information under different disparities.
[0039] Furthermore, step S4 specifically includes the following steps.
[0040] S41. Divide the 1 / 4 scale features into multiple groups along the channel dimension within different parallax ranges.
[0041] S42. Calculate the correlation graph for each group to obtain multiple sub-cost bodies.
[0042] S43. Using multiple sub-cost bodies, a grouping correlation method is employed for matching to obtain a 4D cost body.
[0043] Furthermore, the calculation formula for the group correlation method is as follows.
[0044] .
[0045] In the formula, For the first Group feature map at location ( Offset Feature vector of 1 pixel; The number of feature channels; Number of groups; It is the dot product of vectors; This is the left feature map of the g-th group; The right feature map of group g.
[0046] Optionally, the group correlation cost body can be regarded as A set of cost bodies, each cost body being calculated from the corresponding feature set.
[0047] In practical applications, the grouped correlation method involves grouping features, with each group only having its correlation calculated with another group of features, thus reducing computational complexity. By constructing grouped correlation cost bodies at different disparity ranges using the input left and right feature maps, the features are divided into several groups along the channel dimension, and a correlation map is calculated for each group.
[0048] S5. For example Figure 4 As shown, a lightweight 3D convolutional network is constructed based on 3D depth separable convolution, and the 4D cost volume is regularized and aggregated using the lightweight 3D convolutional network according to multi-scale features to obtain the aggregated 4D cost volume.
[0049] Furthermore, step S5 specifically includes the following steps.
[0050] S51. Construct a lightweight 3D convolutional network based on 3D depth separable convolution.
[0051] S52. Project the 4D cost volume onto a higher dimension using a 1×1×1 point convolution, and then perform 3D grouping normalization and ReLU activation on the input after convolution.
[0052] S53. Decompose the depth-direction convolution into disparity-direction convolution and spatial-direction convolution; wherein, the disparity-direction convolution uses a K×1×1 convolution kernel; the spatial-direction convolution uses a 1×K×K convolution kernel; the disparity-direction convolution is used to capture the correlation in the depth dimension; the spatial-direction convolution is used to aggregate contextual information in the planar domain.
[0053] Optionally, both parallax-oriented convolution and spatial-oriented convolution can be performed in the depth direction, meaning the number of groups equals the number of channels. Traditional 3D convolution uses... The convolution kernel, and the decomposition used in this embodiment reduces the number of channel parameters from Reduce to .
[0054] S54. Based on the channel attention mechanism, global average pooling is used to compress the global spatial difference information corresponding to multi-scale features into channel descriptors.
[0055] Optionally, step S54 is primarily used to compensate for the potential information loss inherent in lightweight convolutions.
[0056] S55. Use two 1×1×1 convolutions and an activation function to generate channel weights.
[0057] S56. The feature map is recalibrated by multiplying the output of the depthwise convolution with the channel weights.
[0058] S57. Project the features back to the output channel dimension using a 1×1×1 point convolution to obtain the aggregated 4D cost volume.
[0059] Optionally, the aggregation of the 4D cost volume further includes processing the output of the multi-scale spatial attention network using a lightweight 3D convolutional network. The lightweight 3D convolutional network employs 3D depthwise separable convolutions instead of traditional 3D convolutions to reduce the number of parameters and accelerate inference.
[0060] In practical applications, ordinary 3D convolutions generate significant computational complexity and memory consumption when processing 4D cost volumes that include height, width, parallax, and channel dimensions. Therefore, this embodiment uses a network structure based on 3D depth-separable convolutions to regularize and further optimize the cost volume. Specifically, this is achieved by decoupling the traditional standard 3D convolution into depth convolution and spatial convolution, thereby reducing the computational burden.
[0061] Furthermore, the lightweight 3D convolutional network structure consists of three downsampling modules and three upsampling modules. Each downsampling module consists of two 3×3×3 3D convolutions with 16, 32 and 48 channels respectively. Each upsampling module consists of a 4×4×4 3D deconvolution and two 3×3×3 3D convolutions.
[0062] In practical applications, the weighted cost volume channels calculated using left-view features are used for cost aggregation. For a Cost volume ( =4, 8, 16, 32) In cost aggregation, the guiding cost volume incentive is expressed as follows.
[0063] .
[0064] in, To guide the incentive of cost volume, For the sigmoid function, For Hadama accumulation.
[0065] The 3D regularized network with inserted guiding cost volume excitation operations can effectively infer and propagate geometric information at multiple scales, thereby generating a multi-range geometrical encoding volume (MGEV). The MGEV encodes multi-range and multi-granular geometric information, providing more comprehensive and concise information for subsequent disparity optimization by ConvGRU, which in turn helps to eliminate matching noise and resolve matching ambiguities in ill-conditioned regions.
[0066] S6. Perform disparity regression calculation on the aggregated 4D cost volume to obtain the initial disparity map.
[0067] Furthermore, step S6 specifically includes the following steps.
[0068] S61. The Soft-argmin function is used to perform a weighted summation on the aggregated 4D cost volume along the disparity dimension to calculate the expected disparity value of each pixel, thus obtaining the initial disparity map; the calculation formula for the initial disparity map is as follows.
[0069] .
[0070] In the formula, The initial disparity map obtained; For parallax indexing; A set of disparity indexes; It is a normalized exponential function; This is the cost body after aggregation.
[0071] S7. Based on the multi-scale contextual features and the initial disparity map, a hierarchical residual refinement module based on convolutional gated recurrent units is used to perform multiple rounds of residual learning and iterative updates to obtain the scene disparity estimation results.
[0072] Furthermore, step S7 specifically includes the following steps.
[0073] S71. Using the initial disparity map as the starting point for the convolution-gated recurrent unit, in each iteration, a set of geometric features is indexed from the 4D cost volume using the previous disparity map.
[0074] S72. Selective geometric feature fusion selectively fuses geometric features across different parallax ranges.
[0075] S73. Using the initial disparity map and the left-side feature map as input, a set of selective weights is predicted through two convolutional layers; wherein, the prediction formula for the selective weights is as follows.
[0076] .
[0077] In the formula, For the selective weights of the prediction; , , These are disparity maps for different disparity ranges. The feature map on the left is shown below. This is the sigmoid function.
[0078] S74. Adaptively fuse geometric features based on selective weights to obtain fused features; the formula for the fused features is as follows.
[0079] .
[0080] In the formula, Features after fusion; These are geometric features with different parallax ranges; For Hadama accumulation.
[0081] S75. In each iteration, the hidden state is updated using multi-scale contextual features through convolutional gated recurrent units, with the goal of minimizing the loss function of regularization loss and iteration loss. Finally, a residual is decoded to optimize the disparity map, and the optimized disparity map is output as the scene disparity estimation result; the formula for the optimized disparity map is as follows.
[0082] .
[0083] .
[0084] In the formula, The optimized disparity map; This is the initial disparity map; The depth residual map for decoding; This is an update operator based on convolution-gated recurrent units; This is the hidden state from the previous moment; These are the features after fusion.
[0085] Optionally, the aggregation network and the multi-scale spatial attention network (efficient multi-scale spatial attention network) in this embodiment can both be trained end-to-end. The overall training objective includes two components: regularization loss and iterative loss. The regularization loss is used to supervise the initial disparity map, and the iterative loss is calculated on a series of iteratively updated disparity maps. The formula is as follows.
[0086]
[0087] In the formula, For regularization loss, For iteration loss, For balancing weights across different parallax ranges, For true parallax, For the disparity map that is updated iteratively, This is the attenuation factor.
[0088] The technical effects of this application are as follows: This application acquires a binocular view; extracts multi-scale features using a lightweight feature extraction network; extracts multi-scale contextual features using a multi-scale spatial attention network; constructs a 4D cost volume on the 1 / 4 scale features using a group correlation method, and regularizes it using a lightweight 3D convolutional network; performs disparity regression on the aggregated 4D cost volume to obtain an initial disparity map. Based on the multi-scale contextual features and the initial disparity map, a hierarchical residual refinement module based on convolutional gated recurrent units is used for multi-round residual learning and iterative updates to output scene disparity estimation results. This application solves the problem of low matching accuracy in weak texture and occluded areas when estimating the depth of field in front of a vehicle in traditional technologies. While ensuring computational efficiency and real-time performance, it achieves high-precision recognition of the depth of field in front of the vehicle, balancing the prediction speed and accuracy of the algorithm, and improving the safety of driving in complex environments.
[0089] Example 2: This example provides a computer system, which can be a server or a terminal, and its internal structure diagram can be as follows. Figure 5As shown, the computer system includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements the methods described above.
[0090] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer system to which the present application is applied. A specific computer system may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0091] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).
[0092] The processors involved in the various embodiments provided in this application may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited thereto.
[0093] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0094] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution, wherein the vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution is applied to a processing device for a target vehicle, characterized in that, The method includes: Obtain a binocular view of the scene in front of the target vehicle; the binocular view includes a left view and a right view; A lightweight feature extraction network is constructed based on a lightweight bottleneck structure, and the lightweight feature extraction network is used to extract features from the binocular view according to different spatial resolutions to obtain multi-scale features; the multi-scale features include: 1 / 4 scale features, 1 / 8 scale features, 1 / 16 scale features and 1 / 32 scale features. The 1 / 32 scale features are input into a multi-scale spatial attention network to perform cross-spatial information interaction and fusion, resulting in multi-scale contextual features. A 4D cost volume is constructed within different disparity ranges corresponding to 1 / 4 scale features using a group correlation method; the 4D cost volume is used to capture geometric matching information under different disparities. A lightweight 3D convolutional network is constructed based on 3D depth separable convolution, and the 4D cost volume is regularized and aggregated using the lightweight 3D convolutional network according to multi-scale features to obtain the aggregated 4D cost volume. Disparity regression calculations are performed on the aggregated 4D cost volume to obtain the initial disparity map; Based on multi-scale contextual features and the initial disparity map, a hierarchical residual refinement module based on convolutional gated recurrent units is used to perform multiple rounds of residual learning and iterative updates to obtain scene disparity estimation results.
2. The vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution as described in claim 1, characterized in that, The 1 / 32 scale features are input into a multi-scale spatial attention network to obtain multi-scale contextual features, specifically including: Input 1 / 32 scale features into a multi-scale spatial attention network; Global contextual information is captured through multi-scale pooling and spatial weight mapping; By fusing global contextual information with local features, multi-scale contextual features are obtained.
3. The vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution as described in claim 1, characterized in that, The multi-scale spatial attention network is a network structure with group processing, parallel branches, and cross-space learning; the lightweight feature extraction network is used to recalibrate the channel weights in each parallel branch by encoding global information, and is also used to further aggregate the output features of two parallel branches through cross-dimensional interaction to capture pixel-level pairwise relationships.
4. The vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution as described in claim 1, characterized in that, A 4D cost volume is constructed using a group correlation method across different disparity ranges corresponding to 1 / 4 scale features, specifically including: The 1 / 4 scale features are divided into multiple groups along the channel dimension within different parallax ranges; For each group, the correlation graph is calculated to obtain multiple sub-cost bodies; The 4D cost body is obtained by matching multiple sub-cost bodies using a grouping correlation method.
5. The vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution according to claim 1, characterized in that, The calculation formula for the group correlation method is as follows: ； In the formula, For the first Group feature map at location ( Offset Feature vector of 1 pixel; The number of feature channels; Number of groups; It is the dot product of vectors; This is the left feature map of the g-th group; The right feature map of group g.
6. The vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution according to claim 1, characterized in that, A lightweight 3D convolutional network is constructed based on 3D depthwise separable convolution. Then, based on multi-scale features, the lightweight 3D convolutional network is used to perform regularization and aggregation optimization on the 4D cost volume, resulting in the aggregated 4D cost volume, specifically including: A lightweight 3D convolutional network is constructed based on 3D depthwise separable convolution. The 4D cost volume is projected onto a higher dimension through a 1×1×1 point convolution, and the input after convolution is subjected to 3D grouping normalization and ReLU activation function. The depth-oriented convolution is decomposed into disparity-oriented convolution and spatial-oriented convolution; wherein, the disparity-oriented convolution uses a K×1×1 convolution kernel; the spatial-oriented convolution uses a 1×K×K convolution kernel; the disparity-oriented convolution is used to capture the correlation in the depth dimension; the spatial-oriented convolution is used to aggregate contextual information in the planar domain. Based on the channel attention mechanism, global average pooling is used to compress the global spatial difference information corresponding to multi-scale features into channel descriptors. Channel weights are generated by using two 1×1×1 convolutions followed by an activation function. The feature map is recalibrated by multiplying the output of the depthwise convolution with the channel weights; The features are projected back to the output channel dimension using a 1×1×1 point convolution to obtain the aggregated 4D cost volume.
7. The vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution according to claim 1, characterized in that, The lightweight 3D convolutional network structure consists of three downsampling modules and three upsampling modules. Each downsampling module consists of two 3×3×3 3D convolutions with 16, 32 and 48 channels respectively. Each upsampling module consists of a 4×4×4 3D deconvolution and two 3×3×3 3D convolutions.
8. The vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution according to claim 1, characterized in that, Disparity regression calculations are performed on the aggregated 4D cost volume to obtain an initial disparity map, specifically including: The expected disparity value of each pixel is calculated by weighted summation of the aggregated 4D cost volume along the disparity dimension using the Soft-argmin function, thus obtaining the initial disparity map. The calculation formula for the initial disparity map is as follows: In the formula, The initial disparity map obtained; For parallax indexing; A set of disparity indexes; It is a normalized exponential function; This is the cost body after aggregation.
9. The vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution according to claim 1, characterized in that, Based on multi-scale contextual features and the initial disparity map, a hierarchical residual refinement module based on convolutional gated recurrent units is used for multiple rounds of residual learning and iterative updates to obtain scene disparity estimation results, specifically including: The initial disparity map is used as the starting point for the convolution-gated recurrent unit. In each iteration, a set of geometric features is indexed from the 4D cost volume using the previous disparity map. Selective geometric feature fusion is used to selectively fuse geometric features across different parallax ranges. Using the initial disparity map and the left-side feature map as input, a set of selective weights is predicted through two convolutional layers; wherein the prediction formula for the selective weights is as follows: ； In the formula, For the selective weights of the prediction; , , These are disparity maps for different disparity ranges. The feature map on the left is shown below. It is the sigmoid function; Adaptive fusion of geometric features is performed based on selective weights to obtain the fused features; the formula for the fused features is as follows: In the formula, Features after fusion; These are geometric features with different parallax ranges; For Hadamah accumulation; In each iteration, the hidden state is updated using multi-scale contextual features through convolutional gated recurrent units, aiming to minimize the loss function of regularization loss and iterative loss. Finally, a residual is decoded to optimize the disparity map, and the optimized disparity map is output as the scene disparity estimation result. The formula for the optimized disparity map is as follows: ； ； In the formula, The optimized disparity map; This is the initial disparity map; The depth residual map for decoding; This is an update operator based on convolution-gated recurrent units; This is the hidden state from the previous moment; These are the features after fusion.
10. A computer system, comprising: The memory, the processor, and the computer program stored in the memory and executable on the processor are characterized in that the processor executes the computer program to implement the vehicle scene depth estimation method based on multi-scale attention and lightweight 3D convolution as described in any one of claims 1-9.