Lightweight target detection and distance measurement method and device based on binocular vision and storage medium

Abstract

The invention relates to the technical field of intelligent visual perception, in particular to a lightweight target detection and distance measurement method and device based on binocular vision and a storage medium, and the method comprises the steps: S1, synchronously collecting and preprocessing a left view and a right view of a binocular camera; s2, inputting the left and right views into a lightweight multi-scale fusion network LMSFN to obtain a target two-dimensional detection frame, and by integrating a multi-scale lightweight fusion module MSF, a channel mixing module CML and a space attention module SLA, the network enhances feature representation while reducing model parameter quantity; s3, generating a disparity map by using an improved semi-global block matching SGBM algorithm by taking the detection frame as a constraint, and performing optimization through a self-adaptive post-processing strategy based on edge perception; and S4, calculating a target three-dimensional distance according to the optimized disparity map and camera parameters. According to the method, high-precision detection and reliable distance measurement are realized while the light weight of the model is ensured, and the comprehensive sensing performance and the real-time performance of the system in a resource-constrained environment are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent visual perception, and in particular to a lightweight target detection and ranging method based on binocular vision, a device and a storage medium. BACKGROUND

[0002] In the field of intelligent visual perception, target detection and distance estimation are two core tasks for environment understanding and spatial interaction in applications such as autonomous driving, mobile robots, and intelligent surveillance. Existing technical solutions mainly develop along two paths of monocular vision detection and stereo vision ranging, and face the challenge of balancing lightweight deployment and high-precision perception in their combination.

[0003] In terms of target detection, deep learning-based methods, especially single-stage detectors represented by the YOLO series, have become mainstream due to their good balance between precision and speed. These methods extract image features through convolutional neural networks, directly predict the class and two-dimensional bounding box of the target, and can achieve real-time detection on embedded devices. However, monocular images lack depth information, and their output is only a two-dimensional position in the pixel coordinate system, which cannot directly provide the actual physical distance of the target. To obtain distance information, it is usually necessary to rely on additional monocular depth estimation algorithms or use other sensors, which not only increases the complexity and cost of the system, but also may introduce error accumulation problems.

[0004] In terms of distance estimation, stereo vision technology directly recovers scene depth by calculating the disparity between left and right views. Traditional algorithms such as Semi-Global Block Matching (SGBM) are widely used due to their balance between efficiency and accuracy. In recent years, deep learning-based stereo matching networks have shown stronger robustness in challenging scenarios. However, these methods usually have high computational complexity and are difficult to meet real-time requirements on resource-constrained edge devices; while traditional algorithms are prone to matching errors in object edges, weak textures or occluded areas, resulting in noisy, blurred or empty disparity maps, which directly affects the ranging accuracy.

[0005] To simultaneously complete detection and ranging, existing technologies propose solutions that combine the two. Typical strategies are to first independently perform target detection and stereo matching, and then fuse the results; or to use the detection results to constrain the area of stereo matching to improve efficiency. Although these combined methods can combine semantic and geometric information, in actual deployment, especially in edge computing scenarios that require lightweight, they still face significant contradictions: on the one hand, to meet real-time requirements, the detection network must be extremely compressed, but this often weakens its feature representation ability, leading to a decrease in detection accuracy, which in turn affects the distance estimation accuracy based on detection frames; on the other hand, the instability of the stereo matching module in complex scenarios makes it difficult to guarantee the quality of the final depth map, resulting in large ranging errors.

[0006] Therefore, how to coordinate and optimize the two links of target detection and stereo matching under limited hardware resources, while ensuring high real-time, to achieve high-precision and high-robust target detection and distance estimation is a technical problem to be solved in the current field. SUMMARY

[0007] The purpose of the present application is to solve the shortcomings in the prior art, and to provide a lightweight target detection and ranging method based on binocular vision, comprising the following steps: S1: acquiring left view and right view synchronously collected by a binocular camera; S2: inputting the left view and the right view into a lightweight multi-scale fusion network LMSFN to obtain a two-dimensional detection box containing target categories and target positions, wherein the lightweight multi-scale fusion network LMSFN integrates a multi-scale lightweight fusion module MSF, a channel mixing module CML and a spatial attention module SLA in a feature extraction backbone to enhance the feature representation and positioning ability of multi-scale targets while reducing the model parameter amount; S3: applying an improved semi-global block matching algorithm SGBM for stereo matching in the corresponding image regions of the left view and the right view as spatial constraints, generating an initial disparity map, and performing region-adaptive progressive post-processing on the initial disparity map to obtain an optimized high-quality disparity map; S4: calculating the three-dimensional spatial distance of the target in the two-dimensional detection box according to the optimized high-quality disparity map and the calibration parameters of the binocular camera, and completing the synchronous output of the target detection result and the ranging result.

[0008] Preferably, in step S2, inputting the left view and the right view into the lightweight multi-scale fusion network LMSFN to obtain a two-dimensional detection box containing target categories and target positions comprises: S21: inputting the left view and the right view into a lightweight feature extraction backbone of the lightweight multi-scale fusion network LMSFN respectively, extracting primary visual features of different scales through multi-layer depth separable convolution and downsampling operation; S22: inputting the primary visual features into the multi-scale lightweight fusion module MSF, extracting and fusing features of different receptive fields through parallel multi-branch lightweight convolution branches to obtain multi-scale fusion feature maps; S23: inputting the multi-scale fusion features into the channel mixing module CML, obtaining reorganized feature maps through channel grouping, convolution and dimension rearrangement operations to realize cross-channel information interaction and feature reorganization; S24: inputting the reorganized feature maps into the spatial attention module SLA, enhancing target region features through spatial attention weights, and outputting enhanced features; S25: input the enhanced features into a bidirectional feature pyramid for multi-scale fusion to obtain a final multi-scale feature map for detection; S26: input the final multi-scale feature map into a detection head to respectively predict target class probabilities and two-dimensional bounding box coordinates on different scale feature maps, and perform non-maximum suppression processing to eliminate redundant detection boxes, and output a final two-dimensional bounding box set.

[0009] Preferably, in step S22, the primary visual features are input into the multi-scale lightweight fusion module MSF, and features of different receptive fields are extracted and fused through parallel multi-branch lightweight convolution branches, including: The multi-scale lightweight fusion module MSF sets parallel lightweight convolution branches of different scales, each branch respectively performs convolution operation on the input features output by the feature extraction backbone to obtain multi-scale feature maps of corresponding scales , k takes 1, 3, 5, 7; The multi-scale feature maps of each scale are spliced along the channel dimension to obtain a feature splicing tensor ; 1×1 lightweight convolution operation is performed on the spliced feature tensor to integrate channel dimension features and output multi-scale fusion features , as follows:

[0010] Preferably, in step S23, the multi-scale fusion features are input into the channel mixing module CML to obtain reorganized feature maps through channel grouping, convolution and dimension rearrangement operations, including: The multi-scale fusion feature map is divided into G channel groups in the channel dimension, and the multi-scale fusion feature map is dimensionally reshaped to obtain a reshaped feature tensor , as follows: , where C / G is the number of channels in each channel group, and the dimension of the multi-scale fusion feature map is , where B is the batch size, C is the total number of feature map channels, H is the feature map height, and W is the feature map width; 1×1 lightweight group convolution operation is performed on each channel group in the reshaped feature tensor to complete the intra-group feature transformation, and the feature tensor dimension remains unchanged; Permutation operation is performed on the feature tensor after group convolution in the channel dimension to obtain a permuted feature tensor , as follows: ; ; ; ; ; ; ;

[0011] ; ; ; ; ; ; ; ; ;

[0012] ; ; ; ; ; ; ;

[0013] ; S41: Obtain the optimized high-quality parallax map, and establish a mapping relationship from a parallax value to a physical distance according to pre-calibration parameters of the binocular camera, the pre-calibration parameters including a baseline distance, a camera focal length, an intrinsic matrix, and an extrinsic matrix; S42: For each two-dimensional detection box, screen effective parallax values in a corresponding region of the optimized high-quality parallax map, and calculate an effective parallax value mean; S43: Convert the effective parallax value mean to a depth value in a three-dimensional space based on the mapping relationship; S44: Calculate a three-dimensional space position of the target in a camera coordinate system according to the depth value and a center pixel coordinate of the two-dimensional detection box, and output a final perception result including a target category, a two-dimensional bounding box, and a corresponding three-dimensional distance.

[0014] Preferably, in step S1, the left and right views captured by the binocular camera are obtained, including: S11: Initialize the binocular camera system, and perform stereo rectification on the left and right cameras to ensure that their imaging planes are parallel and aligned; S12: Synchronously trigger the left and right cameras to capture an initial left view and an initial right view of the scene at the same timestamp; S13: Perform preprocessing including denoising, brightness equalization, and geometric alignment on the initial left view and the initial right view to obtain the left view and the right view.

[0015] Based on the same concept, the present application also provides a computer device comprising a memory and one or more processors, the memory storing computer code, the computer code being executed by the one or more processors to cause the one or more processors to perform the steps of the light-weight target detection and ranging method based on binocular vision according to any one of the embodiments.

[0016] Based on the same concept, the present application also provides a storage medium storing computer readable instructions, wherein the computer readable instructions are executed by one or more processors to cause the one or more processors to perform the steps of the light-weight target detection and ranging method based on binocular vision according to any one of the embodiments.

[0017] Compared with the prior art, the present application has the following advantages: The application realizes the significant reduction of model parameter quantity and the synchronous enhancement of multi-scale feature representation ability by designing and integrating a multi-scale lightweight fusion module MSF, a channel mixing module CML and a spatial attention module SLA, thereby achieving an excellent balance between high detection accuracy and high real-time performance on resource-constrained edge devices, and solving the industry problem that lightweight deployment and sensing performance are difficult to balance. The application realizes noise suppression, edge sharpening and hole filling on the initial disparity map by applying an improved semi-global block matching SGBM algorithm and performing a region-adaptive progressive post-processing strategy in the constraint region of the target detection frame, generates a high-quality optimized disparity map, thereby significantly improving the robustness of stereo matching and the integrity of depth information in complex scenes (such as weak texture, occlusion), and laying a reliable foundation for high-precision ranging. The application realizes the organic integration and mutual enhancement of target detection and three-dimensional ranging tasks by the cooperative framework of guiding the stereo matching region with the detection frame and associating and outputting the optimized depth information with the detection result, thereby guaranteeing the high accuracy and high reliability of the final perception result (target category, position, distance) at the system level, and providing an efficient and reliable end-to-end solution for real-time visual perception applications such as autonomous driving and robots. BRIEF DESCRIPTION OF DRAWINGS

[0018] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for purposes of illustration only and are not considered a limitation of the present application.

[0019] Figure 1 Flowchart of the application based on binocular vision lightweight target detection and ranging method; Figure 2 Overall architecture diagram of the lightweight multi-scale fusion network LMSFN of the application; Figure 3 Overall structure diagram of the multi-scale lightweight fusion module MSF in the application; Figure 4 Overall structure diagram of the channel mixing module CML in the application; Figure 5 Overall structure diagram of the spatial attention module SLA in the application; Figure 6 Disparity map comparison diagram before and after improvement of the semi-global block matching algorithm SGBM in the application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0021] Those skilled in the art can understand that the singular form "a", "an", "the" used herein can include the plural form, unless specifically stated otherwise. It should be further understood that the use of the word "comprise" in the specification of the present application means that the features, integers, steps, operations, elements and / or components described exist, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0022] First embodiment Please refer to Figure 1 and Figure 2 It is shown that the present embodiment provides a lightweight target detection and ranging method based on binocular vision, mainly for real-time visual perception scenes such as automatic driving and intelligent robots. Through improving the lightweight detection model and region adaptive SGBM disparity optimization strategy, the synchronous improvement of detection and ranging is realized, and the internal relationship between detection accuracy and ranging accuracy is deeply explored. In the whole design and implementation process, the compactness and operation efficiency of the model are maintained, and the robustness and stability in complex environment are effectively enhanced, which ensures that it can run efficiently on resource-limited edge devices, solves the problems of high model complexity and large calculation resource consumption of traditional detection network, and the lightweight multi-scale fusion network LMSFN reduces the parameter quantity and calculation complexity under the premise of maintaining the detection accuracy, which can better adapt to the deployment scene of resource-limited edge devices, including the following steps: S1: acquiring left view and right view synchronously collected by binocular camera.

[0023] Preferably, in step S1, the left and right views collected by the binocular camera are acquired, including: S11: initializing the binocular camera system, using Zhang's calibration method to correct the left and right cameras, solving the camera intrinsic parameters, distortion coefficients and extrinsic parameters, and ensuring that the imaging planes are parallel and aligned through re-projection transformation; S12: based on the hardware synchronization trigger mechanism, synchronously triggering the left and right cameras, collecting the initial left view and the initial right view of the scene at the same timestamp; S13: preprocessing the initial left view and the initial right view, including 5x5 Gaussian filter denoising, histogram equalization to achieve brightness balance, and geometric alignment based on polar line constraint, to obtain a left view and a right view. Specifically, in this embodiment, the preprocessing operations all adopt a pixel-by-pixel fast calculation manner, without redundant floating-point operations.

[0024] S2: inputting the left view and the right view into a lightweight multi-scale fusion network LMSFN to obtain a two-dimensional detection frame containing a target category and a target position. The lightweight multi-scale fusion network LMSFN integrates a multi-scale lightweight fusion module MSF, a channel mixing module CML, and a spatial attention module SLA in a feature extraction backbone, so as to enhance the feature representation and positioning ability of a multi-scale target while reducing the model parameter amount. Specifically, in this embodiment, the lightweight multi-scale fusion network LMSFN introduces three original modules in the Backbone, which significantly reduces the model parameter amount and computational complexity while enhancing the multi-scale feature representation ability, and is suitable for resource-constrained edge devices. In the Neck part, an optimized FPN+PAN structure is used for bidirectional feature fusion, and an ECA attention mechanism and a CML module are embedded, which further improves the detection performance of targets of different scales.

[0025] Please refer to Figure 2 In step S2, the left view and the right view are input into a lightweight multi-scale fusion network LMSFN to obtain a two-dimensional detection frame containing a target category and a target position. The hierarchical design of the backbone network, the feature fusion network, and the detection head is used, but targeted innovation and modification are made in each part to improve the feature expression ability and reduce the computational cost, ensuring that the feature extraction under different scales can balance accuracy and efficiency, thereby effectively improving the detection effect of small, medium, and large targets, including: S21: inputting the left view and the right view into a lightweight feature extraction backbone of the lightweight multi-scale fusion network LMSFN, extracting primary visual features of different scales (dimension ) through multi-layer depth separable convolution and downsampling operations, where B is the batch size, is the number of s-scale feature channels, , is the height and width of the s-scale feature map), Specifically, in this embodiment, in the backbone network, multi-scale feature layers (P2-P5) are constructed by stepwise downsampling. S22: after the key convolution layer of the feature extraction backbone, the primary visual features are input into a multi-scale lightweight fusion module MSF to extract and fuse features of different receptive fields through parallel multi-branch lightweight convolution branches, and obtain multi-scale fusion feature maps. S23: input the multi-scale fusion features into a channel mixing module CML, obtain a reorganized feature map through channel grouping, 1x1 lightweight group convolution and dimension rearrangement operation, realize cross-channel information interaction and feature reorganization; S24: input the reorganized feature map into a spatial attention module SLA, enhance the target region features through spatial attention weight, output enhanced features, specifically, in this embodiment, calculate the maximum and minimum values in the channel dimension to generate a spatial attention weight map, multiply the weight map with the remodeled feature map pixel by pixel to obtain spatial attention features, so as to enhance the target region feature response and suppress background interference; S25: input the multi-scale fusion feature map, the remodeled feature map and the enhanced features into a bidirectional feature pyramid for multi-scale fusion, obtain the final multi-scale feature map for detection, specifically, in this embodiment, in the feature fusion network part, an improved bidirectional feature pyramid structure (FPN+PAN) is adopted, the FPN path transmits high-level semantic information to the shallow layer through up-sampling to enhance the small target detection capability, the PAN path supplements the shallow layer details to the high-level features through down-sampling, so as to improve the positioning accuracy of the target boundary, the efficient channel attention mechanism ECA and the CML module are introduced at the key fusion node, which effectively enhances the fusion effect of the multi-scale features and further improves the detection performance and stability, this design not only maintains the integrity of the detection framework in the structure level, but also realizes the unity of lightweight and high performance through modular innovation; S26: input the final multi-scale feature map into the detection head, predict the target class probability and two-dimensional bounding box coordinates on different scale feature maps respectively, and remove redundant detection boxes through non-maximum suppression processing, output the final two-dimensional detection box set, specifically, in this embodiment, the final output multi-scale feature map (P3, P4 and P5) can be used for detection of different size targets, realizing accurate identification and positioning of multiple targets in complex scenes.

[0026] In a complex scene containing multiple overlapping targets, small-scale objects (such as distant pedestrians or signs) usually coexist with large-scale objects (such as nearby vehicles), and a single receptive field is not enough to effectively process both. Traditional convolutional networks usually rely on down-sampling stacks to expand the receptive field for multi-scale object processing. However, this approach faces two key problems: (1) deep features are not sensitive to small objects; (2) shallow features lack global semantics; the MSF module solves these limitations by introducing multiple receptive field convolution operations within the same feature map and fusing them in a lightweight manner.

[0027] Please refer to Figure 3As shown, in step S22, the primary visual features are input into the multi-scale lightweight fusion module MSF. The features of different receptive fields are extracted and fused through parallel multi-branch lightweight convolution branches. Different convolution kernels capture local and global feature information. In the design of the MSF, the output channels of each branch are limited to one quarter of the total number of channels. Finally, the output is combined through a 1x1 convolution to achieve a stable channel dimension while maintaining feature diversity, improving the network's ability to perceive objects at different scales, including: The multi-scale lightweight fusion module MSF sets up several parallel lightweight convolution branches of different scales. Each branch performs convolution operations on the input features output by the feature extraction backbone to obtain multi-scale feature maps of corresponding scales , k takes 1, 3, 5, and 7; The multi-scale feature maps of each scale are spliced along the channel dimension to obtain a feature splicing tensor 、 、 、 ; A 1x1 lightweight convolution operation is performed on the spliced feature tensor to integrate channel dimension features and output multi-scale fusion features , as shown below: Specifically, in this embodiment, this design enables the network to extract multi-receptive field features in parallel at the same resolution, while capturing local texture (through smaller convolution kernels) and global context (through larger convolution kernels). It has obvious advantages in small target recognition and long-distance target detection. By fusing multi-scale convolution branches, the feature representation is more rich, which alleviates the problems of long-distance target blur and small target loss. The lightweight structure design ensures limited computational overhead, making it suitable for real-time detection systems and embedded devices, while balancing high precision and high efficiency.

[0028] In deep detection networks, channels usually carry rich semantic information. However, traditional convolution methods either ignore the interaction between channels, leading to insufficient information utilization, or rely on standard convolution, which introduces a large number of parameters and computational overhead. To enhance the interaction between channels, traditional convolution design usually requires additional convolution kernels (such as 3x3 or 1x1 convolution), which inevitably increases the number of parameters and computational overhead. The core idea of the channel mixing module CM is to break the fixed connection between channels without adding new convolution kernels by using channel grouping and rearrangement mechanisms.

[0029] Please refer to Figure 4 ​As shown, in step S23, the multi-scale fusion feature is input into the channel mixing module CML, and the reconstructed feature map is obtained through channel grouping, convolution, and dimension rearrangement operations, including: Multi-scale fusion feature maps The feature maps are divided into G channel groups along the channel dimension and fused at multiple scales. Dimensional reshaping is performed to obtain the reshaped feature tensor. As shown below: Where C / G represents the number of channels in each channel group, and the multi-scale fused feature map. The dimension is Where B is the batch size, C is the total number of channels in the feature map, H is the feature map height, and W is the feature map width. Specifically, in this embodiment, firstly, group convolution reduces computation. The input channels C are divided into G groups, and each group undergoes a lightweight 1×1 convolution. Since each kernel only operates on a small portion of the channels, the overall parameter size is reduced from traditional... reduce This significantly reduces computing costs; Reconstructing the feature tensor Each channel group performs a 1×1 lightweight group convolution operation to complete the feature transformation within the group while keeping the feature tensor dimension unchanged. Specifically, in this embodiment, channel rearrangement makes inter-group interaction possible. The disadvantage of group convolution is that the channels become isolated within the group, preventing direct interaction between groups. Perform channel dimension permutation on the feature tensor after group convolution. The permuted feature tensor is obtained. As shown below: Specifically, in this embodiment, in order to address the shortcomings of grouped convolution, the channel blending module CML introduces a channel permutation operation after grouped convolution. This operation reshuffles and reorders the outputs of different groups, ensuring that each group contains features from multiple previous groups in the next layer. This shaping and permutation step ensures that features are globally blended, effectively simulating the effect of fully connected channel convolution. Will Remodeled into a feature map with multi-scale fusion With consistent dimensions, the recombined feature map F is obtained, as shown below: , specifically, in this embodiment, this design can enhance the interaction between channels without increasing parameters, because the reshaping and permutation steps reorganize the existing channel features so that they can interact across groups in subsequent layers. Essentially, it maximizes the expressive power of the network by reusing the same features in different combinations, achieving a near fully connected channel mixing effect without additional convolutional kernel parameters. In this way, CML not only improves the smoothness of information flow, but also enhances the discriminability in multi-class scenarios, which is particularly evident in traffic environments, because vehicles and pedestrians are often very close in space: CML improves class separability by reducing inter-class confusion. In addition, channel rearrangement ensures that multi-scale fusion features are neither redundant nor isolated at the channel level, thereby enhancing the overall quality of the fusion representation. This will bring higher bounding box classification confidence and more accurate positioning, providing clearer target constraints for subsequent depth estimation tasks.

[0030] In complex environments, the background often contains a large amount of interference information, such as shadows, reflections, vegetation, and other non-target structures. If the network reacts too strongly to these areas, it is very easy to produce false positives and boundary deviations. When the target is partially occluded, it is also difficult for traditional networks to capture the remaining effective features, which often leads to detection failure. The design of the SLA module is precisely to address the problem of insufficient separation of foreground and background. It introduces a lightweight spatial attention mechanism to guide the network to focus on the target area. Its implementation includes three main steps. First, calculate the maximum and minimum values along the channel dimension. The maximum pooling captures the most prominent activations, while the minimum pooling preserves the inhibitory information. Combining the two forms a more comprehensive spatial description. Second, apply a 1x1 convolution to the concatenated maximum and minimum value features to generate attention weights, which hardly increases the computational cost. Finally, multiply these attention weights with the original feature map element by element. This operation increases the relative weight of the target-related regions, while reducing the activation related to the background or noise, allowing the network to focus more accurately on the objects of interest.

[0031] Referring to Figure 5 In step S24, the reorganized feature map is input into the spatial attention module SLA to enhance the target region features through spatial attention weights, including: The reorganized feature map F is subjected to maximum value pooling and minimum value pooling operations along the channel dimension respectively to obtain a maximum value feature map Max(F) and a minimum value feature map Min(F); The maximum value feature map Max(F) and the minimum value feature map Min(F) are concatenated along the channel dimension to obtain a concatenated feature map [Max(F), Min(F)]; A 1x1 lightweight convolution is performed on the concatenated feature map, and a Sigmoid activation function Generating spatial attention weight maps As shown below: Specifically, in this embodiment, the SLA can effectively extract the most representative spatial response, highlight the foreground region, and suppress the redundant background response. This mechanism enhances the network's ability to maintain attention on the true target boundary in the case of edge blur or partial occlusion. The lightweight design of the SLA ensures that its computational overhead is almost negligible, while significantly improving the robustness of the network. The enhanced foreground attention also provides higher-quality area constraints for subsequent depth estimation. The spatial attention weight map is multiplied pixel by pixel with the reorganized feature map F to obtain a spatial attention feature map, realizing target region feature enhancement and background suppression.

[0032] The above three types of modules optimize the feature representation from three aspects of scale perception, channel interaction, and spatial focusing, so that the LMSFN network still maintains excellent detection performance while being miniaturized.

[0033] In terms of neck structure, this embodiment adopts an improved FPN and PAN combination strategy to pass high-level semantic information from top to bottom to the shallow layer, and pass low-level detail features from bottom to top to the deep layer, thereby enhancing the multi-scale feature fusion effect. ECA attention and CML modules are introduced at the fusion node to further improve the representation consistency of cross-scale features. Finally, the detection head part outputs prediction results of P3, P4, and P5 three scales, respectively corresponding to the detection tasks of small, medium, and large targets. This design not only retains the real-time advantage of the YOLO series, but also meets the high-precision detection requirements under the condition of lightweight.

[0034] In the ranging module, this embodiment makes systematic improvements to the traditional SGBM matching algorithm. The original SGBM often has problems such as edge blur, texture missing area holes, and serious noise pollution in complex scenes, which seriously affect the quality of the depth map. To solve these problems, this embodiment uses five stages: In the first stage, the original disparity map is median filtered to suppress isolated noise points; In the second stage, the Canny operator is used to extract edge information and construct a region-aware mask; In the third stage, the Unsharp Mask technique is used to sharpen the edge area and enhance the target boundary; In the fourth stage, a conditional fusion strategy is used, using the sharpening result in the edge area and the smoothing result in the non-edge area to balance detail preservation and noise suppression; In the fifth stage, for the hole area caused by matching failure, morphological operation and region consistency strategy are combined for intelligent filling to ensure the continuity and integrity of the disparity map. Through step-by-step optimization and region-aware control, this method effectively improves the blur and hole problems of the original SGBM, and the generated depth map is significantly improved in target boundary clarity and global consistency.

[0035] S3: applying a modified semi-global block matching algorithm SGBM to the corresponding image regions of the left view and the right view within the two-dimensional detection frame as a spatial constraint to perform stereo matching, generating an initial disparity map, and performing region-adaptive progressive post-processing on the initial disparity map to obtain an optimized high-quality disparity map. Specifically, in this embodiment, through a five-stage progressive optimization process (including noise suppression, edge detection, detail enhancement, conditional fusion, and hole filling), the edge blur, noise, and hole problems caused by the original SGBM algorithm are effectively solved, a higher-quality and more continuous disparity map is generated, and the ranging accuracy is improved.

[0036] Preferably, in step S3, the region-adaptive progressive post-processing is further performed on the initial disparity map, which further includes: S31: performing a median filtering operation with an adaptive window size on the initial disparity map to obtain a first intermediate disparity map, and removing isolated noise points, specifically, in this embodiment, the window size is adaptively selected as 3x3 (for low-texture regions) or 5x5 (for high-texture regions) according to the local texture complexity of the disparity map, isolated noise points are suppressed, and the first intermediate disparity map is obtained; S32: extracting edge information of the first intermediate disparity map using a Canny edge detection algorithm, dynamically adjusting a gradient threshold based on the global gradient mean of the first intermediate disparity map (upper threshold = global gradient mean x 1.2, lower threshold = global gradient mean x 0.5), and generating a binary edge region mask (mask pixel value 1 represents an edge region, and 0 represents a non-edge region; S33: performing sharpening processing on the first intermediate disparity map in the edge region indicated by the edge region mask using a non-sharpening mask technique (the sharpening coefficient is set to 0.8), enhancing the object boundary definition, and obtaining a second intermediate disparity map; S34: using a pixel-level conditional fusion strategy to retain the sharpening result of the second intermediate disparity map in the edge region and retain the median filtering result of the first intermediate disparity map in the non-edge region, and obtaining a third intermediate disparity map; S35: identifying invalid hole regions (regions with a disparity value of 0 or beyond the reasonable disparity range of the binocular camera) in the third intermediate disparity map due to stereo matching failure, filling small-size holes through a morphological closing operation with a 3x3 structural element, and filling (calculating the mean and variance of the valid disparity values in the hole neighborhood, and selecting the neighborhood disparity value with the smallest variance as the filling value) the remaining large-size holes based on the region consistency strategy of adjacent valid disparity values, and obtaining an optimized high-quality disparity map.

[0037] Please refer to Figure 6As shown, from left to right, the original color image, the disparity map generated by the traditional SGBM algorithm, and the disparity map generated by the improved ISGBM algorithm are respectively displayed. The selected scenes are representative examples, aiming to evaluate the robustness and depth estimation performance of the algorithm under various challenging conditions. The first row is a cluttered indoor scene with various materials and occlusions, testing the stability of the algorithm in high-texture and partially occluded environments. The second row shows a wooden chair and a chessboard, checking the accuracy of depth estimation in structured geometric patterns. The third row depicts a classroom with a repeated arrangement of seats, evaluating the continuity of depth estimation in low-texture areas. The fourth row demonstrates the improvement of the improved ISGBM algorithm in preserving detail and edge sharpness through analysis of images of stacked wooden boards with complex spatial geometry. Overall, the selection of these images highlights the improved ISGBM algorithm's improvements in edge handling, texture consistency, and depth continuity in different scene types.

[0038] As can be seen directly from the figure, the results of the traditional SGBM algorithm have obvious limitations. In the edge area of the object, the disparity distribution appears blurred and faulted, the object outline is not clear enough, especially at high-contrast boundaries, the depth is discontinuous. For complex texture or weak texture background, SGBM is prone to large-area holes (black areas), which represent pixels that cannot effectively calculate the disparity value, resulting in missing depth information. Overall, the disparity map generated by SGBM has low quality, with problems such as noise, unclear levels, and blurred object shape.

[0039] In contrast, the improved ISGBM algorithm shows significant improvements in several aspects. First, in the edge area of the object, the ISGBM disparity profile is clearer and sharper, with natural depth boundary transitions, significantly improving the edge blur problem of the original algorithm. This is due to the introduction of edge preservation strategies and cost aggregation optimization mechanisms in the algorithm, making the matching process between pixels more consistent in direction. Second, in complex backgrounds, the hole area is significantly filled, and the area of black invalid regions is significantly reduced. By introducing confidence-based cost correction and disparity filling strategies, ISGBM can effectively restore the lost depth information, making the disparity distribution more complete and continuous. In addition, the improved algorithm also enhances the noise suppression capability in smooth areas, with more natural depth levels and more realistic near-far relationships.

[0040] S4: According to the optimized high-quality disparity map and the calibration parameters of the binocular camera, the three-dimensional spatial distance of the target in the two-dimensional detection frame is calculated, and the synchronous output of the target detection result and the ranging result is completed.

[0041] Preferably, in step S4, calculating the three-dimensional spatial distance of the target in the two-dimensional detection frame comprises: S41: Obtain an optimized high-quality disparity map, and establish a mapping relationship of disparity value to physical distance according to pre-calibration parameters of the binocular camera, the pre-calibration parameters including a baseline distance , a camera focal length f, an intrinsic matrix K, and an extrinsic matrix R / T, as follows: wherein, is a disparity value; S42: For each two-dimensional detection frame, in a corresponding region of the optimized high-quality disparity map, filter effective disparity values and remove abnormal values beyond a range of , wherein is a minimum / maximum effective disparity value obtained by binocular camera calibration, calculate an effective disparity value mean ; S43: Based on the mapping relationship, convert the effective disparity value mean to a depth value in a three-dimensional space ; S44: According to the depth value and the center pixel coordinates of the two-dimensional detection frame , calculate a three-dimensional space position (X, Y, Z) of the target in a camera coordinate system in combination with camera intrinsic parameters, and output a final perception result containing a target category, a two-dimensional bounding box, and a corresponding three-dimensional distance, as follows: , wherein , is a camera principal point coordinate, is a focal length of the camera in the x / y axis; and the final output is a final perception result containing a target category, a two-dimensional bounding box, and a corresponding three-dimensional distance.

[0042] The embodiment continues the idea of combining target detection and stereo matching, and the input binocular image is subjected to foreground target detection and positioning by a multi-scale lightweight detection network LMSFN, candidate regions are extracted under the constraint of the detection frame, and an improved SGBM matching algorithm is input to generate a high-quality disparity map, so as to complete three-dimensional distance estimation of the target. The overall process is closely connected from network structure optimization, algorithm design, experimental verification to theoretical analysis, and forms a complete implementation path.

[0043] In order to evaluate the contribution of each module to the overall detection performance, the embodiment uses the KITTI dataset for step-by-step ablation research, and the results are shown in Table 1.

[0044] Table 1 Ablation experiment of different modules on the KITTI dataset Starting from the baseline YOLOv11n model with a parameter size of 2.6M, the mAP50 and mAP50-95 are 0.584 and 0.345, respectively. These results can be used as a reference for subsequent improvements. After adding the MSF module, the model size is reduced to 1.65M, while the detection accuracy is improved, with mAP50 increasing to 0.596 and mAP50-95 increasing to 0.350. This shows that MSF can reduce model redundancy while enhancing the robustness of multi-scale feature representation. The further combination of the CML module reduces the model parameters to 1.38M, with mAP50 and mAP50-95 increasing to 0.614 and 0.372, respectively. This shows that CML effectively optimizes feature distribution and improves target discriminability. In addition, the integration of the SLA mechanism further compresses the model to 1.25M, with mAP50 and mAP50-95 reaching 0.630 and 0.390, achieving the best performance. These results show that SLA significantly improves feature selectivity, highlighting key target regions, complementing the effects of previous modules, and achieving high precision and model compactness. In summary, the ablation study shows that MSF, CML, and SLA each improve detection performance at different levels, and their joint use shows a clear synergistic effect. The final integrated model achieves significant improvements over the baseline YOLOv11n while maintaining extremely low parameter counts. Table 2 summarizes the comparison of our proposed LMSFN model with several state-of-the-art lightweight and standard YOLO variants in terms of model size, computational complexity (GFLOP), and KITTI dataset detection performance.

[0045] Table 2 Comparison of different algorithms Overall, the YOLOv8 series shows a clear hierarchical performance difference at different scales. The YOLOv8n model contains only 2.86M parameters, requires 8.1 GFLOPs, and has an mAP50 of 0.61, but the precision is limited. As the model size increases, the mAP50 of YOLOv8s, YOLOv8m, and YOLOv8l gradually increases to 0.751, 0.817, and 0.867, respectively, among which YOLOv8l has the highest detection accuracy, with an mAP50-95 of 0.61. However, its computational cost is as high as 164.8 GFLOPs, which is not suitable for real-time deployment. The YOLOv10 series achieves a better balance of performance through structural optimization and detection head adjustment. The lightweight YOLOv10n model contains 11.73M parameters, with an mAP50 of 0.453 while maintaining basic accuracy. As the parameters increase to 16.5M and 25.7M, the mAP50 of YOLOv10m and YOLOv10l reaches 0.688 and 0.715, respectively, showing a typical positive correlation between performance and computational cost. As the latest generation, YOLOv11 further enhances the feature expression capability. YOLOv11n maintains a lightweight structure of 2.6M while achieving an mAP50 of 0.584; the medium-sized YOLOv11s and YOLOv11m achieve an mAP50 of 0.705 and 0.771, respectively, after parameter expansion, showing good scalability; while YOLOv11l achieves an mAP50 of 0.768 with 25.3M parameters, close to YOLOv8l, but with much lower computational cost, reflecting the improvement of architecture efficiency.

[0046] In traditional anchor-based methods, SSD, RetinaNet, and ATSS are strong baseline models. The SSD model (24.68M parameters, 184 GFLOPs) achieves 0.785 on mAP50, while RetinaNet (36.48M parameters, 110 GFLOPs) and ATSS (32.13M parameters, 113 GFLOPs) further improve this metric to 0.805 and 0.872. However, their high computational cost makes real-time inference impractical. These algorithms achieve high accuracy at the expense of speed and energy efficiency, showing a significant trade-off between structural complexity and detection performance. Compared with the above models, LMSFN demonstrates superior performance at extremely low complexity. With only 1.25M parameters and 7.9 GFLOPs, it achieves 0.63 on mAP50 and 0.39 on mAP50-95. Compared with YOLOv11n, LMSFN reduces parameters by more than 50% and improves accuracy by nearly 8%; compared with YOLOv10n, the parameter compression is more than 90%, and the accuracy improvement is greater. Even compared with medium-sized models such as YOLOv10s or YOLOv11s, LMSFN can achieve almost the same accuracy (within a difference of about 0.07), while reducing parameters and computational cost to less than one-tenth. This indicates that its architecture design, especially in terms of multi-scale feature fusion and channel compression strategies, has significant advantages, enabling stable detection performance with minimal resource cost.

[0047] In addition, the joint analysis of GFLOPs and mAP shows that LMSFN has significantly higher performance density (detection accuracy per unit computational cost) compared to other models. While large models maintain high accuracy, their performance improvement gradually weakens as computational load increases; notably, despite a several-fold increase in GFLOPs, the mAP50 growth of YOLOv8l and ATSS remains limited. In contrast, LMSFN achieves approximately 65% of YOLOv8l detection accuracy with only 5% of the computational cost, demonstrating its significant advantage in energy efficiency. This high cost-effectiveness makes LMSFN particularly suitable for deployment in autonomous vehicles, embedded systems, and low-power intelligent devices that require real-time and high-precision detection. In summary, LMSFN demonstrates outstanding structural efficiency and robustness across different-sized models. It leads in lightweight models, performs close to medium-sized models, and still competes with large networks with minimal resource consumption. These results prove that the optimization of LMSFN in model compression, feature fusion, and inference speed is both effective and balanced, verifying its advanced nature and practical value as a lightweight object detection network.

[0048] In combination with the above experimental results, it can be seen that the binocular detection and distance measurement optimization method proposed in this embodiment achieves significant improvement in both detection and distance measurement. In the detection link, the LMSFN model realizes the possibility of efficient deployment on edge devices through the lightweight structure innovation; in the distance measurement link, the region adaptive SGBM post-processing significantly improves the quality of the disparity map. Overall, this embodiment realizes a good balance between detection accuracy, distance measurement accuracy and real-time performance, and has a wide application prospect.

[0049] In practical application, this method can be directly deployed in the vehicle-mounted binocular camera system of an autonomous vehicle to realize real-time environmental perception and distance estimation; in the field of intelligent robots, this method can be used to realize accurate detection and obstacle avoidance of obstacles in the environment; in augmented reality and security monitoring scenarios, this method can also provide high-reliability three-dimensional perception capability. Compared with traditional single detection or single distance measurement methods, this embodiment organically integrates both, and realizes lightweight optimization and precision improvement at the algorithm level, providing a feasible path for low-power high-performance visual perception.

[0050] Second embodiment Based on the same idea, the present application further provides a computer device comprising a memory and one or more processors, the memory storing computer code, the computer code being executed by the one or more processors to cause the one or more processors to perform the steps of the lightweight target detection and distance measurement method based on binocular vision in the first embodiment.

[0051] Based on the same idea, the present application further provides a storage medium storing computer readable instructions, characterized in that the computer readable instructions are executed by one or more processors to cause the one or more processors to perform the steps of the lightweight target detection and distance measurement method based on binocular vision as claimed in any one of the claims.

[0052] It can be understood that, for the aforementioned light weight target detection and ranging method based on binocular vision, if all are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer server, a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0053] The computer readable storage medium can include a data signal carried in the baseband or as a part of a carrier wave propagating through the program code. Such a propagating data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, which can send, propagate or transmit programs for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained on the readable storage medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0054] The above is only the preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solution falling within the scope of the present application should be considered as falling within the protection scope of the present application. It should be noted that, for ordinary skilled persons in the art, some improvements and refinements without departing from the principles of the present application should also be considered as falling within the protection scope of the present application.

Claims

1. A lightweight target detection and ranging method based on binocular vision, characterized in that, Includes the following steps: S1: Obtain the left and right views simultaneously captured by the binocular cameras; S2: Input the left view and the right view into the lightweight multi-scale fusion network LMSFN to obtain a two-dimensional detection box containing the target category and target location. The lightweight multi-scale fusion network LMSFN integrates the multi-scale lightweight fusion module MSF, the channel mixing module CML and the spatial attention module SLA in the feature extraction backbone to enhance the feature representation and localization ability of multi-scale targets while reducing the number of model parameters. S3: Using the two-dimensional detection box as a spatial constraint, the improved semi-global block matching algorithm SGBM is applied to the corresponding image regions of the left view and the right view to perform stereo matching, generate an initial disparity map, and perform region-adaptive progressive post-processing on the initial disparity map to obtain an optimized high-quality disparity map. S4: Based on the optimized high-quality disparity map and the calibration parameters of the binocular camera, calculate the three-dimensional spatial distance of the target within the two-dimensional detection frame, and complete the synchronous output of the target detection result and the ranging result.

2. The lightweight target detection and ranging method based on binocular vision according to claim 1, characterized in that, In step S2, the left view and the right view are input into a lightweight multi-scale fusion network (LMSFN) to obtain a two-dimensional detection box containing the target category and target location, including: S21: Input the left view and the right view into the lightweight feature extraction backbone of the lightweight multi-scale fusion network LMSFN respectively, and extract primary visual features of different scales through multi-layer deep separable convolution and downsampling operations. S22: Input the primary visual features into the multi-scale lightweight fusion module MSF, extract and fuse features from different receptive fields through parallel multi-branch lightweight convolution branches to obtain a multi-scale fusion feature map; S23: Input the multi-scale fusion features into the channel mixing module CML, and obtain the reconstructed feature map through channel grouping, convolution and dimension rearrangement operations to realize cross-channel information interaction and feature reconstruction; S24: Input the reconstructed feature map into the spatial attention module SLA, enhance the target region features through spatial attention weights, and output the enhanced features; S25: The enhanced features are fed into a bidirectional feature pyramid for multi-scale fusion to obtain the final multi-scale feature map for detection; S26: Input the final multi-scale feature map into the detection head, predict the target class probability and two-dimensional bounding box coordinates on the feature maps of different scales respectively, and remove redundant detection boxes through non-maximum suppression processing, and output the final set of two-dimensional detection boxes.

3. The lightweight target detection and ranging method based on binocular vision according to claim 2, characterized in that, In step S22, the primary visual features are input into the multi-scale lightweight fusion module MSF, and features from different receptive fields are extracted and fused through parallel multi-branch lightweight convolutional branches, including: The multi-scale lightweight fusion module (MSF) is configured with several parallel lightweight convolutional branches of different scales. Each branch performs a convolution operation on the input features output from the feature extraction backbone to obtain a multi-scale feature map of the corresponding scale. k takes the values ​​1, 3, 5, or 7. The multi-scale feature maps at various scales are concatenated along the channel dimension to obtain the feature concatenation tensor. ; Perform a 1×1 lightweight convolution operation on the spliced ​​feature tensor to integrate channel-dimensional features and output multi-scale fused features. As shown below: 。 4. The lightweight target detection and ranging method based on binocular vision according to claim 2, characterized in that, In step S23, the multi-scale fusion features are input into the channel blending module CML, and a reconstructed feature map is obtained through channel grouping, convolution, and dimensionality rearrangement operations, including: The multi-scale fused feature map The feature map is divided into G channel groups along the channel dimension, and the multi-scale fusion feature map is... Dimensional reshaping is performed to obtain the reshaped feature tensor. As shown below: Wherein, C / G is the number of channels in each channel group, and the multi-scale fusion feature map The dimension is Where B is the batch size, C is the total number of channels in the feature map, H is the feature map height, and W is the feature map width; For the reshaped feature tensor Each channel group in the process performs a 1×1 lightweight group convolution operation to complete the feature transformation within the group while keeping the feature tensor dimension unchanged. Perform channel dimension permutation on the feature tensor after group convolution. The permuted feature tensor is obtained. As shown below: ； Will Reshape into a feature map fused with the multi-scale feature map With consistent dimensions, the recombined feature map F is obtained, as shown below: 。 5. The lightweight target detection and ranging method based on binocular vision according to claim 2, characterized in that, In step S24, the reconstructed feature map is input into the spatial attention module SLA to enhance the target region features through spatial attention weights, including: For the recombined feature map F, perform maximum pooling and minimum pooling operations along the channel dimension to obtain the maximum feature map Max(F) and the minimum feature map Min(F); The maximum value feature map Max(F) and the minimum value feature map Min(F) are concatenated along the channel dimension to obtain the concatenated feature map [Max(F), Min(F)]. A 1×1 lightweight convolution is performed on the stitched feature map, followed by a Sigmoid activation function. Generate spatial attention weight map As shown below: 。 6. The lightweight target detection and ranging method based on binocular vision according to claim 1, characterized in that, In step S3, a region-adaptive progressive post-processing is performed on the initial disparity map, further including: S31: Perform median filtering with an adaptive window size on the initial disparity map and isolate noise points to obtain a first intermediate disparity map; S32: The edge information of the first intermediate disparity map is extracted using the Canny edge detection algorithm. The gradient threshold is dynamically adjusted based on the global gradient mean of the first intermediate disparity map to generate a binarized edge region mask. S33: Within the edge region indicated by the edge region mask, the first intermediate disparity map is sharpened using an unsharpened mask technique to obtain a second intermediate disparity map; S34: Using a pixel-level conditional fusion strategy, the sharpening result of the second intermediate disparity map is retained in the edge region, and the median filtering result of the first intermediate disparity map is retained in the non-edge region to obtain the third intermediate disparity map; S35: Identify the invalid hole regions generated by stereo matching failure in the third intermediate disparity map, first fill the small holes with morphological closing operations, and then fill the remaining large holes with a region consistency strategy based on adjacent valid disparity values ​​to obtain the optimized high-quality disparity map.

7. The lightweight target detection and ranging method based on binocular vision according to claim 1, characterized in that, In step S4, the three-dimensional spatial distance of the target within the two-dimensional detection frame is calculated, including: S41: Obtain the optimized high-quality disparity map, and establish a mapping relationship between disparity values ​​and physical distance based on the pre-calibration parameters of the binocular camera. The pre-calibration parameters include baseline distance, camera focal length, intrinsic parameter matrix, and extrinsic parameter matrix. S42: For each of the two-dimensional detection boxes, within the corresponding region of the optimized high-quality disparity map, filter effective disparity values ​​and calculate the average effective disparity value; S43: Based on the mapping relationship, convert the mean effective disparity value into a depth value in three-dimensional space; S44: Based on the depth value and the center pixel coordinates of the two-dimensional detection box, calculate the three-dimensional spatial position of the target in the camera coordinate system, and output the final perception result including the target category, the two-dimensional bounding box and its corresponding three-dimensional distance.

8. The lightweight target detection and ranging method based on binocular vision according to claim 1, characterized in that, In step S1, the left and right views captured by the binocular cameras are obtained, including: S11: Initialize the binocular camera system and perform stereo correction on the left and right cameras to ensure that their imaging planes are parallel and aligned. S12: Simultaneously trigger the left and right cameras to capture the initial left and initial right views of the scene at the same timestamp; S13: Perform preprocessing on the initial left view and the initial right view, including denoising, brightness equalization and geometric alignment, to obtain the left view and the right view.

9. A computer device comprising a memory and one or more processors, the memory storing computer code that, when executed by the one or more processors, causes the one or more processors to perform the steps of a lightweight target detection and ranging method based on binocular vision as described in any one of claims 1-8.

10. A storage medium storing computer-readable instructions, characterized in that, When the computer-readable instructions are executed by one or more processors, the one or more processors perform the steps of a lightweight target detection and ranging method based on binocular vision as described in any one of claims 1 to 8.

Images