Three-dimensional target detection algorithm system based on single-purpose traffic monitoring

Abstract

The invention discloses a single-purpose traffic monitoring-based three-dimensional target detection algorithm system, which comprises a monocular image acquisition module, a data processing module, a feature extraction module, a depth estimation module, a three-dimensional detection module, a result output module and a multi-scale feature pyramid structure, by setting a multi-scale feature pyramid structure and a multi-scale space attention mechanism in a three-dimensional detection module, the problem that targets of different scales are difficult to accurately detect in existing traffic monitoring is solved by fusing features of different scales and focusing traffic targets of different scales; therefore, the detection capability of traffic targets of various scales is improved, and the problem of insufficient real-time performance of a traffic object detection algorithm in the prior art is solved by adopting a model lightweight design and algorithm acceleration strategy mode through setting a real-time performance optimization module. Therefore, the real-time requirement is met when the system processes a large amount of traffic monitoring data.

Description

Technical Field

[0001] The present invention relates to the field of visual technology, and in particular to a three-dimensional target detection algorithm system based on single-purpose traffic monitoring. Background Art

[0002] Traditional traffic monitoring systems mainly rely on two-dimensional image processing, which can only provide limited information, thus limiting the accurate identification and stereoscopic perception capabilities of traffic targets. To solve this problem, in recent years, researchers have begun to integrate 3D target detection technology in the field of computer vision into roadside monitoring viewpoints to provide more comprehensive and accurate traffic target information. However, this task faces some challenges and difficulties. First, the cameras in the roadside monitoring system have different installation heights, pitch angles and complex road environments. Due to the limitation of the single camera viewpoint, the camera can only provide a limited field of view, resulting in the target being blocked or the viewpoint changing, thereby increasing the complexity of object detection. Secondly, due to the lack of accurate depth information in the image, it is difficult to obtain the true size and shape of the target by relying solely on 2D images for target detection, which affects precise positioning and posture estimation. In addition, environmental factors such as lighting changes and weather conditions can also affect the accuracy of single-camera 3D object detection.

[0003] Therefore, a three-dimensional target detection algorithm system based on single-purpose traffic monitoring is proposed. Summary of the invention

[0004] In response to the above problems, the present invention provides a three-dimensional target detection algorithm system based on single-purpose traffic monitoring to solve the problems of target occlusion, viewpoint change, depth information loss, environmental interference and high real-time performance requirements existing in the existing traffic monitoring system.

[0005] The technical solution of the present invention is:

[0006] A three-dimensional target detection algorithm system based on monocular traffic monitoring includes a monocular image acquisition module, a data processing module, a feature extraction module, a depth estimation module, a three-dimensional detection module, a result output module, a multi-scale feature pyramid structure, a real-time performance optimization module and an environmental adaptability enhancement module.

[0007] The working principle of the above technical solution is as follows:

[0008] The monocular image acquisition module collects image information of traffic scenes. The data processing module performs preprocessing such as denoising, graying, and resizing on the images to improve the efficiency and accuracy of subsequent processing. The feature extraction module uses convolutional neural networks to extract the two-dimensional features of the target object from the preprocessed images, providing a basis for subsequent three-dimensional detection. The depth estimation module combines adaptive depth interval division and instance-level visual clues, and realizes depth information estimation of the target object through global scene analysis and local feature representation. The three-dimensional detection module detects and generates a three-dimensional target bounding box based on the multi-scale spatial attention mechanism and depth information, combined with the 2D bounding box of the traffic object, the projected corner key points, and the offset vector relative to the center of the 2D bounding box. The result output module outputs the three-dimensional target detection results, including the three-dimensional position, size, direction and other information of the target, and facilitates real-time monitoring and decision-making of traffic management personnel through the visualization display function. The multi-scale feature pyramid structure enhances the feature significance of targets of different scales by fusing features of different scales. The real-time performance optimization module optimizes the real-time performance of the system by adopting model lightweight design and algorithm acceleration strategies. The environmental adaptability enhancement module improves the adaptability of the system to environmental factors by introducing algorithms such as adaptive brightness adjustment and color correction.

[0009] In a further technical solution, the monocular image acquisition module adopts a specific image sensor and installation method, which can adapt to different traffic monitoring scenarios and collect high-quality traffic scene image information.

[0010] The above technical solution solves the technical problem of unstable quality of traffic monitoring image acquisition in the prior art.

[0011] In a further technical solution, the image denoising algorithm in the data processing module adopts an innovative filtering method, which can effectively remove noise from traffic monitoring images while retaining the detailed information of the image.

[0012] The above technical solution solves the technical problem of poor image denoising effect in the prior art.

[0013] In a further technical solution, the convolutional neural network structure in the feature extraction module is optimized and designed to have higher feature extraction capability and efficiency, and can accurately extract the two-dimensional features of traffic targets.

[0014] The above technical solution solves the technical problem of inaccurate two-dimensional feature extraction in the prior art.

[0015] In a further technical solution, the adaptive depth interval division method in the depth estimation module is based on a unique algorithm model, which can automatically adjust the depth interval according to different traffic scenarios to improve the accuracy of depth estimation.

[0016] The above technical solution solves the technical problem of inaccurate depth estimation in the prior art.

[0017] In further technical solutions, the multi-scale spatial attention mechanism in the three-dimensional detection module adopts a novel computational method, which can better focus on traffic targets of different scales and improve the accuracy of three-dimensional detection.

[0018] The above technical solution solves the technical problem of low three-dimensional detection accuracy in the prior art.

[0019] In further technical solutions, the visualization function in the result output module uses advanced graphics processing technology to intuitively present the three-dimensional target detection results, making it easier for traffic management personnel to conduct real-time monitoring and decision-making.

[0020] The above technical solution solves the technical problem of non-intuitive display of detection results in the prior art.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] By setting up a multi-scale feature pyramid structure and a multi-scale spatial attention mechanism in the three-dimensional detection module, the problem of difficulty in accurately detecting targets of different scales in existing traffic monitoring is solved by fusing features of different scales and focusing on traffic targets of different scales, thereby achieving the benefit of improving the detection capability of traffic targets of various scales. By setting up a real-time performance optimization module, the problem of insufficient real-time performance of traffic object detection algorithms in existing technologies is solved by adopting model lightweight design and algorithm acceleration strategies, thereby achieving the benefit of ensuring that the system meets real-time requirements when processing large amounts of traffic monitoring data. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is the overall network framework flow chart of the present invention;

[0024] Figure 2 The task branch of the present invention generates different depth interval flow charts for different visual scenes;

[0025] Figure 3 It is the flow chart of the MSPE module of the present invention. DETAILED DESCRIPTION

[0026] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0027] In the description of the present invention, it should be noted that the terms "upper", "lower", "inner", "outer", "front end", "rear end", "two ends", "one end", "the other end" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0028] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0029] Example:

[0030] See also Figure 1 -3, a three-dimensional target detection algorithm system based on monocular traffic monitoring, including a monocular image acquisition module, a data processing module, a feature extraction module, a depth estimation module, a three-dimensional detection module, a result output module, a multi-scale feature pyramid structure, a real-time performance optimization module and an environmental adaptability enhancement module.

[0031] The working principle of the above technical solution is as follows:

[0032] During operation, the monocular image acquisition module collects image information of the traffic scene and transmits it to the data processing module. After the data processing module performs pre-processing such as denoising, graying, and resizing on the image, the processed image is transmitted to the feature extraction module. The feature extraction module uses a convolutional neural network to extract the two-dimensional features of the target object, and transmits the two-dimensional features to the depth estimation module and the three-dimensional detection module. The depth estimation module combines adaptive depth interval division and instance-level visual clues to estimate the depth information of the target object, and transmits the depth information to the three-dimensional detection module. The three-dimensional detection module is based on a multi-scale spatial attention mechanism and depth information, combined with the 2D bounding box, The projected corner key points and the offset vector relative to the center of the 2D bounding box are used to detect and generate the 3D target bounding box. The result output module outputs the 3D target detection results, including the 3D position, size, direction and other information of the target, and displays them to traffic management personnel through a visualization function. The multi-scale feature pyramid structure enhances the feature significance of targets of different scales by fusing features of different scales, and improves the detection capability of the model. The real-time performance optimization module optimizes the real-time performance of the system by adopting model lightweight design and algorithm acceleration strategies. The environmental adaptability enhancement module improves the system's adaptability to environmental factors by introducing algorithms such as adaptive brightness adjustment and color correction.

[0033] like Figure 2 As shown in Figure 1, the adaptive instance depth estimation branch considers the distribution of objects in the entire scene by generating different depth intervals for different visual scenes, rather than relying solely on the visual cues of a single local target. In contrast, the method based on adaptive depth interval division can better perceive the changes in target scale in the global scene and promote adaptive learning. Its basic idea is to use the linear combination of the probabilistic representation of instance-level local features and the adaptive center of the scene-level depth interval as the final depth prediction value. The entire branch design mainly involves three steps: (1) adaptive depth interval width estimation; (2) instance-level visual cue probability distribution generation; (3) instance depth value calculation, as shown in Figure 1. Figure 3 As shown in the figure, in order to enhance the spatial perception of targets at different scales, a multi-scale spatial perception attention module is proposed to capture the dense pixel relationship in the local feature representation while ensuring the high performance and lightweight computation of the 3D detection head. The pyramid pooling layer is used to extract feature attention maps of different scales, and the original features are combined with the contextual information to provide an overall contextual field of view perception. The intermediate features of the adaptive instance depth estimation branch are integrated into the module to improve the stability and relevance of the learning of the 3D detection task branch and the scene-level adaptive instance depth estimation task branch.

[0034] The monocular image acquisition module adopts a specific image sensor and installation method, which can adapt to different traffic monitoring scenarios and collect high-quality traffic scene image information. The overall network framework includes a backbone network and three branches: 2D detection task branch, adaptive instance depth estimation task branch and 3D detection task branch. The image denoising algorithm in the data processing module adopts an innovative filtering method, which can effectively remove the noise in the traffic monitoring image while retaining the image detail information. The convolutional neural network structure in the feature extraction module has been optimized and designed to have higher feature extraction ability and efficiency, and can accurately extract the two-dimensional features of traffic targets. The adaptive depth interval division method in the depth estimation module is based on a unique algorithm model, which can automatically adjust the depth interval according to different traffic scenarios and improve the accuracy of depth estimation. The multi-scale spatial attention mechanism in the three-dimensional detection module adopts a novel calculation method, which can better focus on traffic targets of different scales and improve the accuracy of three-dimensional detection. The visualization function in the result output module adopts advanced graphics processing technology, which can intuitively present the three-dimensional target detection results, which is convenient for traffic management personnel to conduct real-time monitoring and decision-making. When working, each module works together to achieve accurate detection and real-time monitoring of three-dimensional targets in traffic scenes, providing strong support for traffic management and safety.

[0035] In the ablation experiment, the DLA-34 network is used as the backbone network for feature extraction. The backbone network extracts 4 downsampled feature maps with a total of 64 channels. The resolution of the input image is set to 384×1280, and the downsampling ratio of the feature map is set to 4. In order to improve the generalization of the model, various data enhancement strategies are applied during the training process, including random flipping, random cropping, and color jittering. The PyTorch framework is used to implement the method proposed in this section, and the model proposed in this section is trained on a single NVIDIA Tesla V100 GPU. The batch size is set to 16 on the KITTI dataset, and 200 epochs are trained. In order to analyze the contribution of different sub-modules and different design choices in this framework, an ablation study is performed on the car category on the KITTI validation set.

[0036] Ablation analysis of the adaptive instance depth estimation branch:

[0037] In the first group of experiments, a center-guided depth regression strategy, namely the baseline model MonoCon, was used as a comparative experiment in this section. In the second group of experiments, a combination of center-guided depth regression and visual height projection depth estimation was used as the final depth inference. Experimental data showed that due to the non-uniqueness of the physical height itself, the performance was reduced compared with the baseline model. In the third group of experiments, the center-guided depth regression was combined with the proposed depth estimation strategy branch as the final depth inference. The experimental results showed that the overall performance was better than the baseline model. In the fourth group of experiments, the above three depth estimation strategies were combined. The experimental results showed that the average method of the three depth estimation methods had the best effect.

[0038] Ablation experiment analysis of multi-scale spatial perception attention module: In order to verify the effectiveness of the proposed multi-scale spatial perception attention (MSPE) module and the rationality of the internal design structure, this section conducts relevant internal ablation experiments in Table 3. Specifically, experiments are conducted on adding MSPE modules and removing pyramid pooling (PP) operations in the MSPE modules based on the baseline model. From the results, it can be observed that after the MSPE module is fully added, the performance of the baseline model is significantly improved. In addition, when the pyramid pooling operation is removed, the performance decreases slightly. These experimental results confirm the rationality and effectiveness of the module design, which also proves that the internal pyramid pooling operation can effectively improve the model's multi-scale target visual perception ability, thereby focusing on local visual depth clues.

[0039] Table 1. Comparison of ablation experiments with different depth intervals

[0040]

[0041] Table 2 Ablation experiment comparison of different depth combination strategies

[0042]

[0043] Table 3 Internal ablation experiment of multi-scale spatial perception attention module (MSPE)

[0044]

[0045] The above-mentioned embodiments only express the specific implementation of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A three-dimensional target detection algorithm system based on single-purpose traffic monitoring, characterized in that: include: Monocular image acquisition module, used to collect image information of traffic scenes; A data processing module is used to receive and pre-process traffic monitoring video or image data, including image denoising, grayscale, size adjustment and other pre-processing steps to improve the efficiency and accuracy of subsequent processing; Feature extraction module, which is a feature extraction module based on convolutional neural network (CNN) and is used to extract the two-dimensional features of the target object from the preprocessed image, including edge, texture, shape and other information, providing a basis for subsequent three-dimensional detection; Depth estimation module: This module combines adaptive depth interval division and instance-level visual cues to estimate the depth information of the target object through global scene analysis and local feature representation, solving the problem of lack of accurate depth information in the image; 3D detection module: This module is a 3D detection module based on multi-scale spatial attention mechanism and depth information. It uses the extracted 2D features and depth information, combined with the 2D bounding box of the traffic object, the projected corner key points and the offset vector relative to the center of the 2D bounding box, to detect and generate the 3D target bounding box; Result output module, which is used to output the 3D target detection results, including the target's 3D position, size, direction and other information, as well as a visualization function, to facilitate real-time monitoring and decision-making by traffic management personnel; Multi-scale feature pyramid structure, which is used to enhance the saliency of features of targets at different scales. By fusing features at different scales, the model’s ability to detect targets of various scales in traffic monitoring scenarios is improved. Real-time performance optimization module, which is used to optimize the real-time performance of the system, including model lightweight design, algorithm acceleration strategy, etc., to ensure that the system can meet the real-time requirements when processing a large amount of traffic monitoring data; Environmental adaptability enhancement module: This module is used to improve the system's adaptability to environmental factors such as lighting changes and weather conditions. By introducing algorithms such as adaptive brightness adjustment and color correction, it ensures that the system can work stably in various complex environments.

2. According to claim 1, a three-dimensional target detection algorithm system based on single-purpose traffic monitoring is characterized in that: The monocular image acquisition module adopts a specific image sensor and installation method, can adapt to different traffic monitoring scenes, and collect high-quality traffic scene image information.

3. The three-dimensional target detection algorithm system based on single-purpose traffic monitoring according to claim 1 is characterized in that: The image denoising algorithm in the data processing module adopts an innovative filtering method, which can effectively remove noise in traffic monitoring images while retaining the detailed information of the images.

4. The three-dimensional target detection algorithm system based on single-purpose traffic monitoring according to claim 1 is characterized in that: The convolutional neural network structure in the feature extraction module is optimized and designed to have higher feature extraction capability and efficiency, and can accurately extract the two-dimensional features of traffic targets.

5. The three-dimensional target detection algorithm system based on single-purpose traffic monitoring according to claim 1 is characterized in that: The adaptive depth interval division method in the depth estimation module is based on a unique algorithm model, which can automatically adjust the depth interval according to different traffic scenes and improve the accuracy of depth estimation.

6. The three-dimensional target detection algorithm system based on single-purpose traffic monitoring according to claim 1 is characterized in that: The multi-scale spatial attention mechanism in the three-dimensional detection module adopts a novel calculation method, which can better focus on traffic targets of different scales and improve the accuracy of three-dimensional detection.

7. The three-dimensional target detection algorithm system based on single-purpose traffic monitoring according to claim 1 is characterized in that: The visualization function in the result output module adopts advanced graphics processing technology, which can intuitively present the three-dimensional target detection results, making it easier for traffic management personnel to conduct real-time monitoring and decision-making.

Images