A method and system for target detection in complex traffic scenes based on deep learning

Abstract

The present invention discloses a method and system for detecting targets in complex traffic scenes based on deep learning, which relates to the technical field of image processing for autonomous driving scenes. The method and system include obtaining and synchronously organizing image-collected data from all perspectives of the driving vehicle; initially labeling the sub-objects within each group of synchronized image data; performing boundary labeling comparisons on targets of the same category, obtaining labeling differences and screening out standard data sets; and performing traffic scene target detection using the standard data sets and a convolutional neural network model. The method and system synchronously organize image data obtained from multiple perspectives of the vehicle, then determine the degree of difference after labeling the images from each perspective, and use image data that meet the required degree of difference as a data set for convolutional neural network learning and training, thereby reducing the error caused by collecting labeled data from a single perspective for learning and training, and ensuring the accuracy of target learning and subsequent judgment accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving scene image processing, and in particular to a method and system for target detection in complex traffic scenes based on deep learning. Background Art

[0002] Object detection is a fundamental and important task in computer vision. It determines the class of the detected object and locates its location within a given image, and has broad application prospects. Autonomous vehicles must constantly monitor their driving status and interact with the external environment to improve safety. However, when performing obstacle detection in complex road scenarios, vehicle targets are susceptible to interference from scale variations between different object instances and occlusion by other traffic objects. This can lead to inconsistent feature attributes extracted by the model, especially for target objects and small-scale objects, resulting in missed and false detections.

[0003] In the existing technology, the above-mentioned problems are generally addressed by considering at least one of the following application aspects: multi-scale feature detection, data enhancement, attention mechanism, and optimization for specific scenarios. This can alleviate the above-mentioned problems to a certain extent and improve the accuracy and robustness of vehicle obstacle detection. However, based on the aforementioned application aspects, there are still certain false detections and missed detections, and the improvement brought about by simply improving the convolutional network neural algorithm is becoming increasingly limited. From other application scenarios, some situations have emerged where the input data set of the convolutional network neural algorithm is optimized and screened. The results show that this can enable the algorithm model to achieve higher detection accuracy, thereby achieving the goal of high-precision detection.

[0004] As mentioned above, to achieve more reliable and accurate visual detection in autonomous driving scenarios, sensory data captured by onboard devices, such as image data, must be further processed to ensure that trained models can more accurately identify and learn relevant content from the sensory data, thereby meeting driving safety requirements. However, existing technologies for processing onboard sensory data generally directly label the data acquired by a single sensor. This single "perspective" data can contain significant errors, which can mislead subsequent model learning and judgment, reducing target detection accuracy.

[0005] In view of this, this application is hereby filed. Summary of the Invention

[0006] The purpose of the present invention is to provide a method and system for target detection in complex traffic scenes based on deep learning. The detection method and system use image data obtained from multiple perspectives of a vehicle for synchronous sorting, and then judge the degree of difference after the images of each perspective are annotated. The image data that meets the requirements of the degree of difference is used as the data set for convolutional neural network learning and training, thereby reducing the error caused by collecting annotated data from a single perspective for learning and training, and ensuring the accuracy of target learning and subsequent judgment accuracy.

[0007] The embodiment of the present invention is achieved as follows:

[0008] In a first aspect, a method for detecting targets in complex traffic scenes based on deep learning comprises the following steps: acquiring image-captured data from all perspectives of a driving vehicle, synchronizing and organizing all the image-captured data using timestamp information to obtain multiple groups of synchronized image data sets; performing initial annotation on sub-objects within each group of the synchronized image data sets, wherein the initial annotation refers to boundary and category annotation processing for different targets within the image; performing boundary annotation comparison on targets of the same category in each sub-object of the same synchronized image data set to obtain annotation difference, and using the synchronized image data set with the annotation difference not greater than a preset value as a standard data set; performing traffic scene target detection using the standard data set and based on a convolutional neural network model; wherein the boundary annotation processing includes key point annotation and contour annotation, constructing a feature point coordinate comparison table using the key point annotation information and contour annotation information, and calculating the annotation difference based on the feature point coordinate comparison table.

[0009] In some optional implementations, the construction of a feature point coordinate comparison table using key point annotation information and contour annotation information includes the following steps: obtaining a key point coordinate group and a contour coordinate group of the same category of targets in each sub-object, and determining typical feature point coordinates in the key point coordinate group and the contour coordinate group; matching and calibrating the typical feature point coordinates of the same category of targets in different sub-objects to obtain a calibrated coordinate layer group; setting a layer of the calibrated coordinate layer group as a base layer, and setting the remaining layers as sub-layers; matching and marking the coordinate points on each sub-layer with the corresponding coordinate points on the base layer in turn, and using the matching marking results to construct the feature point coordinate comparison table; wherein, the coordinate points include key point coordinates and contour coordinates.

[0010] In some optional implementations, the calculation of the annotation difference based on the feature point coordinate comparison table includes the following steps: calculating all difference values ​​of each coordinate point in the feature point coordinate comparison table, combining all difference values ​​to obtain the total difference value of the coordinate point; using all the total difference values ​​to calculate the annotation difference; wherein the difference value refers to the difference distance between the sub-layer coordinate point and the corresponding base layer coordinate point.

[0011] In some optional implementations, after calculating all the difference values ​​of each coordinate point in the feature point coordinate comparison table, the following steps are also included: correcting the data of the sub-layer coordinate point based on the radar detection data to obtain the corrected sub-layer coordinate point, and then using the corrected sub-layer coordinate point and the corresponding base layer coordinate point to calculate the difference distance.

[0012] In some optional implementations, determining the coordinates of typical feature points in the key point coordinate group and the contour coordinate group includes the following steps: judging at least one special pixel point from the key point coordinate group and the contour coordinate group, constructing a special feature area using the associated special pixel points, and obtaining multiple feature area points, where the feature area points refer to coordinate points that can reflect the typical features of the target; screening the feature area points based on the labeling rules to obtain the coordinates of typical feature points.

[0013] In some optional implementations, the determining of at least one special pixel point from the key point coordinate group and the contour coordinate group specifically includes the following steps: obtaining a target annotation type, the annotation type including at least one of occlusion annotation, small target annotation, and living object annotation; defining an annotation range according to the annotation type, and selecting at least some of the pixel points of the key point coordinate group and the contour coordinate group within the defined annotation range as the special pixel points.

[0014] In some optional implementations, when the annotation type is occlusion annotation, the annotation range needs to delineate the visible part and the invisible part; when the annotation type is small target annotation, the annotation range needs to delineate the target object part and the background circle part; when the annotation type is living object annotation, the annotation range needs to delineate the moving part and the non-moving part; wherein, when the annotation type is a combination of occlusion annotation and living object annotation, the annotation range needs to at least delineate the moving part in the non-visible part.

[0015] In some optional implementations, the annotation type also includes abnormal annotation. When the annotation type is abnormal annotation, the annotation range needs to delineate the normal recognition part and the abnormal recognition part; wherein, when the annotation type is a combination of occlusion annotation and abnormal annotation, the annotation range needs to at least delineate the normal recognition part in the non-visible part.

[0016] In some optional implementations, the marking range needs to be delineated into a normal recognition part and an abnormal recognition part and also includes the following steps: determining the image background type of the abnormal recognition part, where the image background type refers to the category of the framing environment in which the image is taken; calculating the influence factor of the image background type on target recognition, and adjusting the abnormal recognition part corresponding to the situation where the influence factor is greater than the threshold value to a normal recognition part.

[0017] The second aspect is a deep learning-based target detection system for complex traffic scenes, including:

[0018] a first acquisition unit configured to acquire image-collected data from all viewing angles of the driving vehicle, and to synchronize and organize all the image-collected data using timestamp information to obtain multiple sets of synchronized image data;

[0019] a first processing unit, configured to perform initial labeling on sub-objects of each group of the synchronized image data sets, wherein the initial labeling refers to performing boundary and category labeling processing on different targets in the image;

[0020] A first calculation unit is configured to perform boundary annotation comparison on targets of the same category in each sub-object of the same synchronized image data group to obtain a annotation difference, and use the synchronized image data group with the annotation difference not greater than a preset value as a standard data set; wherein the boundary annotation processing includes key point annotation and contour annotation, and a feature point coordinate comparison table is constructed using the key point annotation information and the contour annotation information, and the annotation difference is calculated based on the feature point coordinate comparison table.

[0021] The second processing unit is used to use the standard data set and perform traffic scene target detection based on a convolutional neural network model.

[0022] The beneficial effects of the embodiments of the present invention are:

[0023] The deep learning-based target detection method and system for complex traffic scenes provided by the embodiment of the present invention synchronizes and organizes the image collection data obtained from all perspectives of the driving vehicle, and uses timestamp information to organize the image collection data from all perspectives at the same time into the same synchronized image data group; then, initial annotation is performed on the image collection data from all perspectives in the synchronized image data group, and the boundaries and categories are independently annotated and then the annotated information is checked and compared. By comparing the boundary annotations of targets of the same category, the obtained annotation difference results are used to determine whether the image collection data obtained from each perspective have a high degree of similarity and correlation, eliminating the problem of accidental factors in obtaining image data from only a single perspective. This is equivalent to a proofreading and review of the synchronized image data group, thereby seating the synchronized image data group that meets the preset requirements into a standard data set, so that the convolutional network neural model can obtain more accurate and fitting training set and test set data, thereby ensuring the accuracy and precision of learning and training and subsequent judgment and detection.

[0024] In general, the deep learning-based complex traffic scene target detection method and system provided by the embodiments of the present invention, based on the use of convolutional neural networks to perform complex traffic scene target detection, considers the optimization and screening of input data sets, can reduce the situation of invalid data and erroneous data, and at the same time adopts multi-perspective comparison and review methods for the acquired data to further ensure the accuracy and matching of the data, thereby ensuring high-quality image data at the input end of the convolutional neural network model, which is used for model learning and training to meet the requirements of high-precision target recognition and detection in complex traffic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 A flowchart of the main steps of the detection method provided by an embodiment of the present invention;

[0027] Figure 2 for Figure 1 The main steps shown are one of which, step S300, is a flow chart of an embodiment;

[0028] Figure 3 for Figure 1 One of the main steps shown is step S300 in a process flow according to another embodiment;

[0029] Figure 4 A flowchart of sub-steps of step S300 provided in one embodiment of the present invention;

[0030] Figure 5 A modular schematic diagram of a detection system provided in an embodiment of the present invention.

[0031] Icon: 500 - detection system; 510 - first acquisition unit; 520 - first processing unit; 530 - first calculation unit; 540 - second processing unit. DETAILED DESCRIPTION

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0033] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0034] It should be understood that the terms "system," "device," and / or "module" used herein are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0035] As used herein and in the claims, unless the context clearly indicates otherwise, the terms "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list; a method or apparatus may also include additional steps or elements.

[0036] Flowcharts are used in the present invention to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes. Example

[0037] While navigating traffic, autonomous vehicles need to constantly identify various objects in their surroundings to accurately identify and plan driving routes and maneuvers. Because autonomous vehicles need to constantly monitor their driving status and interact with the external environment, image processing methods are essential for more accurate environmental perception. Currently, we are applying and designing novel lightweight perception architecture models and object detection algorithms to enable autonomous driving technology to perceive and understand high-resolution road images on a more portable and user-friendly processing platform, enhancing intelligent vehicles' ability to make informed decisions and assist in complex traffic scenarios. However, during implementation, we considered improvements and optimizations in multiple application areas, including multi-scale feature detection, data augmentation, attention mechanisms, and scenario-specific optimization. While this approach significantly improved object detection accuracy, further significant improvements proved challenging. Given the high safety standards associated with autonomous driving, even higher levels of recognition and detection accuracy are required.

[0038] Our research has found that in other intelligent recognition fields, we have begun to pre-process the collected image data. This is to ensure that the image data is usable on the one hand, and that the image data content can be accurately and effectively identified on the other hand. Therefore, we need to consider the image data pre-processing method. In addition to pixel enhancement of image data, accurate image labeling and classification are also required for complex traffic scenes. Traditional labeling methods only label individual images without considering whether the individual image can be used as a more suitable learning and training object for subsequent convolutional neural network models, which indirectly reduces the detection accuracy of the trained convolutional neural network model. Therefore, in response to the aforementioned problems, we considered the verification and verification of image labeling to ensure that the collected image data has higher adaptability and accuracy.

[0039] Please refer to the following for details: Figure 1 , this embodiment provides a method for detecting targets in complex traffic scenes based on deep learning, comprising the following steps:

[0040] S100: Acquire image-collected data from all viewing angles of the driving vehicle, and synchronize and organize all of the image-collected data using timestamp information to obtain multiple sets of synchronized image data. This step involves acquiring images captured by cameras installed at multiple locations on the driving vehicle to obtain image-collected data from different viewing angles. It should be noted that the camera viewing angles at different locations have overlapping ranges, and these images acquired from these overlapping viewing angles are subsequently processed to determine the target within the image. Each image has timestamp information, and all of the image-collected data are synchronized and organized using the same timestamp information to obtain multiple sets of synchronized image data. The synchronized image data sets obtained in this manner are multi-view image data sets captured at the same time, thereby facilitating the determination of the target content in the driving vehicle's surrounding environment at the time of capture from multiple perspectives.

[0041] S200: Initially labeling the sub-objects within each synchronized image data set. This initial labeling involves labeling the boundaries and categories of the different objects within the image. This step involves initially labeling each image (i.e., the sub-objects) within the synchronized image data set. The target content within the image must be labeled with its boundaries and its category. It should be noted that initial labeling here refers to at least labeling the boundaries and categories of the different objects within the image. Other implementations may also include attribute labeling, relationship labeling, quality labeling, and other methods.

[0042] After the initial labeling in the above step S200, the labeling information of each sub-object in the synchronized image data group can be obtained. In order to avoid accidental errors caused by single-view labeling, multi-view labeling is adopted and labeling comparison is performed to determine whether the image information after labeling each sub-object is highly correlated and similar, so as to judge whether the synchronized image data group is a qualified data set and can be used for subsequent model learning and training.

[0043] Specifically, S300: in each sub-object of the same synchronized image data set, the boundary annotations of the targets of the same category are compared to obtain the annotation difference, and the synchronized image data set with the annotation difference not greater than the preset value is used as the standard data set; this step means selecting targets of the same category from each sub-object (such as the targets of the same category can be selected from the images with overlapping perspectives as described in step S100), performing a difference comparison on the boundary annotations of the targets of the same category, thereby obtaining the compared annotation difference. If the annotation difference obtained from the boundary annotations of the targets of the same category selected from multiple perspectives is not greater than the preset value, the corresponding synchronized image data set is used as the standard data set, can be used as a qualified data set and can be used for subsequent model learning and training. It should be noted that if there are multiple targets of the same category in each sub-object of the same synchronized image data set, then if the annotation difference obtained for any one of the targets of the same category is not greater than the preset value, the corresponding synchronized image data set is used as the standard data set, which means that this group of synchronized image data sets has the value of model learning and training.

[0044] In step S300, considering that boundary annotation comparison is based on the annotation boundary information of the same object, a composite annotation method is required to obtain a more reliable comparison result, thereby avoiding errors in the comparison result caused by a single or simplistic annotation method. Specifically, in this embodiment, boundary annotation processing includes key point annotation and contour annotation. The key point annotation information and contour annotation information are used to construct a feature point coordinate comparison table. This means that boundary annotation is performed by comparing the key point annotation information of the object with the contour annotation information of the object. Then, a feature point coordinate comparison table is constructed based on the comparison results. For example, the vertical heading of the feature point coordinate comparison table indicates the object to which the corresponding object belongs in each sub-object, the horizontal heading indicates the coordinate point type, and the remaining portion contains the coordinate point data (pixel point or pixel area location). Finally, the annotation difference is calculated based on the feature point coordinate comparison table. This difference is obtained based on the difference calculation results of each column of coordinate point data.

[0045] Through step S300, multiple coordinate points (key coordinate points and contour coordinate points) can be used for similarity comparison to determine the similarity of target labels of the same category in each sub-object, so that the synchronized image data group that meets the similarity requirements is used as the standard data set for subsequent model learning and training steps.

[0046] S400: Utilize the standard data set and perform traffic scene target detection based on the convolutional neural network model; this step means using the standard data set obtained above as the training set and test set of input data to train and optimize the convolutional neural network model, so that the obtained convolutional neural network model can be more accurately applied to traffic scene target detection and obtain more accurate and matching target detection results. In this embodiment, the convolutional neural network model can adopt a single-stage target detection algorithm. The single-stage target detection algorithm treats target detection as a regression problem and does not use a candidate region generation network. Compared with the two-stage target detection algorithm, the network model of the single-stage target detection algorithm has fewer parameters and computational complexity, and has a faster detection speed, which is more suitable for rapid judgment target detection in such traffic driving scenarios.

[0047] Taking into account that the detection accuracy of the single-stage target detection algorithm is slightly inferior to that of the two-stage target detection algorithm, and the number of parameters of the algorithm model will cause the model to occupy a large storage space during the training process, and the large amount of calculation will cause the forward reasoning time of the algorithm model to be longer, making the algorithm overly dependent on computer hardware resources, making it difficult to use directly in hardware devices with limited storage resources and computing power. In order to address the problem that the target detection algorithm in vehicle driving scenarios occupies more system resources, the convolutional neural network model in this embodiment can adopt the lightweight target detection optimization algorithm of YOLOv5s. This algorithm can reduce the number of parameters of the model to a certain extent and improve the detection speed while maintaining a certain detection accuracy, thereby solving the problem that target detection in vehicle driving scenarios occupies more system resources. Specifically, the last three BottleneckCSP1 modules in the backbone network of the YOLOv5s algorithm model are first replaced with the corresponding GhostBottleneck modules, and the CBH module before the SPP module is replaced with a depthwise separable convolution, and an ECA attention module is added after the SPP module; secondly, based on the PANet network structure of the neck network, the ASFF adaptive spatial feature module is integrated; thirdly, CIoU Loss is used as the boundary regression loss function of the YOLOv5s algorithm model.

[0048] In general, the aforementioned algorithms have been improved and optimized in multiple application aspects, including multi-scale feature detection, data enhancement, attention mechanism, and optimization for specific scenarios. YOLOv5s is used as the basic framework, combined with the Ghost module, depthwise separable convolution, ECA attention module, and ASFF feature fusion network module. The CIoU Loss complete intersection-over-union loss function is used as the boundary frame regression loss function. This algorithm can reduce the number of model parameters to a certain extent while maintaining a certain level of accuracy, and the detection speed is also improved. It can be used in hardware devices with limited storage resources and computing power, and can effectively address the problem of target detection algorithms occupying more system resources in vehicle driving scenarios.

[0049] The above technical solution, by collecting data from various perspectives of the driving vehicle, can provide more comprehensive environmental perception capabilities. Timestamp synchronization ensures data consistency from different perspectives, helping to improve the accuracy of detection results. By labeling the boundaries and categories of different targets, the accuracy of target recognition is improved. Comparing the differences in the labels of targets of the same category from different perspectives ensures the quality of the labeled data and reduces the possibility of false detection and missed detection. On this basis, a comparison table is constructed using key point and contour information to more precisely capture the characteristics of the target and ensure the reliability of the data source. Then, by screening out synchronized image data sets with label differences no higher than a preset value, a more reliable standard dataset can be constructed for subsequent training of YOLOv5s' lightweight target detection optimization algorithm. This not only helps enhance the model's robustness and generalization capabilities in complex traffic scenarios, but also ensures a certain level of detection accuracy and speed, which is crucial for application scenarios such as autonomous driving.

[0050] Since the accuracy and matching degree of the input data of the entire convolutional neural network model affect the upper limit of the final detection result accuracy, it is necessary to consider the logical relationship between the two and consider a reasonable and easy-to-operate implementation mode when obtaining the annotation difference based on the feature point coordinate comparison table. Figure 2 In this embodiment, the construction of a feature point coordinate comparison table using key point annotation information and contour annotation information includes the following steps:

[0051] S310: Obtaining a key point coordinate group and a contour coordinate group for the same category of targets in each sub-object, and determining the coordinates of typical feature points in the key point coordinate group and contour coordinate group. This step represents the acquisition of all key point information and contour information for the same category of targets in each sub-object, obtaining the coordinate information of these key points and the points that make up the contour, and integrating this coordinate information into a key point coordinate group (the coordinates of multiple key points) and a contour coordinate group (the coordinates of multiple contour points), and determining representative typical feature point coordinates from these coordinates. It should be noted that the typical feature point coordinates may be determined from at least one of the key point coordinate group and the contour coordinate group, and the determined typical feature point coordinates thus represent the typical information of the category of targets.

[0052] S320: In different sub-objects, the coordinates of the typical feature points of the same category of targets are matched and calibrated to obtain a calibration coordinate layer group. This step indicates calibrating the coordinates of the typical feature points determined for the same category of targets in all sub-objects, that is, aligning and overlapping the corresponding typical feature points in different sub-objects to obtain overlapping matching layers, i.e., the calibration coordinate layer group, thereby unifying the multi-view data into a reference frame, which can effectively reduce the error accumulation caused by perspective changes. The calibration coordinate layer group includes the image data of each sub-object, and the coordinates of the typical feature points marked and determined in the image data of the same target in each sub-object form a one-to-one correspondence in the overlapping direction of the layers (because the coordinates of the typical feature points marked in different layers have different marking distances, this difference needs to be obtained through subsequent calculation. The one-to-one correspondence here requires approximate alignment. For example, the highest overlap of the largest part of the pixel coordinates indicates approximate alignment), thereby facilitating the calibration and comparison step for the typical feature point coordinates.

[0053] S330: Setting a layer in the calibration coordinate layer group as a base layer and the remaining layers as sublayers; this step indicates that the calibration coordinate layer group is obtained to determine a benchmark comparison reference, that is, a layer in the group is used as a base layer, and the remaining layers are used as objects for comparison with the base layer. It should be noted that the base layer setting rule can be randomly selected, but considering the reliability of the subsequent annotation difference results, the base layer setting rule can use the layer with the highest degree of alignment of typical feature point coordinates with the remaining layers (through pairwise comparison, find the layer with the highest degree of overlap of the largest portion of pixel point coordinate projections after comparison with the remaining layers) as the base layer, so that more reasonable comparison difference results can be obtained when comparing with the remaining layers.

[0054] S340: Sequentially match and mark the coordinate points on each sub-layer with the corresponding coordinate points on the base layer. The matching marking results are used to construct a feature point coordinate comparison table. The coordinate points include keypoint coordinates and outline coordinates. This step involves horizontally matching and marking the coordinate points (all coordinates, including keypoint coordinates and outline coordinates) on the base layer with the coordinate points at corresponding locations on the remaining layers (those that substantially overlap after alignment). This matching marking method records the coordinates of the relevant pixels or pixel regions to form the feature point coordinate comparison table. As described above, the vertical heading of this feature point coordinate comparison table indicates the object to which the corresponding object in each sub-object belongs, the horizontal heading indicates the coordinate point type, and the remaining portion contains coordinate point data (pixel or pixel region location). This matching marking method constructs a table that reflects the coordinate points included in the corresponding object in each layer, facilitating subsequent comparison and calculation. It should be noted that the method and number of coordinate points determined in each layer must be consistent (described in the subsequent solution) to facilitate the construction of a relatively standard feature point coordinate comparison table.

[0055] Through the above technical solution, the corresponding coordinate points of the remaining layers can be matched by calibrating the base layer, so that the constructed feature point coordinate comparison table is easier to operate in the subsequent comparison calculation, and the comparison result is more reasonable, avoiding the increase in the error of the final result due to excessive deviation or dispersion as the basis for difference comparison. In this embodiment, please refer to Figure 3 Calculating the annotation difference based on the feature point coordinate comparison table includes the following steps:

[0056] S350: Calculate all difference values ​​for each coordinate point in the feature point coordinate comparison table, and combine all difference values ​​to obtain the total difference value of the coordinate point; wherein, the difference value refers to the difference distance between the sub-layer coordinate point and the corresponding base layer coordinate point. This step indicates that, based on the feature point coordinate comparison table obtained above, for example, a difference distance comparison is performed for each longitudinal coordinate point, that is, the relative difference distance between the coordinate points of the remaining sub-layers and the base layer in the overlapping projection direction. The distance value is recorded as the difference value, which represents the degree of difference in the distance between the sub-layer coordinate point and the base layer coordinate point. If the difference value is larger, the farther the two are from each other, which means that the difference in the annotation of the same category of targets in the two sub-objects is greater. In addition to the error caused by the annotation factor, it means that the image data obtained from the two perspectives show a large inconsistency for the same target, and this group of synchronized image data may not meet the requirements of the standard data set.

[0057] After comparing the coordinate points of all sub-layers with the base layer, multiple difference values ​​can be obtained. After merging and calculating all the difference values ​​(by summing, product, or averaging, etc.), a total difference value is obtained. This total difference value represents the difference in the coordinate points of the same target presented in all perspective image data. If the total difference values ​​of all coordinate points are large, it means that the annotation difference is too large, and vice versa. The judgment step is as follows S360: Calculate the annotation difference degree using all the total difference values; this step means that the total difference values ​​obtained by the above comparison method for all coordinate points are merged again (by summing, product, averaging, or extreme value calculation, etc.) to obtain the annotation difference degree. If the annotation difference degree is not greater than the preset value, the corresponding synchronized image data set is used as the standard data set.

[0058] Based on the above technical solution, since the image acquisition data is affected by the perspective, the target object may appear larger when it is closer and smaller when it is farther away. At this time, this factor needs to be taken into account when restoring the target object (also in order to reduce the influence of this effect of larger when it is closer and smaller when it is farther away in the image data acquired from each perspective, and to restore the essential coordinates of the target as much as possible before comparison). The corresponding coordinate points in the sub-layer to be compared need to eliminate this influence as much as possible, so as to be compared with the coordinate points of the base layer as the benchmark. Therefore, in this embodiment, please refer to Figure 3 After calculating all the difference values ​​of each coordinate point in the feature point coordinate comparison table, the following steps are also included:

[0059] S351: Correcting the sub-layer coordinate point data based on the radar detection data to obtain corrected sub-layer coordinate points. The corrected sub-layer coordinate points are then used to calculate the difference distances with the corresponding base layer coordinate points. This step utilizes radar detection data other than image data to build a target model. The size of the target in the image is then corrected based on the actual target size detected during model building, thereby achieving pixel coordinate adjustment. Specifically, after correcting the sub-layer coordinate point data, the corrected sub-layer coordinate points are fine-tuned at the pixel level. The difference distances are then calculated using the corrected sub-layer coordinate points and the corresponding base layer coordinate points. It should be noted that in various implementations, the base layer coordinate points can also be fine-tuned using radar detection data. However, in this embodiment, considering both the computational complexity and the fact that the base layer coordinate point data itself serves as a comparison reference, the sub-layer coordinate points from other perspectives are more susceptible to the aforementioned object size variations caused by the near-sightedness and far-sightedness. Therefore, the coordinate point data of the remaining sub-layers, excluding the base layer, is corrected.

[0060] As described in the previous solution, the method and number of coordinate points in each layer need to be consistent in order to construct a relatively standard feature point coordinate comparison table. This requires the determination of relatively standard annotation rules (automatic annotation or manual annotation must be performed according to the annotation rules). For details, please refer to Figure 4 The step of determining the coordinates of typical feature points in the key point coordinate group and the contour coordinate group comprises the following steps:

[0061] S311: Determine at least one special pixel from the key point coordinate group and the contour coordinate group. This step involves determining at least one pixel that can represent the characteristics or texture of the target from the determined key point coordinate group and the contour coordinate group. The pixel determination rule is predetermined, and different determination plans can be determined based on different target types or attributes. The special pixel selected using the determination rule should at least be able to combine information such as the characteristics and typical texture of the target.

[0062] S312: Use the associated special pixel points to construct special feature areas and obtain multiple feature area points. The feature area points refer to coordinate points that can reflect the typical characteristics of the target. This step means associating and combining the special pixel points determined above to form pixel point areas, namely special feature areas. These special feature areas are recorded as the feature area points of the target, which can reflect the typical characteristics of the target.

[0063] S313: Filter the feature area points based on the labeling rules to obtain the coordinates of typical feature points. This step means filtering all the feature area points obtained above under the determined labeling rules, and filtering the remaining feature area point seats as the typical feature point coordinates. It should be noted that the labeling rules can be determined according to different target types or attributes, and can include labeling positions, labeling methods, labeling quantities, and labeling restrictions. The labeling rules in the screening of feature area points based on the labeling rules here are more about clarifying the number of labels, that is, determining how many suitable feature area points should be obtained.

[0064] Based on the above technical solution, since the rules for determining special pixels also need to be determined in advance, they can also follow the content of the annotation rules. The annotation rules here are more focused on clarifying the annotation location, annotation method, and annotation restrictions. Specifically, determining at least one special pixel from the key point coordinate group and the contour coordinate group specifically includes the following steps:

[0065] S311a: Obtain the target's annotation type, which includes at least one of occlusion annotation, small target annotation, and living object annotation. This step indicates that the target category is determined in advance, so that the annotation method and annotation restriction content can be determined. In this embodiment, three target annotation types are shown, namely, occlusion annotation, small target annotation, and living object annotation. In actual situations, in addition to targets of other categories, a composite annotation method of the above three annotation types is also included, that is, the three categories appear in pairs in the image data, such as occlusion of small targets, occlusion of living objects, etc.

[0066] S311b: Delimit a labeling range based on the labeling type, and select at least a portion of the pixels in the key point coordinate group and the contour coordinate group within the defined labeling range as the special pixels. This step determines the labeling position based on the determined labeling type, i.e., defines labeling ranges for different object categories, and then selects special pixels from these labeling ranges. The special pixels are derived from at least a portion of all the pixels in the key point coordinate group and the contour coordinate group.

[0067] Through the above technical solution, the labeling rules are determined by the category or attribute of the target object. The labeling rules cover not only the number of labels but also the operational requirements in terms of labeling location, labeling method, and labeling restriction content. This allows for more consistent coordinate points and typical feature coordinate points to be obtained, making the table more standardized when constructing a feature point coordinate comparison table. Since different categories of targets have large differences in labeling methods, it is necessary to consider how to make it more likely that or more special pixels will exist within the labeling range defined for different categories of target species. Therefore, it is necessary to perform regional segmentation during labeling, and the segmented areas must have the basis for producing special pixels.

[0068] Specifically in this embodiment, in the first typical case, when the annotation type is occlusion annotation, the annotation range needs to be delineated into visible and invisible parts; thereby, situations where special pixels may exist can be located in both occluded and non-occluded areas, thus avoiding ignoring the possible typical features of the occluded part due to only dividing the visible part. In the second typical case, when the annotation type is small target annotation, the annotation range needs to be delineated into the target part and the background circle part; thereby, the background circle part outside the small target object has the possibility of producing special pixels, wherein the background circle refers to the environment surrounding the small target object, and there is a high correlation between this environment and the small target object, thereby further judging the attributes of the small target object, thereby utilizing the environmental correlation judgment method to distinguish the characteristics of the small target annotation, and producing special pixels in the small target itself and the surrounding related environment, so that the subsequent model can be learned and trained more accurately.

[0069] In the third typical case, when the annotation type is for a living object, the annotation range needs to be delineated to define the moving and non-moving parts. This allows for the location of special pixels within the active and passive motion areas (non-moving parts) of the living object, facilitating subsequent analysis of the motion trends of the moving parts and the development of more appropriate manipulation strategies. More specifically, special pixels are generated for both the moving and non-moving parts. When determining the motion trend, the volume percentage of the non-moving parts can be considered to more accurately determine the displacement change that the moving part can achieve, further enabling the determination of the action outcome under the motion trend.

[0070] The fourth typical case is also a typical composite case. When the annotation type is a composite of occlusion annotation and living object annotation, the annotation range must at least define the moving part in the non-visible part. That is, when living object annotation and occlusion annotation exist at the same time, the living part is occluded, and it is impossible to accurately judge the movement trend and / or movement result of the living object. At this time, it is necessary to clarify the annotation range of the occluded part, that is, to mark the active movement area in the occluded area. This can be done through deep learning completion, radar detection 3D reconstruction, or multi-frame data fusion. Its main purpose is to roughly restore the range of the moving part of the living object in the occluded part of the image, so as to facilitate the judgment of the movement trend.

[0071] Through the four typical situations shown above, the annotation position, annotation method and annotation restriction conditions are given respectively, which can make more accurate feature area point judgment work for each target category in different images, so as to ensure that the subsequent typical feature point coordinates can have more characteristic and specialized attributes, and are more suitable as calibration objects for layer comparison. In addition, based on the above technical solutions, considering that when labeling, there are targets that cannot be accurately defined in the labeling range, such as unclear target recognition, especially in the case of occlusion, when the occluded target is a living object, it is more likely that the labeling cannot be accurately performed. Therefore, it is necessary to clarify the labeling range rules for such abnormal labeling situations, so that the subsequent model can better perform abnormal alarms or more accurate abnormal recognition for this part during training, learning or detection and recognition.

[0072] Specifically, the annotation type also includes abnormal annotation. When the annotation type is abnormal annotation, the annotation range needs to be delineated into a normal recognition part and an abnormal recognition part; that is, when the abnormal target cannot be accurately labeled, it is necessary to clarify the part that can be identified and the part that cannot be identified, that is, to delineate the normal recognition part and the abnormal recognition part, and mark the two parts well. The normal recognition part can promote the prediction and judgment of the target object, and the abnormal recognition part can promote the judgment of the abnormal proportion of the target, and determine whether it is an unclear target as a whole, so as to judge whether the special pixel points of the target only fall within the normal recognition part. It should be noted that if the abnormal recognition part accounts for too large a proportion, it is necessary to consider re-determining the target object. If there is no suitable target object, when learning the standard data set represented by the synchronous image data set, it is necessary to consider the early warning operation for the abnormal recognition situation, so as to wait for the next image acquisition cycle to make a new judgment.

[0073] In addition, when the annotation type is a combination of occlusion and anomaly annotation, this situation is even more difficult to identify and judge. Therefore, the annotation range must at least define the normally identifiable portion within the non-visible area. In other words, the normally identifiable area must be restored or recovered within the occluded area (also using deep learning, radar detection 3D reconstruction, etc.). Similarly, if it is still impossible to accurately mark the normally identifiable portion within the occluded area, it is necessary to consider warning operations for abnormal recognition.

[0074] On the basis that the above abnormal annotations cannot accurately identify the annotation range, considering that the target object cannot be distinguished due to the influence of the shooting environment, such as exposure, heavy rain, dust and other shooting environments, which have a great impact on the camera imaging data, it is also necessary to consider the target object presentation state caused by environmental influences, so as to achieve the diversity and wide coverage of the standard data set, so that the trained model can be generalized to different scenarios and gradually improve the target detection accuracy. Specifically, the annotation range needs to be demarcated into the normal recognition part and the abnormal recognition part, and then the following steps are included:

[0075] The image background type of the abnormally identified portion is determined. The image background type refers to the type of environment in which the image was captured. The impact factor of the image background type on target recognition is calculated, and the abnormally identified portion corresponding to the impact factor exceeding a threshold is adjusted to a normal recognized portion. This step first determines the environmental background type (such as exposure, heavy rain, or dust) of the abnormally identified portion in the abnormally labeled type. The degree of interference caused by this background type on the captured image, i.e., the impact factor, is then determined. This impact factor can be empirically derived from historical data learning or determined by an optimized prediction model based on collected environmental data. The purpose is to determine whether the impact factor is greater than a threshold (generally an empirical value) to determine whether the background type has a significant impact on image capture. If the impact is significant, the abnormally identified image can be used as a subsequent annotated dataset for training or target detection. This means that the abnormally identified portion is adjusted to a normal recognized portion, and feature region points are determined from the normal recognized portion. The coordinates of the typical feature points are then determined to participate in the subsequent annotation difference calculation and determine whether it can be used as the basis for the standard dataset.

[0076] Through the above-mentioned technical solution, the target object can be labeled more reasonably and accurately according to the labeling rules, so that the special pixel points generated in these labeling ranges have the characteristics of comprehensiveness, typicality and full environmental adaptability. When used as the final typical feature point coordinates, the method of using calibration alignment and then difference comparison is more reliable.

[0077] This embodiment also provides a complex traffic scene target detection system 500 based on deep learning, please refer to Figure 5 The modular schematic diagram of the complex traffic scene target detection system 500 based on deep learning is mainly used to divide the functional modules of the complex traffic scene target detection system 500 based on deep learning according to the embodiment of the above method. For example, each functional module can be divided, or two or more functions can be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the present invention is schematic and is only a logical functional division. There may be other division methods in actual implementation. For example, when each functional module is divided according to each function, Figure 5 The diagram is merely a schematic diagram of a system / device, wherein the deep learning-based complex traffic scene object detection system 500 may include a first acquisition unit 510, a first processing unit 520, a first calculation unit 530, and a second processing unit 540. The functions of each unit module are described below.

[0078] A first acquisition unit 510 is configured to acquire image data collected from all viewing angles of the driving vehicle, and synchronize and organize all the image data collected using timestamp information to obtain multiple sets of synchronized image data sets;

[0079] A first processing unit 520 is configured to perform initial labeling on sub-objects of each group of the synchronized image data sets, wherein the initial labeling refers to performing boundary and category labeling processing on different objects in the image;

[0080] The first calculation unit 530 is configured to perform boundary annotation comparison on targets of the same category in each sub-object of the same synchronized image data set, obtain annotation difference, and use the synchronized image data set with the annotation difference not greater than a preset value as a standard data set; wherein the boundary annotation processing includes key point annotation and contour annotation, constructing a feature point coordinate comparison table using the key point annotation information and the contour annotation information, and calculating the annotation difference based on the feature point coordinate comparison table. In some embodiments, the first calculation unit 530 is further configured to obtain a key point coordinate group and a contour coordinate group of targets of the same category in each sub-object, determine typical feature point coordinates in the key point coordinate group and the contour coordinate group; match and calibrate the typical feature point coordinates of targets of the same category in different sub-objects to obtain a calibrated coordinate layer group; set a layer of the calibrated coordinate layer group as a base layer, and set the remaining layers as sub-layers; sequentially match and mark the coordinate points on each sub-layer with the corresponding coordinate points on the base layer, and construct the feature point coordinate comparison table using the matching and marking results; wherein the coordinate points include key point coordinates and contour coordinates. and calculating all difference values ​​for each coordinate point in the feature point coordinate comparison table, combining all difference values ​​to obtain a total difference value for the coordinate point; and calculating the annotation difference using all the total difference values; wherein the difference value refers to the difference distance between the sub-layer coordinate point and the corresponding base layer coordinate point. Furthermore, after calculating all difference values ​​for each coordinate point in the feature point coordinate comparison table, the method further includes the following steps: correcting the data of the sub-layer coordinate point based on radar detection data to obtain a corrected sub-layer coordinate point, and then calculating the difference distance using the corrected sub-layer coordinate point and the corresponding base layer coordinate point.

[0081] In some embodiments, the first calculation unit 530 is further configured to determine at least one special pixel from the key point coordinate group and the contour coordinate group, construct a special feature region using the associated special pixel points, and obtain multiple feature region points, wherein the feature region points are coordinate points that can reflect typical features of the target; filter the feature region points based on the annotation rules to obtain typical feature point coordinates; and obtain the target annotation type, wherein the annotation type includes at least one of occlusion annotation, small target annotation, and living object annotation; define an annotation range based on the annotation type, and select at least some of the pixel points in the key point coordinate group and the contour coordinate group within the defined annotation range as the special pixel points.

[0082] The second processing unit 540 is configured to perform traffic scene target detection using the standard dataset and based on a convolutional neural network model.

[0083] In the above embodiments, the more specific working process of each functional unit can refer to the corresponding content disclosed in the above embodiments. In addition, each functional unit can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive (SSD)).

[0084] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0085] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0087] Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations.

Claims

1. A method for target detection in complex traffic scenes based on deep learning, characterized in that: The steps include: Acquire image-collected data from all viewing angles of the driving vehicle, and synchronize and organize all the image-collected data using timestamp information to obtain multiple sets of synchronized image data sets; Performing initial annotation on sub-objects in each group of the synchronized image data, wherein the initial annotation refers to performing boundary and category annotation processing on different targets in the image; In each sub-object of the same synchronized image data set, boundary annotations are compared for objects of the same category to obtain annotation differences, and synchronized image data sets with annotation differences not greater than a preset value are used as standard data sets; Utilizing the standard dataset and based on a convolutional neural network model, traffic scene target detection is performed; The boundary annotation process includes key point annotation and contour annotation, and a feature point coordinate comparison table is constructed using the key point annotation information and the contour annotation information, and the annotation difference is calculated based on the feature point coordinate comparison table; The method of constructing a feature point coordinate comparison table using key point annotation information and contour annotation information includes the following steps: Obtain a key point coordinate group and a contour coordinate group of the same category of targets in each sub-object, and determine the coordinates of typical feature points in the key point coordinate group and the contour coordinate group; match and calibrate the coordinates of the typical feature points of the same category of targets in different sub-objects to obtain a calibrated coordinate layer group; set a layer of the calibrated coordinate layer group as a base layer, and set the remaining layers as sub-layers; sequentially match and mark the coordinate points on each sub-layer with the corresponding coordinate points on the base layer, and use the matching marking results to construct the feature point coordinate comparison table; wherein the coordinate points include key point coordinates and contour coordinates; Determining the coordinates of typical feature points in the key point coordinate group and the contour coordinate group comprises the following steps: Determining at least one special pixel point from the key point coordinate group and the contour coordinate group, constructing a special feature region using the associated special pixel points to obtain a plurality of feature region points, wherein the feature region points are coordinate points that can reflect typical features of the target; screening the feature region points based on the labeling rules to obtain coordinates of typical feature points; The determining of at least one special pixel point from the key point coordinate group and the contour coordinate group specifically comprises the following steps: Obtaining a target annotation type, where the annotation type includes at least one of occlusion annotation, small target annotation, and living object annotation; defining a annotation range according to the annotation type, and selecting at least some of the pixel points of the key point coordinate group and the contour coordinate group within the defined annotation range as the special pixel points.

2. The method for target detection in complex traffic scenes based on deep learning according to claim 1 is characterized in that: Calculating the annotation difference based on the feature point coordinate comparison table includes the following steps: Calculate all difference values ​​of each coordinate point in the feature point coordinate comparison table, combine all difference values ​​to obtain the total difference value of the coordinate point; use all the total difference values ​​to calculate the annotation difference; wherein, the difference value refers to the difference distance between the sub-layer coordinate point and the corresponding base layer coordinate point.

3. The method for target detection in complex traffic scenes based on deep learning according to claim 2, characterized in that: After calculating all the difference values ​​of each coordinate point in the feature point coordinate comparison table, the following steps are also included: correcting the data of the sub-layer coordinate point based on the radar detection data to obtain the corrected sub-layer coordinate point, and then using the corrected sub-layer coordinate point and the corresponding base layer coordinate point to calculate the difference distance.

4. The method for target detection in complex traffic scenes based on deep learning according to claim 1, characterized in that: When the annotation type is occlusion annotation, the annotation range needs to be demarcated into visible and invisible parts; when the annotation type is small target annotation, the annotation range needs to be demarcated into the target part and the background circle part; when the annotation type is living object annotation, the annotation range needs to be demarcated into the moving part and the non-moving part; When the annotation type is a combination of occlusion annotation and living object annotation, the annotation range needs to at least define the moving part in the non-visible part.

5. The method for target detection in complex traffic scenes based on deep learning according to claim 1, characterized in that: The marking type also includes abnormal marking. When the marking type is abnormal marking, the marking range needs to be demarcated into a normal identification part and an abnormal identification part; When the annotation type is a combination of occlusion annotation and abnormal annotation, the annotation range needs to at least define the normally identifiable part in the non-visible part.

6. The method for target detection in complex traffic scenes based on deep learning according to claim 5 is characterized in that: The marking range needs to be demarcated into a normal identification part and an abnormal identification part, and then further includes the following steps: Determine the image background type of the abnormal recognition part, where the image background type refers to the category of the framing environment when the image is taken; calculate the influence factor of the image background type on target recognition, and adjust the abnormal recognition part corresponding to the situation where the influence factor is greater than the threshold value to a normal recognition part.

7. A deep learning-based target detection system for complex traffic scenes, characterized by: include: a first acquisition unit configured to acquire image-collected data from all viewing angles of the driving vehicle, and to synchronize and organize all the image-collected data using timestamp information to obtain multiple sets of synchronized image data; a first processing unit, configured to perform initial labeling on sub-objects of each group of the synchronized image data sets, wherein the initial labeling refers to performing boundary and category labeling processing on different targets in the image; a first calculation unit configured to perform boundary annotation comparison on targets of the same category in each sub-object of the same synchronized image data set to obtain a degree of annotation difference, and to use the synchronized image data set having the degree of annotation difference not greater than a preset value as a standard data set; wherein the boundary annotation processing includes key point annotation and contour annotation, and a feature point coordinate comparison table is constructed using the key point annotation information and the contour annotation information, and the annotation difference is calculated based on the feature point coordinate comparison table; and for obtaining a key point coordinate group and a contour coordinate group of the same category of targets in each sub-object, determining the coordinates of typical feature points in the key point coordinate group and the contour coordinate group; matching and calibrating the coordinates of the typical feature points of the same category of targets in different sub-objects to obtain a calibrated coordinate layer group; setting a layer of the calibrated coordinate layer group as a base layer, and setting the remaining layers as sub-layers; sequentially matching and marking the coordinate points on each sub-layer with the corresponding coordinate points on the base layer, and constructing the feature point coordinate comparison table using the matching marking results; wherein the coordinate points include key point coordinates and contour coordinates; and for determining at least one special pixel point from the key point coordinate group and the contour coordinate group, constructing a special feature region using the associated special pixel points, and obtaining a plurality of feature region points, wherein the feature region points refer to coordinate points that can reflect typical features of the target; and screening the feature region points based on the labeling rules to obtain coordinates of typical feature points; and obtaining a target annotation type, the annotation type including at least one of occlusion annotation, small target annotation, and living object annotation; defining an annotation range according to the annotation type, and selecting at least some of the pixel points of the key point coordinate group and the contour coordinate group within the defined annotation range as the special pixel points; The second processing unit is used to use the standard data set and perform traffic scene target detection based on a convolutional neural network model.

Images