Traffic complex scene target detection method and system based on deep learning

Abstract

The invention discloses a traffic complex scene target detection method and system based on deep learning, and relates to the technical field of automatic driving scene image processing, and the method comprises the steps: obtaining image collection data of all visual angles of a driving vehicle, and carrying out the synchronous arrangement; performing initial labeling on the sub-objects in each synchronous image data group; performing boundary labeling comparison on the targets of the same category to obtain a labeling difference degree and screening out a standard data set; and carrying out traffic scene target detection by using the standard data set and based on the convolutional neural network model. According to the method and the system, image data obtained from a plurality of visual angles of a vehicle are synchronously arranged, then the difference degree of each visual angle image after annotation is judged, and the image data with the difference degree meeting the requirement is used as a convolutional neural network learning and training data set; therefore, errors caused by acquisition of annotation data of a single view angle for learning training are reduced, and the target learning accuracy and the subsequent judgment precision are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving scene image processing, and in particular to a method and system for detecting targets in complex traffic scenes based on deep learning. Background Art

[0002] Object detection is a basic and important task in the field of computer vision. It determines the category of the detected object and locates its position from a specified image, and has very broad application prospects. Autonomous driving vehicles need to keep track of their own driving status and interact with external environmental information in a timely manner to help improve safety performance. When performing vehicle obstacle detection tasks in complex road scenes, vehicle targets are easily affected by scale changes between different object instances and occlusion by other traffic targets. The feature attributes extracted by the model are inconsistent, especially the target objects and small-scale objects are missed and misdetected.

[0003] In the prior art, the above problems are generally considered in at least one application aspect of multi-scale feature detection, data enhancement, attention mechanism, and optimization of specific scenarios, which can alleviate the above problems to a certain extent and improve the accuracy and robustness of vehicle obstacle detection. However, based on the aforementioned application aspects, there are still certain false detections and missed detections, and the improvement brought by the improvement of convolutional network neural algorithms alone is becoming increasingly limited. From the remaining application scenarios, there have been some cases of optimizing and screening the input data sets of the convolutional network neural algorithm. The results show that the algorithm model can achieve higher detection accuracy, thereby achieving the purpose of high-precision detection.

[0004] As mentioned above, in order to achieve more reliable and accurate visual detection goals in autonomous driving scenarios, it is necessary to further process the perception data such as image data obtained by the vehicle-mounted equipment to ensure that the training model can more accurately identify and accurately learn the relevant content in the perception data to meet driving safety requirements. However, the existing technology for processing vehicle-mounted perception data generally directly labels the data obtained by a single sensor. The data obtained from such a single "perspective" may have large errors, which may mislead the subsequent model learning and judgment, reducing the accuracy of target detection.

[0005] In view of this, this application is hereby filed. Summary of the invention

[0006] The purpose of the present invention is to provide a method and system for target detection in complex traffic scenes based on deep learning. The detection method and system use image data obtained from multiple perspectives of a vehicle for synchronous sorting, and then judge the degree of difference between the images after annotation of each perspective, and use the image data that meets the requirements as a data set for convolutional neural network learning and training, thereby reducing the error caused by collecting annotated data from a single perspective for learning and training, and ensuring the accuracy of target learning and subsequent judgment accuracy.

[0007] The embodiment of the present invention is achieved as follows: In a first aspect, a method for detecting targets in complex traffic scenes based on deep learning comprises the following steps: acquiring image acquisition data from all perspectives of a driving vehicle, synchronizing and arranging all the image acquisition data using timestamp information, and obtaining multiple groups of synchronized image data groups; performing initial annotation on sub-objects in each group of the synchronized image data groups, wherein the initial annotation refers to performing boundary and category annotation processing on different targets in the image; performing boundary annotation comparison on targets of the same category in each sub-object of the same synchronized image data group, obtaining an annotation difference, and using a synchronized image data group whose annotation difference is not greater than a preset value as a standard data set; performing traffic scene target detection using the standard data set and based on a convolutional neural network model; wherein the boundary annotation processing includes key point annotation and contour annotation, constructing a feature point coordinate comparison table using the key point annotation information and the contour annotation information, and calculating the annotation difference based on the feature point coordinate comparison table.

[0008] In some optional implementations, the construction of a feature point coordinate comparison table using key point annotation information and contour annotation information includes the following steps: obtaining a key point coordinate group and a contour coordinate group of targets of the same category in each sub-object, and determining typical feature point coordinates in the key point coordinate group and the contour coordinate group; matching and calibrating typical feature point coordinates of targets of the same category in different sub-objects to obtain a calibrated coordinate layer group; setting a layer of the calibrated coordinate layer group as a base layer, and setting the remaining layers as sub-layers; matching and marking the coordinate points on each sub-layer with the corresponding coordinate points on the base layer in turn, and constructing the feature point coordinate comparison table using the matching marking results; wherein the coordinate points include key point coordinates and contour coordinates.

[0009] In some optional implementations, the calculation of the annotation difference based on the feature point coordinate comparison table includes the following steps: calculating all difference values ​​of each coordinate point in the feature point coordinate comparison table, combining all difference values ​​to obtain the total difference value of the coordinate point; using all the total difference values ​​to calculate the annotation difference; wherein the difference value refers to the difference distance between the sub-layer coordinate point and the corresponding base layer coordinate point.

[0010] In some optional implementations, after calculating all the difference values ​​of each coordinate point in the feature point coordinate comparison table, the following steps are also included: correcting the data of the sub-layer coordinate point based on the radar detection data to obtain the corrected sub-layer coordinate point, and then using the corrected sub-layer coordinate point and the corresponding base layer coordinate point to calculate the difference distance.

[0011] In some optional implementations, determining the coordinates of typical feature points in the key point coordinate group and the contour coordinate group includes the following steps: judging at least one special pixel point from the key point coordinate group and the contour coordinate group, constructing a special feature area using associated special pixel points, and obtaining multiple feature area points, wherein the feature area points refer to coordinate points that can reflect the typical features of the target; screening the feature area points based on the labeling rules to obtain typical feature point coordinates.

[0012] In some optional implementations, the method of determining at least one special pixel point from the key point coordinate group and the contour coordinate group specifically includes the following steps: obtaining a target annotation type, the annotation type including at least one of occlusion annotation, small target annotation and living object annotation; defining an annotation range according to the annotation type, and selecting at least some of the pixel points of the key point coordinate group and the contour coordinate group within the defined annotation range as the special pixel points.

[0013] In some optional implementations, when the annotation type is occlusion annotation, the annotation range needs to delineate the visible part and the invisible part; when the annotation type is small target annotation, the annotation range needs to delineate the target object part and the background circle part; when the annotation type is living object annotation, the annotation range needs to delineate the moving part and the non-moving part; wherein, when the annotation type is a combination of occlusion annotation and living object annotation, the annotation range needs to at least delineate the moving part in the non-visible part.

[0014] In some optional implementations, the annotation type also includes exception annotation. When the annotation type is exception annotation, the annotation range needs to delineate a normal recognition part and an exception recognition part; wherein, when the annotation type is a combination of occlusion annotation and exception annotation, the annotation range needs to at least delineate a normal recognition part in the non-visible part.

[0015] In some optional implementations, the marking range needs to delineate the normal recognition part and the abnormal recognition part and also includes the following steps: determining the image background type of the abnormal recognition part, the image background type refers to the category of the framing environment when the image is taken; calculating the influence factor of the image background type on target recognition, and adjusting the abnormal recognition part corresponding to the situation where the influence factor is greater than the threshold value to a normal recognition part.

[0016] In the second aspect, a complex traffic scene target detection system based on deep learning includes: A first acquisition unit is used to acquire image acquisition data from all viewing angles of the driving vehicle, and synchronize and sort all the image acquisition data using timestamp information to obtain multiple groups of synchronized image data groups; A first processing unit, which is used to perform initial annotation on sub-objects of each group of the synchronous image data group, wherein the initial annotation refers to performing boundary and category annotation processing on different targets in the image; A first calculation unit is used to perform boundary annotation comparison on targets of the same category in each sub-object of the same synchronized image data group to obtain annotation difference, and use the synchronized image data group with the annotation difference not greater than a preset value as a standard data set; wherein the boundary annotation processing includes key point annotation and contour annotation, and a feature point coordinate comparison table is constructed using the key point annotation information and the contour annotation information, and the annotation difference is calculated based on the feature point coordinate comparison table.

[0017] The second processing unit is used to perform traffic scene target detection based on a convolutional neural network model using the standard data set.

[0018] The beneficial effects of the embodiments of the present invention are: The deep learning-based target detection method and system for complex traffic scenes provided by the embodiment of the present invention synchronously organizes the image acquisition data obtained from all perspectives of the driving vehicle, and uses timestamp information to organize the image acquisition data from all perspectives at the same time into the same synchronous image data group; then, the image acquisition data from all perspectives in the synchronous image data group are initially labeled, the boundaries and categories are independently labeled, and then the labeled information is checked and compared, and the boundary labeling comparison of targets of the same category is performed to obtain the labeling difference results to judge whether the image acquisition data obtained from each perspective has a high degree of similarity and correlation, thereby eliminating the problem of accidental factors in obtaining image data from only a single perspective, which is equivalent to a proofreading and review of the synchronous image data group, thereby providing a standard data set for the synchronous image data group that meets the preset requirements, so that the convolutional network neural model can obtain more accurate and fitting training set and test set data, thereby ensuring the accuracy and precision of learning training and subsequent judgment and detection.

[0019] In general, the deep learning-based complex traffic scene target detection method and system provided in the embodiments of the present invention take into account the optimization and screening of the input data set on the basis of using convolutional neural networks to detect targets in complex traffic scenes, which can reduce the situation of invalid data and erroneous data. At the same time, the acquired data is compared and reviewed from multiple perspectives to further ensure the accuracy and matching of the data, thereby ensuring high-quality image data at the input end of the convolutional neural network model, which is used for model learning and training to meet the requirements of high-precision target recognition and detection in complex traffic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flow chart of the main steps of the detection method provided by an embodiment of the present invention; Figure 2 for Figure 1 The main steps shown are one of which, step S300, in a flow chart of an implementation mode; Figure 3 for Figure 1 The main steps shown are one of which, step S300, is a flow chart of another embodiment; Figure 4 A flowchart of sub-steps of step S300 provided for an embodiment of the present invention; Figure 5 A modular schematic diagram of a detection system provided in an embodiment of the present invention.

[0022] Icon: 500 - detection system; 510 - first acquisition unit; 520 - first processing unit; 530 - first calculation unit; 540 - second processing unit. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0024] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] It should be understood that the "system", "device" and / or "module" used in the present invention is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0026] As shown in the present invention and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular, but also include the plural. In general, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements.

[0027] Flowcharts are used in the present invention to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, the various steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes. Example

[0028] When driving in traffic, autonomous vehicles need to identify various targets in the surrounding environment at any time so that they can accurately identify and plan driving routes and driving operations. Since autonomous vehicles need to keep track of their own driving status and interact with external environmental information, they need to use image processing methods to achieve more accurate environmental perception. At present, we use and design novel lightweight perception structure models and target detection algorithms to help autonomous driving technology perceive and understand high-resolution road images on a more portable and easy-to-operate processing platform, and improve the judgment and auxiliary control capabilities of intelligent vehicles for complex traffic scenes. However, during the implementation process, we considered improving and optimizing multiple application aspects such as multi-scale feature detection, data enhancement, attention mechanism, and specific scene optimization. Although we can achieve a significant improvement in target detection accuracy, it is very difficult to further improve it in the future. Because it involves autonomous driving scenarios, the safety standards are extremely high, which requires that the recognition and detection accuracy needs to be further improved.

[0029] In our research, we found that in other intelligent recognition fields, we began to pre-process the collected image data to ensure that the image data is usable on the one hand, and on the other hand, to ensure that the image data content can be accurately and effectively identified. Therefore, we need to consider the image data pre-processing method. In addition to pixel enhancement of image data, accurate image annotation and classification are also required for complex traffic scenes. The traditional annotation method is to annotate a single image without considering whether the single image can be used as a more suitable learning and training object for the subsequent convolutional neural network model, which indirectly reduces the detection accuracy of the trained convolutional neural network model. Therefore, in response to the above problems, we considered the verification and verification of image annotation to ensure that the collected image data has higher adaptability and accuracy.

[0030] Please refer to Figure 1 , a method for detecting complex traffic scene objects based on deep learning provided by this embodiment includes the following steps: S100: Obtain the image acquisition data of all viewing angles of the driving vehicle, synchronize and organize all the image acquisition data using the timestamp information, and obtain multiple groups of synchronized image data groups; this step means acquiring images acquired by cameras installed at multiple positions on the driving vehicle, so as to obtain image acquisition data at different viewing angles. It should be noted that the camera shooting angles at different positions have a cross-overlapping range, and the images acquired from the shooting angles with a cross-overlapping range are subsequently processed to judge the target in the image. There is timestamp information on each image, and all the image acquisition data are synchronized and organized using the same timestamp information to obtain multiple groups of synchronized image data groups. The synchronized image data group obtained at this time refers to a multi-view image data group shot at the same time, so as to facilitate the judgment of the target content of the driving vehicle's surrounding environment at the shooting time from multiple viewing angles.

[0031] S200: Initially annotate the sub-objects in each group of the synchronized image data, and the initial annotation refers to the boundary and category annotation processing for different targets in the image; this step means that each image data (i.e., sub-object) in the synchronized image data group is initially annotated, and the target content in the image needs to be marked with boundaries and the category to which it belongs. It should be noted that the initial annotation here refers to at least the boundary and category annotation processing for different targets in the image, and other implementation methods may also include attribute annotation, relationship annotation, quality annotation, etc.

[0032] After the initial labeling in the above step S200, the labeling information of each sub-object of the synchronized image data group can be obtained. In order to avoid accidental errors caused by single-view labeling, multi-view labeling is adopted and labeling comparison is performed to determine whether the image information after labeling each sub-object is highly correlated and similar, so as to judge whether the synchronized image data group is a qualified data set and can be used for subsequent model learning and training.

[0033] Specifically, S300: in each sub-object of the same synchronous image data group, the boundary annotations of the targets of the same category are compared to obtain the annotation difference, and the synchronous image data group with the annotation difference not greater than the preset value is used as the standard data set; this step means selecting the targets of the same category from each sub-object (such as the targets of the same category can be selected from the images with cross-overlapping perspectives as described in step S100), and performing difference comparison on the boundary annotations of the targets of the same category, so as to obtain the annotation difference after comparison. If the annotation difference obtained from the boundary annotations of the targets of the same category selected from multiple perspectives is not greater than the preset value, the corresponding synchronous image data group is used as the standard data set, which can be used as a qualified data set and can be used for subsequent model learning and training. It should be noted that if there are multiple targets of the same category in each sub-object of the same synchronous image data group, the corresponding synchronous image data group is used as the standard data set if the annotation difference obtained by any one of the targets of the same category is not greater than the preset value, which means that the synchronous image data group has the value of model learning and training.

[0034] In the above step S300, considering that the boundary annotation comparison is obtained by comparing the annotation boundary information of the same target, in order to obtain a more reliable comparison result, it is necessary to use a composite annotation method for comparison to avoid the increase of the comparison result error due to the single or simplified annotation method. Specifically, in this embodiment, the boundary annotation processing includes key point annotation and contour annotation, and the key point annotation information and contour annotation information are used to construct a feature point coordinate comparison table, that is, the boundary annotation uses the key point annotation information of the target and the contour annotation information of the target for comparison, and then constructs a feature point coordinate comparison table according to the comparison situation. The feature point coordinate comparison table, for example, the vertical heading indicates the object situation of the corresponding target in each sub-object, the horizontal heading indicates the coordinate point type situation, and the rest is filled with the coordinate point data (pixel point or pixel area position). Finally, the annotation difference is calculated based on the feature point coordinate comparison table, that is, the annotation difference is obtained according to the difference calculation result of each column of coordinate point data.

[0035] Through step S300, multiple coordinate points (key coordinate points and contour coordinate points) can be used for similarity comparison to determine the similarity of target annotations of the same category in each sub-object, so that the synchronized image data set that meets the similarity requirements is used as a standard data set for subsequent model learning and training steps.

[0036] S400: Utilize the standard data set and perform traffic scene target detection based on the convolutional neural network model; this step means using the standard data set obtained above as the training set and test set of input data to train and optimize the convolutional neural network model, so that the obtained convolutional neural network model can be more accurately applied to traffic scene target detection, and obtain more accurate and matching target detection results. In this embodiment, the convolutional neural network model can adopt a single-stage target detection algorithm. The single-stage target detection algorithm treats target detection as a regression problem and does not use a candidate region generation network. Compared with the two-stage target detection algorithm, the network model of the single-stage target detection algorithm has fewer parameters and calculations, and the detection speed is faster, which is more suitable for rapid judgment of target detection in such traffic driving scenes.

[0037] Taking into account that the detection accuracy of the single-stage target detection algorithm is slightly inferior to that of the two-stage target detection algorithm, and the number of parameters of the algorithm model will cause the model to occupy a large storage space during the training process, and the large amount of calculation will cause the forward reasoning time of the algorithm model to be longer, making the algorithm overly dependent on computer hardware resources, making it difficult to use directly in hardware devices with limited storage resources and computing power. In order to solve the problem that the target detection algorithm occupies more system resources in vehicle driving scenarios, the convolutional neural network model in this embodiment can adopt the lightweight target detection optimization algorithm of YOLOv5s. This algorithm can reduce the number of parameters of the model to a certain extent and improve the detection speed while maintaining a certain detection accuracy, thereby solving the problem that target detection in vehicle driving scenarios occupies more system resources. Specifically, the last three BottleneckCSP1 modules in the backbone network of the YOLOv5s algorithm model are first replaced with the corresponding GhostBottleneck modules, and the CBH module before the SPP module is replaced with a depthwise separable convolution, and an ECA attention module is added after the SPP module; secondly, based on the PANet network structure of the neck network, the ASFF adaptive spatial feature module is integrated; thirdly, CIoU Loss is used as the boundary regression loss function of the YOLOv5s algorithm model.

[0038] In general, the above-mentioned algorithms have been improved and optimized in multiple application aspects such as multi-scale feature detection, data enhancement, attention mechanism, and specific scenario optimization. YOLOv5s is used as the basic framework, combined with Ghost module, depthwise separable convolution, ECA attention module and ASFF feature fusion network module, and CIoU Loss complete intersection over union loss function is used as the boundary frame regression loss function. The algorithm can reduce the number of model parameters to a certain extent while maintaining a certain accuracy, and the detection speed is also improved. It can be used in hardware devices with limited storage resources and computing power, and can well deal with the problem of target detection algorithms occupying more system resources in vehicle driving scenarios.

[0039] Through the above technical solution, by collecting data from various perspectives of the driving vehicle, a more comprehensive environmental perception capability can be provided, and the consistency of data from different perspectives is ensured based on timestamp synchronization, which helps to improve the accuracy of the detection results. Then, by marking the boundaries and categories of different targets, the accuracy of target recognition is improved, and the difference in the annotations of targets of the same category from different perspectives is compared to ensure the quality of the annotation data and reduce the possibility of false detection and missed detection. On this basis, the key points and contour information are used to construct a comparison table, which can capture the characteristics of the target more finely and ensure the reliability of the data source. Then, by screening out the synchronized image data group whose annotation difference is not higher than the preset value, a more reliable standard data set can be constructed for the subsequent training of the YOLOv5s lightweight target detection optimization algorithm, which not only helps to enhance the robustness and generalization ability of the model in complex traffic scenarios, but also has a certain guarantee for detection accuracy and speed, which is crucial for application scenarios such as autonomous driving.

[0040] Since the accuracy and matching degree of the input data of the entire convolutional neural network model affect the upper limit of the final detection result accuracy, it is necessary to consider the logical relationship between the two and consider a reasonable and easy-to-operate implementation mode for obtaining the annotation difference degree based on the feature point coordinate comparison table. Figure 2 In this embodiment, the construction of a feature point coordinate comparison table using key point annotation information and contour annotation information includes the following steps: S310: Obtain the key point coordinate group and contour coordinate group of the same category of targets in each sub-object, and determine the typical feature point coordinates in the key point coordinate group and contour coordinate group; this step represents the acquisition of all key point information and contour information of the same category of targets in each sub-object, and the acquisition of the coordinate information of these key points and the points that constitute the contour, so as to integrate these coordinate information into a key point coordinate group (coordinates of multiple key points) and a contour coordinate group (coordinates of multiple contour points), and determine representative typical feature point coordinates in these coordinates. It should be noted that the typical feature point coordinates may be determined from at least one of the key point coordinate group and the contour coordinate group, and the determined typical feature point coordinates represent the typical information of the category of targets.

[0041] S320: In different sub-objects, the coordinates of typical feature points of the same category of targets are matched and calibrated to obtain a calibration coordinate layer group; this step means calibrating the coordinates of the typical feature points determined for the same category of targets in all sub-objects, that is, aligning and overlapping the corresponding typical feature points in different sub-objects to obtain overlapping matching layers, that is, calibration coordinate layer groups, thereby unifying multi-view data into a reference frame, which can effectively reduce the error accumulation caused by the change of viewing angle. The calibration coordinate layer group includes the image data of each sub-object, and the coordinates of the typical feature points marked and determined in the image data of the same target in each sub-object form a one-to-one correspondence in the overlapping direction of the layer (because the coordinates of the marked typical feature points in different layers have different marking distances, this difference needs to be obtained through subsequent calculations, and the one-to-one correspondence here requires approximate alignment, for example, the highest overlap of the projection of the largest part of the pixel coordinates indicates approximate alignment), so as to facilitate the calibration and comparison step for the typical feature point coordinates.

[0042] S330: Setting a certain layer of the calibration coordinate layer group as a base layer, and setting the remaining layers as sublayers; this step indicates that the calibration coordinate layer group is obtained to determine the benchmark comparison reference, that is, a certain layer is the base layer, and the remaining layers are used as objects to be compared with the base layer. It should be noted that the base layer setting rule can be randomly selected, but considering the reliability of the subsequent annotation difference results, the base layer setting rule can use the layer with the highest alignment degree of typical feature point coordinates with the remaining layers among all layers (by comparing the layers two by two, find a layer with the highest overlap of the largest part of the pixel point coordinate projections after comparing it with the remaining layers) as the base layer, so that a more reasonable comparison difference result can be obtained when comparing it with the remaining layers.

[0043] S340: Match and mark the coordinate points on each sub-layer with the corresponding coordinate points on the base layer in turn, and use the matching marking results to construct the feature point coordinate comparison table; wherein, the coordinate points include key point coordinates and contour coordinates. This step means horizontally matching and marking the coordinate points on the base layer (all coordinates, including key point coordinates and contour coordinates) with the coordinate points at the corresponding positions of the other layers (the positions roughly overlap the corresponding coordinate points after alignment). The matching marking method is to record the relevant pixel points or pixel area coordinates to form the feature point coordinate comparison table. As mentioned above, the vertical heading of the feature point coordinate comparison table indicates the object situation of the corresponding target in each sub-object, the horizontal heading indicates the type of coordinate point situation, and the rest is filled with coordinate point data (pixel point or pixel area position), so as to construct a table that can reflect the coordinate point situation of the corresponding target in each layer according to this matching marking method, so as to facilitate the subsequent one-by-one comparison calculation. It should be noted that the method and number of determining the coordinate points in each layer need to be consistent (the subsequent scheme will be explained) in order to construct a relatively standard feature point coordinate comparison table.

[0044] Through the above technical solution, the corresponding coordinate points of the remaining layers can be matched by calibrating the base layer, so that the constructed feature point coordinate comparison table is easier to operate in the subsequent comparison calculation, and the comparison result is more reasonable, avoiding the increase of the error of the final result due to excessive deviation or dispersion as the basis for difference comparison. Figure 3 , the step of calculating the annotation difference based on the feature point coordinate comparison table comprises the following steps: S350: Calculate all difference values ​​of each coordinate point in the feature point coordinate comparison table, and combine all difference values ​​to obtain the total difference value of the coordinate point; wherein, the difference value refers to the difference distance between the sub-layer coordinate point and the corresponding base layer coordinate point. This step means that according to the feature point coordinate comparison table obtained above, for example, the difference distance is compared with each longitudinal coordinate point, that is, the relative difference distance between the coordinate points of the remaining sub-layers and the base layer in the overlapping projection direction, and the distance value is recorded as the difference value. The difference value represents the degree of difference in the distance between the sub-layer coordinate point and the base layer coordinate point. If the difference value is larger, the farther the two are from each other, the greater the difference in the annotation of the same category of targets in the two sub-objects. In addition to the error caused by the annotation factor, it means that the image data obtained from the two perspectives, the same target presents a large inconsistency, and this group of synchronized image data sets may not meet the requirements of the standard data set.

[0045] After comparing the coordinate points of all sub-layers with the base layer, multiple difference values ​​can be obtained. After merging and calculating all the difference values ​​(by summing, product or averaging, etc.), a total difference value is obtained. The total difference value represents the difference in coordinate points of the same target in all perspective image data. If the total difference values ​​of all coordinate points are large, it means that the annotation difference is too large, otherwise it is smaller. The judgment step is as follows S360: Calculate the annotation difference degree using all the total difference values; this step means that all the coordinate points are combined again by the above-mentioned comparison method to obtain the total difference value (by summing, product, averaging or extreme value, etc.) to obtain the annotation difference degree. If the annotation difference degree is not greater than the preset value, the corresponding synchronous image data group is used as the standard data set.

[0046] On the basis of the above technical solution, since the image acquisition data is affected by the distance of the viewing angle, the target object may be larger when it is near and smaller when it is far away. At this time, this factor needs to be considered on the basis of restoring the target object (also to reduce the influence of the larger when it is near and smaller when it is far away in the image data acquired from each viewing angle, and to restore the essential coordinates of the target as much as possible for comparison). The corresponding coordinate points in the sub-layer to be compared need to eliminate this influence as much as possible, so as to be compared with the coordinate points of the base layer as the benchmark. Therefore, in this embodiment, please refer to Figure 3 , after calculating all the difference values ​​of each coordinate point in the feature point coordinate comparison table, the following steps are also included: S351: Correct the data of the sub-layer coordinate points based on the radar detection data to obtain the corrected sub-layer coordinate points, and then use the corrected sub-layer coordinate points and the corresponding base layer coordinate points to calculate the difference distance. This step means using the radar detection data of non-image data to build a model of the target, and then correct the size of the target in the image according to the actual target size detected in the model building, so as to achieve the purpose of pixel coordinate adjustment, that is, after the data of the sub-layer coordinate points are corrected, the corrected sub-layer coordinate points are fine-tuned on the pixel level, and then the corrected sub-layer coordinate points are used to calculate the difference distance with the corresponding coordinate points on the base layer. It should be noted that in different implementations, the coordinate points of the base layer can also be fine-tuned using radar detection data, but in this embodiment, on the one hand, the amount of calculation is taken into account, and on the other hand, the coordinate point data of the base layer itself is taken into account as the comparison reference. The sub-layer coordinate points brought by the remaining perspectives are more susceptible to the above-mentioned near large and far small changes in the target object, so the method of correcting the coordinate point data of the remaining sub-layers except the base layer is adopted.

[0047] As described in the previous solution, the method and number of coordinate points in each layer need to be consistent in order to build a relatively standard feature point coordinate comparison table. This requires the determination of relatively standard annotation rules (automatic annotation or manual annotation needs to be operated according to the annotation rules). For details, please refer to Figure 4 The step of determining the coordinates of typical feature points in the key point coordinate group and the contour coordinate group comprises the following steps: S311: Determine at least one special pixel from the key point coordinate group and the contour coordinate group; this step means determining at least one pixel that can represent the target feature or texture from the determined key point coordinate group and the contour coordinate group. The judgment rule for the pixel is determined in advance, and different judgment plans can be determined according to different target types or attributes. The special pixel selected by the judgment rule can at least combine information such as the characteristics and typical texture of the target.

[0048] S312: Construct special feature regions using associated special pixel points to obtain multiple feature region points, where the feature region points refer to coordinate points that can reflect typical features of the target. This step represents associating and combining the special pixel points determined above to form pixel point regions, i.e., special feature regions. These special feature regions are recorded as feature region points of the target, which can reflect typical features of the target.

[0049] S313: Screen the feature area points based on the labeling rules to obtain the coordinates of typical feature points. This step means screening all the feature area points obtained above under the determined labeling rules, and screening the remaining feature area point seats as the typical feature point coordinates. It should be noted that the labeling rules can be determined according to different target types or attributes, and may include labeling positions, labeling methods, labeling quantities, and labeling restrictions. The labeling rules in the screening of feature area points based on the labeling rules here are more about clarifying the number of labelings, that is, determining how many adapted feature area points should be obtained.

[0050] On the basis of the above technical solution, since the judgment rules of special pixels also need to be determined in advance, the content of the annotation rules can also be followed. The annotation rules here are more about clarifying the annotation position, annotation method and annotation restriction content. Specifically, judging at least one special pixel from the key point coordinate group and the contour coordinate group specifically includes the following steps: S311a: Obtain the target's annotation type, which includes at least one of occlusion annotation, small target annotation and living object annotation. This step indicates that the target category is determined in advance so that the annotation method and annotation restriction content can be determined. In this embodiment, three target annotation types are shown, namely occlusion annotation, small target annotation and living object annotation. In actual situations, in addition to targets of other categories, a composite annotation method of the above three annotation types is also included, that is, the three categories appear in pairs in the image data, such as occlusion of small targets, occlusion of living objects, etc.

[0051] S311b: Delimit the annotation range according to the annotation type, and select at least part of the pixel points of the key point coordinate group and the contour coordinate group from the defined annotation range as the special pixel points. This step means that the annotation position is determined by the determined annotation type, that is, the annotation ranges of different categories of objects are defined, and then special pixel points are selected from these annotation ranges, and the special pixel points come from at least part of all the pixel points of the key point coordinate group and the contour coordinate group.

[0052] Through the above technical solution, the labeling rules are determined by the category or attribute of the target object. The labeling rules cover the operation requirements in the directions of labeling location, labeling method and labeling restriction content in addition to the labeling quantity, so that more consistent coordinate points and typical feature coordinate points can be obtained, so that the table can be more standardized when constructing the feature point coordinate comparison table. Since there are large differences in the labeling methods of different categories of targets, it is necessary to consider how to make it more likely or more special pixels exist within the labeling range defined by different categories of target species. Therefore, it is necessary to perform regional segmentation during labeling, and the segmented areas need to have the basis for producing special pixels.

[0053] Specifically in this embodiment, in the first typical case, when the annotation type is occlusion annotation, the annotation range needs to delineate the visible part and the invisible part; thereby, the situation where special pixels may exist can be located in both the occluded area and the non-occluded area, avoiding the possibility of ignoring the typical features that may exist in the occluded part because only the visible part is demarcated. In the second typical case, when the annotation type is small target annotation, the annotation range needs to delineate the target part and the background circle part; thereby, the background circle part outside the small target object has the possibility of producing special pixels, wherein the background circle refers to the environment surrounding the small target object, and there is a high correlation with the small target object in this environment, thereby further judging the properties of the small target object, thereby using the environmental association judgment method to distinguish the characteristics of the small target annotation, and producing special pixels in the small target itself and the surrounding associated environment, so that the subsequent model can be learned and trained more accurately.

[0054] In the third typical case, when the annotation type is a living object annotation, the annotation range needs to delineate the moving part and the non-moving part; thus, special pixels can be located in the active motion area and the passive motion area (non-moving part) of the living object, so as to facilitate the subsequent analysis of the motion trend of the moving part, thereby forming a more appropriate control strategy. More specifically, special pixels are generated in both the moving part and the non-moving part, and the volume proportion of the non-moving part can be comprehensively considered when judging the trend of the moving part, so as to more accurately judge the displacement change that the moving part can achieve in its motion trend, thereby further judging the action result under the motion trend.

[0055] The fourth typical case is also a typical case of compound. When the annotation type is a compound case of occlusion annotation and living object annotation, the annotation range needs to at least define the moving part in the non-visible part. That is, when living body annotation and occlusion annotation exist at the same time, the living part is occluded, and the movement trend and / or movement result of the living object cannot be accurately judged. At this time, it is necessary to clarify the annotation range of the occluded part, that is, to mark and define the active movement area in the occluded area. It can be completed by deep learning, or by radar detection 3D reconstruction, or by multi-frame data fusion. Its main purpose is to roughly restore the range of the moving part of the living object in the occluded part of the image, so as to facilitate the judgment of the movement trend.

[0056] Through the four typical situations shown above, the annotation position, annotation method and annotation restriction conditions are given respectively, which can make more accurate feature area point judgment work for each target category in different images, so as to ensure that the subsequent typical feature point coordinates can have more characteristic and specialized attributes, and are more suitable as calibration objects for layer comparison. In addition, on the basis of the above technical scheme, considering that when marking, there are targets that cannot be accurately defined in the marking range, such as unclear target recognition, especially in the case of occlusion, when the occluded target is a living object, it is more likely to be unable to accurately mark, so it is necessary to clarify the marking range rules for such abnormal marking situations, so that the subsequent model can better perform abnormal alarms or more accurate abnormal recognition for this part during training, learning or detection and recognition.

[0057] Specifically, the annotation type also includes abnormal annotation. When the annotation type is abnormal annotation, the annotation range needs to be demarcated into a normal recognition part and an abnormal recognition part; that is, when the abnormal target cannot be accurately labeled, it is necessary to clarify the part that can be recognized and the part that cannot be recognized, that is, to demarcate the normal recognition part and the abnormal recognition part, and mark the two parts. The normal recognition part can promote the prediction and judgment of the target object, and the abnormal recognition part can promote the judgment of the abnormal proportion of the target, and determine whether it is an unclear target as a whole, so as to judge whether the special pixel points of the target only fall within the normal recognition part. It should be noted that if the abnormal recognition part accounts for too large a proportion, it is necessary to consider re-determining the target object. If there is no suitable target object, when learning the standard data set represented by the synchronous image data set, it is necessary to consider the early warning operation for the abnormal recognition situation, so as to wait for the next image acquisition cycle to make a new judgment.

[0058] In addition, when the annotation type is a combination of occlusion annotation and abnormal annotation, it is more difficult to identify and judge. Therefore, it is necessary to define the normal identification part of the non-visible part in the annotation range, that is, to restore or recover the normally identifiable area for the occluded area (also completed by deep learning, radar detection 3D reconstruction, etc.). Similarly, if it is still impossible to accurately mark the normally identifiable part in the occluded area, it is necessary to consider early warning operations for abnormal identification situations.

[0059] On the basis that the above abnormal annotations cannot accurately identify the annotation range, considering the influence of the shooting environment, which leads to unclear identification of the target object, such as exposure, heavy rain, dust and other shooting environments, which have a great impact on the camera imaging data, it is also necessary to consider the presentation state of the target object caused by the environmental influence, so as to achieve the diversification and wide coverage of the standard data set, so that the trained model can be generalized to different scenes and gradually improve the target detection accuracy. Specifically, the annotation range needs to delineate the normal recognition part and the abnormal recognition part, and then also includes the following steps: Determine the image background type of the abnormal recognition part, the image background type refers to the category of the framing environment when the image is taken; calculate the impact factor of the image background type on target recognition, and adjust the abnormal recognition part corresponding to the situation where the impact factor is greater than the threshold value to the normal recognition part. This step means first determining the environmental background type (exposure, rainstorm, dust, etc.) of the abnormal recognition part in the abnormal annotation type, so as to determine the degree of interference on the captured image under this background type, that is, the impact factor. The impact factor can be the empirical result obtained by historical data learning, or it can be the result determined by the optimization prediction model based on the environmental collection data. The purpose is to determine whether the size of the impact factor is greater than the threshold value (generally an empirical value), so as to determine whether the background type has a greater impact on the image shooting. If the impact is too large, the abnormal recognition image can be used as a subsequent annotation data set for training learning or target detection, that is, adjusting the abnormal recognition part to the normal recognition part, determining the feature area points from the normal recognition part, and then determining the coordinates of the typical feature points, so as to participate in the subsequent annotation difference calculation and determine whether it has the basis for being used as a standard data set.

[0060] Through the above-mentioned technical scheme, the marking range of the target object can be delineated more reasonably and accurately according to the marking rules, so that the special pixel points generated in these marking ranges have the characteristics of comprehensiveness, typicality and full environmental adaptability. When they are used as the final typical feature point coordinates, the method of using calibration alignment and then performing difference comparison is more reliable.

[0061] This embodiment also provides a complex traffic scene target detection system 500 based on deep learning, please refer to Figure 5 The modular schematic diagram of the complex traffic scene target detection system 500 based on deep learning is mainly used to divide the functional modules of the complex traffic scene target detection system 500 based on deep learning according to the embodiment of the above method. For example, each functional module can be divided, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the present invention is schematic and is only a logical functional division. There may be other division methods in actual implementation. For example, in the case of dividing each functional module corresponding to each function, Figure 5 What is shown is only a schematic diagram of a system / device, wherein the deep learning-based complex traffic scene target detection system 500 may include a first acquisition unit 510, a first processing unit 520, a first calculation unit 530, and a second processing unit 540. The functions of each unit module are described below.

[0062] A first acquisition unit 510 is used to acquire image acquisition data from all viewing angles of the driving vehicle, and synchronize and sort all the image acquisition data using timestamp information to obtain multiple groups of synchronized image data groups; A first processing unit 520, which is used to perform initial labeling on the sub-objects of each group of the synchronous image data group, wherein the initial labeling refers to performing boundary and category labeling processing on different targets in the image; The first calculation unit 530 is used to compare the boundary annotations of the same category of targets in each sub-object of the same synchronized image data group, obtain the annotation difference, and use the synchronized image data group whose annotation difference is not greater than the preset value as the standard data set; wherein the boundary annotation processing includes key point annotation and contour annotation, and a feature point coordinate comparison table is constructed using the key point annotation information and the contour annotation information, and the annotation difference is calculated based on the feature point coordinate comparison table. In some embodiments, the first calculation unit 530 is also used to obtain the key point coordinate group and the contour coordinate group of the same category of targets in each sub-object, determine the typical feature point coordinates in the key point coordinate group and the contour coordinate group; in different sub-objects, match and calibrate the typical feature point coordinates of the same category of targets to obtain a calibrated coordinate layer group; set a layer of the calibrated coordinate layer group as a base layer, and set the other layers as sub-layers; sequentially match and mark the coordinate points on each sub-layer with the corresponding coordinate points on the base layer, and construct the feature point coordinate comparison table using the matching marking results; wherein the coordinate points include key point coordinates and contour coordinates. And for calculating all difference values ​​of each coordinate point in the feature point coordinate comparison table, combining all difference values ​​to obtain the total difference value of the coordinate point; using all the total difference values ​​to calculate the annotation difference; wherein the difference value refers to the difference distance between the sub-layer coordinate point and the corresponding base layer coordinate point. It is also used to further include the following steps after calculating all difference values ​​of each coordinate point in the feature point coordinate comparison table: correcting the data of the sub-layer coordinate point based on the radar detection data to obtain the corrected sub-layer coordinate point, and then using the corrected sub-layer coordinate point and the corresponding base layer coordinate point to calculate the difference distance.

[0063] In some embodiments, the first calculation unit 530 is further used to determine at least one special pixel point from the key point coordinate group and the contour coordinate group, construct a special feature region using the associated special pixel points, and obtain multiple feature region points, wherein the feature region points refer to coordinate points that can reflect the typical features of the target; the feature region points are screened based on the annotation rules to obtain the coordinates of the typical feature points. It is used to obtain the annotation type of the target, wherein the annotation type includes at least one of occlusion annotation, small target annotation, and living object annotation; the annotation range is defined according to the annotation type, and at least some of the pixel points of the key point coordinate group and the contour coordinate group are selected from the defined annotation range as the special pixel points.

[0064] The second processing unit 540 is used to perform traffic scene target detection based on the convolutional neural network model using the standard data set.

[0065] In the above embodiments, the more specific working process of each functional unit can refer to the corresponding content disclosed in the above embodiments. In addition, each functional unit can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk (sol-identified state disk, SSD)), etc.

[0066] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0067] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0068] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0069] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for target detection in complex traffic scenes based on deep learning, characterized in that: The steps include: Acquire image data collected from all viewing angles of the driving vehicle, and synchronize and sort all the image data collected using timestamp information to obtain multiple groups of synchronized image data; Initially labeling the sub-objects in each group of the synchronous image data sets, wherein the initial labeling refers to labeling the boundaries and categories of different targets in the image; In each sub-object of the same synchronous image data set, boundary marking comparison is performed on targets of the same category to obtain a marking difference, and the synchronous image data set whose marking difference is not greater than a preset value is used as a standard data set; Using the standard data set and based on a convolutional neural network model, traffic scene target detection is performed; The boundary annotation processing includes key point annotation and contour annotation, and a feature point coordinate comparison table is constructed using the key point annotation information and the contour annotation information, and the annotation difference is calculated based on the feature point coordinate comparison table.

2. The method for complex traffic scene target detection based on deep learning according to claim 1 is characterized in that: The method of constructing a feature point coordinate comparison table using key point annotation information and contour annotation information includes the following steps: Obtain the key point coordinate group and contour coordinate group of the same category of targets in each sub-object, and determine the coordinates of typical feature points in the key point coordinate group and contour coordinate group; match and calibrate the typical feature point coordinates of the same category of targets in different sub-objects to obtain a calibrated coordinate layer group; set a certain layer of the calibrated coordinate layer group as a base layer, and set the remaining layers as sub-layers; match and mark the coordinate points on each sub-layer with the corresponding coordinate points on the base layer in turn, and use the matching marking results to construct the feature point coordinate comparison table; wherein, the coordinate points include key point coordinates and contour coordinates.

3. The method for complex traffic scene target detection based on deep learning according to claim 2 is characterized in that: The step of calculating the annotation difference based on the feature point coordinate comparison table comprises the following steps: Calculate all difference values ​​of each coordinate point in the feature point coordinate comparison table, combine all difference values ​​to obtain the total difference value of the coordinate point; use all the total difference values ​​to calculate the annotation difference; wherein the difference value refers to the difference distance between the sub-layer coordinate point and the corresponding base layer coordinate point.

4. The method for detecting complex traffic scenes based on deep learning according to claim 3 is characterized in that: After calculating all the difference values ​​of each coordinate point in the feature point coordinate comparison table, the following steps are also included: correcting the data of the sub-layer coordinate point based on the radar detection data to obtain the corrected sub-layer coordinate point, and then using the corrected sub-layer coordinate point and the corresponding base layer coordinate point to calculate the difference distance.

5. The method for complex traffic scene target detection based on deep learning according to claim 2 is characterized in that: Determining the coordinates of typical feature points in the key point coordinate group and the contour coordinate group comprises the following steps: At least one special pixel point is determined from the key point coordinate group and the contour coordinate group, and a special feature area is constructed using the associated special pixel points to obtain a plurality of feature area points, where the feature area points refer to coordinate points that can reflect typical features of the target; the feature area points are screened based on the labeling rules to obtain typical feature point coordinates.

6. The method for complex traffic scene target detection based on deep learning according to claim 5 is characterized in that: The step of determining at least one special pixel point from the key point coordinate group and the contour coordinate group specifically comprises the following steps: Obtaining a target annotation type, wherein the annotation type includes at least one of occlusion annotation, small target annotation, and living object annotation; defining an annotation range according to the annotation type, and selecting at least part of the pixel points of the key point coordinate group and the contour coordinate group within the defined annotation range as the special pixel points.

7. The method for complex traffic scene target detection based on deep learning according to claim 6, characterized in that: When the annotation type is occlusion annotation, the annotation range needs to delineate the visible part and the invisible part; when the annotation type is small target annotation, the annotation range needs to delineate the target part and the background circle part; when the annotation type is living object annotation, the annotation range needs to delineate the moving part and the non-moving part; When the annotation type is a combination of occlusion annotation and living object annotation, the annotation range at least needs to define the moving part in the non-visible part.

8. The method for complex traffic scene target detection based on deep learning according to claim 6, characterized in that: The annotation type also includes abnormal annotation. When the annotation type is abnormal annotation, the annotation range needs to be demarcated into a normal identification part and an abnormal identification part; When the annotation type is a combination of occlusion annotation and abnormal annotation, the annotation range at least needs to define a normally identifiable portion in the non-visible portion.

9. The method for complex traffic scene target detection based on deep learning according to claim 8, characterized in that: The marking range needs to delineate the normal identification part and the abnormal identification part and further includes the following steps: Determine the image background type of the abnormal recognition part, where the image background type refers to the category of the framing environment when the image is taken; calculate the influence factor of the image background type on target recognition, and adjust the abnormal recognition part corresponding to the situation where the influence factor is greater than the threshold value to a normal recognition part.

10. A complex traffic scene target detection system based on deep learning, characterized in that: include: A first acquisition unit is used to acquire image acquisition data from all viewing angles of the driving vehicle, and synchronize and sort all the image acquisition data using timestamp information to obtain multiple groups of synchronized image data groups; A first processing unit, which is used to perform initial annotation on sub-objects of each group of the synchronous image data group, wherein the initial annotation refers to performing boundary and category annotation processing on different targets in the image; A first calculation unit is used to perform boundary annotation comparison on targets of the same category in each sub-object of the same synchronized image data group to obtain annotation difference, and use the synchronized image data group with the annotation difference not greater than a preset value as a standard data set; wherein the boundary annotation processing includes key point annotation and contour annotation, and a feature point coordinate comparison table is constructed using the key point annotation information and the contour annotation information, and the annotation difference is calculated based on the feature point coordinate comparison table.

11. A second processing unit, which is used to use the standard data set and perform traffic scene target detection based on a convolutional neural network model.

Images