Automatic driving road image marking method and device and medium

Abstract

The invention discloses an automatic driving road image labeling method and device and a medium, and relates to the field of computer vision, and the method comprises the steps: collecting a road scene image in the automatic driving process of a vehicle, extracting the multi-scale visual features of the road scene image, and judging whether there is an object of a known category through a pre-labeling training set; if not, identifying a dense region of the multi-scale visual features, generating a new object candidate region, and generating a corresponding candidate text description; performing matching in a semantic text library based on the candidate text description, and determining a similar semantic text with the highest similarity; fusing the similar semantic text and the multi-scale visual features to generate a visual feature map, and determining an initial object labeling box and a corresponding visual confidence score; and screening the initial object labeling box, and outputting the labeling box of the target object. Visual features and text semantics are fused, and the adaptation ability to emerging objects in a dynamic road scene is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and more specifically to a method, device, and medium for labeling roads in autonomous driving images. Background Art

[0002] Object detection is an image annotation technology based on feature extraction and pattern recognition theory. It identifies objects in images and adds corresponding labels. It is widely used in scenarios such as image database management, intelligent security monitoring, and autonomous driving. In the field of autonomous driving, precise automatic image annotation can effectively improve image retrieval efficiency and assist in scene understanding. Accurately annotating road images is the foundation for vehicles to achieve environmental perception and decision-making.

[0003] Traditional object detection methods are primarily based on a supervised learning paradigm, relying on manually annotating object detection boxes in large numbers of images as true labels. Model parameters are then trained by minimizing a loss function through a backpropagation algorithm. This approach is limited by prior assumptions about data collection and annotation. In autonomous driving scenarios, when objects not included in the training set appear on the road, such as temporarily added smart traffic cones and new LiDAR detection equipment, traditional models lack generalization capabilities and struggle to accurately label them, preventing vehicles from identifying these objects in a timely manner and impacting driving safety. For uniquely shaped concept cars and special models from niche brands, traditional methods are prone to missing or mislabeling due to their limited feature expression capabilities, resulting in the vehicle being unable to correctly judge the type and intentions of surrounding traffic participants. Furthermore, with the advancement of autonomous driving technology, new types of traffic signs and intelligent connected devices are constantly emerging, making it difficult for traditional object detection methods to respond quickly to these changes. Summary of the Invention

[0004] To solve the above problems, this application proposes a method for annotating road images for autonomous driving, including:

[0005] Collecting road scene images during the vehicle's autonomous driving process, extracting multi-scale visual features from the road scene images, and determining whether objects of known categories exist in the road scene images using a pre-labeled training set;

[0006] If not, identifying a dense area of ​​the multi-scale visual features in the road scene image, generating a new object candidate area, and generating a corresponding candidate text description for the new object candidate area;

[0007] Based on the candidate text description, matching is performed in a semantic text library to determine a similar semantic text with the highest similarity to the candidate text description;

[0008] fusing the similar semantic text with the multi-scale visual features to generate a visual feature map, and determining an initial object annotation box and a corresponding visual confidence score in the road scene image;

[0009] The initial object annotation box is screened according to the visual confidence score, and a annotation box of the target object is output.

[0010] On the other hand, this application also proposes an autonomous driving road image annotation device, comprising:

[0011] at least one processor; and,

[0012] a memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute an autonomous driving road image labeling method as described in the above example.

[0014] On the other hand, the present application also proposes a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as: an autonomous driving road image annotation method as described in the above example.

[0015] This application proposes a method for annotating road images for autonomous driving, which can bring the following beneficial effects:

[0016] By extracting multi-scale visual features to capture the details and global information of objects, and combining the text descriptions of the candidate areas of new objects with similar matching of semantic text libraries, the visual features and text semantics are fused, significantly improving the adaptability to emerging objects in dynamic road scenes.

[0017] By generating structured text descriptions of candidate areas of new objects and matching them with the semantic text library for similarity, new objects with high similarity are directly classified into existing categories and supplemented with new attributes. New objects with low similarity are defined as new categories and added to the library, thus expanding the category coverage without human intervention.

[0018] By screening highly reliable candidate frames through visual confidence scores and filtering redundant overlapping frames through area folding ratios, it is possible to reduce invalid calculations while retaining key annotations, allowing the annotation process to run in real time on the on-board computing platform, ensuring that the vehicle can quickly perceive and annotate surrounding objects while driving at high speeds, providing timely environmental information support for the decision-making system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 This is a flowchart of a method for annotating an autonomous driving road image in an embodiment of the present application;

[0021] Figure 2 This is a schematic diagram of an autonomous driving road image annotation device in an embodiment of the present application. DETAILED DESCRIPTION

[0022] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0024] like Figure 1 As shown, the embodiment of the present application provides a method for annotating an autonomous driving road image, comprising:

[0025] S101: Collect road scene images during the vehicle's autonomous driving process, extract multi-scale visual features of the road scene images, and determine whether there are objects of known categories in the road scene images through a pre-labeled training set.

[0026] Specifically, during the vehicle's autonomous driving process, road images are collected through on-board cameras and other devices, and road scene images containing objects are extracted. The road scene images are input into a convolutional neural network or a pre-trained Transformer model for feature extraction to obtain a multi-scale feature map set.

[0027] In the embodiment of the present application, the picture I (I∈R H*W*3 , H, W are the image height and width, 3 represents the RGB three channels) are input into the pre-trained Swin Transformer model for feature extraction to obtain a multi-scale feature map set F, where F = Swin Transformer (I), F is a multi-scale feature map set {F1, F2, F3, F4}. The multi-scale feature map set F of Swin Transformer is passed through FPN by top-down sampling + horizontal connection, and the formula is: P5 = Conv 1*1(F4), generate a unified feature pyramid output P, ​​the formula is: P i =Conv 3*3 (Upsample(P i+1 )+Conv 1*1 (F i )), i = 4, 3, 2, achieving efficient aggregation and enhancement of multi-scale features, where P is the set of feature pyramids {P2, P3, P4, P5}.

[0028] Furthermore, based on a multi-scale feature map set, a visual feature pyramid containing details at different levels is generated. The visual feature pyramid is input into an object detection model trained based on a pre-labeled training set to determine whether there are objects of known categories in the image.

[0029] The object detection model outputs candidate regions, predicted categories, and corresponding prediction confidence scores for all detected objects in the road scene image. The candidate regions are filtered, retaining those with predicted categories that are known and with prediction confidence scores above a first threshold. A determination is then made as to whether at least one candidate region exists after filtering. If so, the annotation results from the pre-annotated training set are obtained and output as the final annotations for the known-category objects.

[0030] Before collecting road scene images during the vehicle's autonomous driving process and determining whether there are objects of known categories in the road scene images through the pre-annotated training set, the object detection model is trained using the pre-annotated training set.

[0031] Specifically, the textual semantic descriptions associated with the annotated historical road images and the annotated objects are obtained, a pre-annotated training set is constructed, the historical multi-scale visual features of the historical road images are extracted, a historical visual feature pyramid is generated, the textual semantic descriptions are structured, a set of semantic fragments with context is generated, the set of semantic fragments is encoded into a textual feature vector, the textual feature vector is fused with the historical visual feature pyramid, a text-enhanced annotated visual feature map is generated, and the target detection model is trained based on the annotated visual feature map.

[0032] In an embodiment of the present application, annotated historical road images and corresponding textual semantic descriptions are collected. The historical road images contain annotation boxes and category labels to construct a pre-annotated training set. Multi-scale visual features are extracted from the historical road images to generate a historical visual feature pyramid. The textual semantic descriptions are segmented and stop words are removed using natural language processing tools. Dependency syntactic analysis is then used to extract semantic fragments containing entities and contexts. These fragments are then encoded into textual feature vectors using semantic encoders such as BERT. The textual feature vectors are fused with the historical visual feature pyramid through feature splicing or attention fusion to generate a text-enhanced annotated visual feature map. This feature map is used to train the target detection model, and the model parameters are optimized through backpropagation so that it can recognize objects of known categories.

[0033] S102: If not, identifying a dense area of ​​the multi-scale visual features in the road scene image, generating a new object candidate area, and generating a corresponding candidate text description for the new object candidate area.

[0034] Specifically, if no known objects exist in the road scene image, indicating the presence of a new object not covered by the pre-annotated training set, the algorithm analyzes pixel gradient changes or feature response strengths in the visual feature pyramid to identify regions with dense visual features and considers them as new object candidates. For each new object candidate, the algorithm extracts an image patch and generates a candidate text description containing the entity category, physical attributes, and spatial location using a pre-trained image description model.

[0035] In an embodiment of the present application, dense areas whose feature values ​​are significantly higher than those of surrounding areas are identified from the visual feature pyramid through sliding window traversal or region proposal network, and the coordinate ranges of these areas are used as a set of new object candidate areas.

[0036] For each candidate area, the corresponding image block is cropped out, and the corresponding text feature vector is generated based on the text prompt template in three dimensions: entity category, physical attribute, and spatial position. By calculating the cosine similarity between the image block visual feature vector and each text prompt feature vector, the text description with the highest similarity is screened out as the candidate output, and then these fragments are combined into a complete candidate text description.

[0037] S103: Based on the candidate text description, matching is performed in a semantic text library to determine a similar semantic text with the highest similarity to the candidate text description.

[0038] Specifically, the candidate text description is matched with the semantic text library, the similarity between the candidate text description and the existing semantic text in the semantic text library is calculated, the similarities are compared, and the highest similarity and the corresponding similar semantic text are determined.

[0039] Furthermore, when the highest similarity is not less than a second threshold, the new object is classified into an existing category corresponding to the similar semantic text, and the new attribute information in the candidate text description is merged into the entry corresponding to the existing category in the semantic text library. When the highest similarity is less than the second threshold, a new category is defined based on the candidate text description and added to the semantic text library.

[0040] In an embodiment of the present application, a cosine similarity is calculated between the candidate text description and each text in the semantic text library. After finding the similar semantic text corresponding to the highest similarity, if the similarity is not less than a second threshold, the new object is classified into the existing category corresponding to the similar semantic text. Optionally, the new attribute in the candidate text is added to the semantic entry of the category. If the similarity is less than the second threshold, a new category is defined based on the candidate text description and added to the semantic text library to achieve dynamic expansion of the library.

[0041] S104: Fusing the similar semantic text with the multi-scale visual features to generate a visual feature map, and determining an initial object annotation box and a corresponding visual confidence score in the road scene image.

[0042] Specifically, through the cross-modal attention mechanism, the feature vectors of similar semantic texts are weightedly fused with the visual feature pyramid to generate an enhanced visual feature map. The target detection head, such as the RPN layer of Faster R-CNN, generates the initial object annotation box from the feature map and outputs the visual confidence score corresponding to each box to reflect the reliability of the box position and category prediction.

[0043] In the embodiment of the present application, the feature vector E of the similar semantic text text Perform cross-modal attention fusion with the visual feature pyramid P = {P2, P3, P4, P5} to enhance the feature response of the text-related area, the formula is: P' l =P l +a l Attention(E text ,P l ), l=2,3,4,5, where a l Is the learnable weight of layer l, which is used to control the influence of text. Based on the text enhancement feature pyramid P', the region proposal network (RPN) is used to generate the initial object annotation box set C and its confidence score S that are managed with text semantics. r , the formula is: C,S r =RPN(P'), C = {C1, C2, ... C i ,...C o}, where C i is the i-th candidate region.

[0044] For the candidate region set C={C1,C2,...C i ,...C o}, from the text enhancement feature pyramid P'=P'2,P'3,P'4,P'5, through ROIPooling and visual projection layer, extract the normalized feature vector of each region and map it to d t dimensional space, the formula is:

[0045]

[0046] Based on the candidate region feature F region,i and similar text feature vector E text Through semantic aggregation and similarity calculation, accurate screening of text-related areas can be achieved.

[0047] Among them, the text semantic aggregation formula is: The formula for calculating the semantic similarity of regional texts is: Combined with the initial confidence score S r,i , introducing text semantic enhancement, constructing fusion scores, and realizing accurate screening and enhancement of candidate regions by text semantics.

[0048] Different from the traditional single visual score screening method, the scientific weight β is used to dynamically integrate visual confidence and text semantics to achieve adaptive control of text guidance strength and improve the semantic accuracy of candidate regions. The formula is: S f,i =S r,i ·(1+βSim(C i ,T)), where β is a learnable weight parameter used to control the influence of text similarity on the final score.

[0049] S105: Filter the initial object annotation box according to the visual confidence score, and output the annotation box of the target object.

[0050] Specifically, the annotation boxes are filtered according to the visual confidence score, and the candidate annotation boxes with scores higher than the preset detection threshold are retained. After sorting them in descending order according to the scores, the region folding ratio is calculated, and redundant boxes with a folding ratio higher than the preset folding threshold are eliminated. Finally, the target annotation box containing position and category information is output.

[0051] In the embodiment of the present application, according to the visual confidence score S f,i Sort the initial object annotation boxes in descending order, build a priority sequence of high confidence areas, and based on the sorting results, select the top n candidate areas with the highest scores to form the processing set R and its corresponding score S' f R, the formula is: S' f =Topn{C i},{S f,i}}), C'={C'1,C'2,...,C' k}, where C' i For the i-th candidate region after text semantic enhancement, accurate screening and enhancement of candidate regions by text semantics are achieved. Topn retains the first n high-confidence regions in the sequence by truncation and screening.

[0052] For the generated Topn high confidence candidate region set R and its corresponding visual confidence score S' f Non-maximum suppression (NMS) is introduced to eliminate redundant prediction boxes with spatial overlap, thus achieving accurate screening of candidate regions. Specifically, according to S' f Sort the candidate regions in R in descending order and build a priority sequence of region-score associations; select the sorted candidate regions in turn and calculate the intersection over union (IoU) between them and the subsequent regions; if the IoU exceeds the set threshold θ, the low-priority overlapping regions are suppressed and only the high-scoring core prediction boxes are retained; finally, the candidate region set R after deduplication is generated final and its corresponding retention score S' f ', the formula is: R final ,S' f '=NMS(R,S' f ,θ), where NMS is the non-maximum suppression algorithm, which selects the candidate regions with unique spatial positions and the best scores; S' f ': The visual confidence score corresponding to the candidate region retained by NMS.

[0053] For the generated precise candidate region R final The text semantic segment set T' is processed through RegionCLIP to perform cross-modal feature enhancement calculation, and the visual features of the candidate region are deeply integrated with the text semantics. The formula is: E region =RegionCLIP(R final ,T'), where Each e i It is a high-dimensional feature vector that fuses visual details with text semantics.

[0054] Calculate the enhanced region embedding E region With text embedded E text The cosine similarity of quantifies the semantic matching between the region and the text. The formula is: in, The higher the value, the higher the semantic fit between the i-th candidate region and the text T'. According to the obtained visual confidence score S' f ' and cross-modal similarity Construct a multimodal fusion score, filter and output the final detection box set B, and the multimodal fusion score formula is: Where λ = 0.7, λ∈[0,1] is the learnable weight, Norm(·) Perform standardization. Keep S final,j >=δ area, δ is the set threshold, according to S final,j Sort in descending order and output the final detection box set B, B={B1,B2,...,B h}h<=m rc .

[0055] By extracting multi-scale visual features to capture the details and global information of objects, and combining the text descriptions of the candidate areas of new objects with similar matching of semantic text libraries, the visual features and text semantics are fused, significantly improving the adaptability to emerging objects in dynamic road scenes.

[0056] By generating structured text descriptions of candidate areas of new objects and matching them with the semantic text library for similarity, new objects with high similarity are directly classified into existing categories and supplemented with new attributes. New objects with low similarity are defined as new categories and added to the library, thus expanding the category coverage without human intervention.

[0057] By screening highly reliable candidate frames through visual confidence scores and filtering redundant overlapping frames through area folding ratios, it is possible to reduce invalid calculations while retaining key annotations, allowing the annotation process to run in real time on the on-board computing platform, ensuring that the vehicle can quickly perceive and annotate surrounding objects while driving at high speeds, providing timely environmental information support for the decision-making system.

[0058] like Figure 2 As shown, the embodiment of the present application also proposes an autonomous driving road image annotation device, including:

[0059] at least one processor; and,

[0060] a memory communicatively connected to the at least one processor; wherein,

[0061] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute an autonomous driving road image labeling method as described in any of the above embodiments.

[0062] An embodiment of the present application further provides a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to be: an autonomous driving road image annotation method as described in any of the above embodiments.

[0063] The various embodiments in this application are described in a progressive manner. Similar portions between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device and medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For relevant portions, refer to the descriptions of the method embodiments.

[0064] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects to their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0065] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0066] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0067] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0068] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0069] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0070] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0071] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0072] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0073] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for labeling road images for autonomous driving, characterized in that: include: Collecting road scene images during the vehicle's autonomous driving process, extracting multi-scale visual features from the road scene images, and determining whether objects of known categories exist in the road scene images using a pre-labeled training set; If not, identifying a dense area of ​​the multi-scale visual features in the road scene image, generating a new object candidate area, and generating a corresponding candidate text description for the new object candidate area; Based on the candidate text description, matching is performed in a semantic text library to determine a similar semantic text with the highest similarity to the candidate text description; fusing the similar semantic text with the multi-scale visual features to generate a visual feature map, and determining an initial object annotation box and a corresponding visual confidence score in the road scene image; The initial object annotation box is screened according to the visual confidence score, and a annotation box of the target object is output.

2. The method for labeling an autonomous driving road image according to claim 1, characterized in that: Before collecting the road scene image during the automatic driving process of the vehicle, the method further includes: Obtain the annotated historical road images and the textual semantic descriptions associated with the annotated objects to construct a pre-annotated training set; extracting historical multi-scale visual features of the historical road image to generate a historical visual feature pyramid; Performing structural processing on the text semantic description to generate a set of semantic segments with context, and encoding the set of semantic segments into a text feature vector; fusing the text feature vector with the historical visual feature pyramid to generate a text-enhanced annotated visual feature map; The target detection model is trained based on the labeled visual feature map.

3. The method for labeling an autonomous driving road image according to claim 2, characterized in that: The determining whether there is an object of a known category in the road scene image using the pre-labeled training set specifically includes: Based on the multi-scale visual features, a visual feature pyramid is generated and input into the object detection model; Outputting, through the target detection model, candidate regions, predicted categories, and corresponding prediction confidences corresponding to all detected objects in the road scene image; Filtering the candidate regions, and retaining candidate regions whose predicted categories are known categories and whose prediction confidences are higher than a first threshold; Determine whether there is at least one candidate region after filtering.

4. The method for labeling an autonomous driving road image according to claim 3, wherein: The fusing of the similar semantic text and the multi-scale visual features to generate a visual feature map, and determining an initial object annotation box and a corresponding visual confidence score in the road scene image specifically includes: Based on the attention weight, the similar semantic text and the visual feature pyramid are fused to generate a text-enhanced visual feature map; Based on the visual feature map, an initial object annotation box in the road scene image is determined, and a visual confidence score corresponding to the initial object annotation box is calculated.

5. The method for labeling an autonomous driving road image according to claim 3, wherein: The identifying of a dense region of the multi-scale visual features in the road scene image to generate a new object candidate region, and generating a corresponding candidate text description for the new object candidate region, specifically includes: Based on the visual feature pyramid, identifying feature-dense areas in the road scene image to obtain a set of new object candidate areas; Extracting an image block of each candidate region in the set of new object candidate regions, and generating a structured text description of the candidate region based on the image block, wherein the structured text description includes an entity category, physical attributes, and spatial location; The structured text descriptions are combined to obtain candidate text descriptions corresponding to the new object candidate region set.

6. The method for labeling an autonomous driving road image according to claim 1, characterized in that: The matching in the semantic text library based on the candidate text description to determine the similar semantic text with the highest similarity to the candidate text description specifically includes: Matching the candidate text description with a semantic text library, and calculating the similarity between the candidate text description and the existing semantic texts in the semantic text library; Comparing the similarities to determine the highest similarity and the corresponding similar semantic text; After determining the semantically similar text with the highest similarity to the candidate text description, the method further includes: When the highest similarity is not less than a second threshold, classifying the new object into the existing category corresponding to the similar semantic text; Merging the new attribute information in the candidate text description into the entry corresponding to the existing category in the semantic text library; When the highest similarity is less than the second threshold, a new category is defined based on the candidate text description and added to the semantic text library.

7. The method for labeling an autonomous driving road image according to claim 1, characterized in that: The filtering of the initial object annotation box according to the visual confidence score and outputting the annotation box of the target object specifically includes: Determining whether the visual confidence score is not less than a preset detection threshold; If yes, retain the initial object annotation box corresponding to the visual confidence score to obtain the candidate annotation box; Based on the visual confidence scores, sorting the candidate annotation boxes in descending order; Calculating the area folding ratios of the candidate annotation frames in sequence, and retaining the valid annotation frames corresponding to the area folding ratios being lower than a preset folding threshold; The position information of the valid annotation box is bound to the category label and output.

8. The method for labeling an autonomous driving road image according to claim 1, characterized in that: After determining whether there are objects of known categories in the road scene image using the pre-labeled training set, the method further includes: If so, the labeling result in the pre-labeled training set is obtained, and the labeling result is output as the final labeling of the known category object.

9. An autonomous driving road image annotation device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the autonomous driving road image labeling method according to any one of claims 1 to 8.

10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are configured to execute an autonomous driving road image labeling method according to any one of claims 1 to 8.

Images