An image annotation method, system, device, and medium

Abstract

This application discloses an image annotation method, system, device, and medium, relating to the field of information processing technology. The method includes: acquiring a first set of line features in a lane-level map and a second set of line features in a vector map; converting the first set of line features to a vehicle coordinate system to obtain a third set of line features, and converting the second set of line features to a vehicle coordinate system to obtain a fourth set of line features; matching the third set of line features and the fourth set of line features based on geometric distance to obtain multiple line segment pairs; acquiring at least one corresponding point pair for each line segment pair to obtain a point pair set; obtaining geometric correction parameters that minimize the overall difference of the point pair set based on an optimization model; correcting the third set of line features based on the geometric correction parameters to obtain a fifth set of line features; and projecting the fifth set of line features onto the environmental image to generate annotation data for the environmental image.

Description

Technical Field

[0001] This application relates to the field of information processing technology, and in particular to an image annotation method, system, device and medium. Background Technology

[0002] Currently, with the development of autonomous driving technology, the demand for high-precision, large-scale labeled data for model training is increasing. Traditional automated labeling methods typically rely heavily on high-precision maps as a baseline ground truth source, achieving labeling by projecting their features onto the perceived image.

[0003] However, existing image annotation methods require the use of expensive maps as initial references, resulting in low accuracy of the final image annotation data. Summary of the Invention

[0004] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This summary section is not intended to limit the key and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.

[0005] In a first aspect, embodiments of this application provide an image annotation method, the method comprising: The first set of line features in the lane-level map and the second set of line features in the vector map are obtained. The lane-level map is obtained based on the vehicle's environment, and the vector map is obtained by annotating the environmental image of the vehicle's environment based on the vectorized map annotation model. The first set of line features is transformed into the vehicle coordinate system to obtain the third set of line features, and the second set of line features is transformed into the vehicle coordinate system to obtain the fourth set of line features. The third set of line features and the fourth set of line features are matched according to the geometric distance to obtain multiple line segment pairs. Each line segment pair includes: a first line segment in the third set of line features and a second line segment in the fourth set of line features that are matched with each other. Obtain at least one corresponding point pair for each line segment pair to obtain a point pair set, and obtain the geometric correction parameter that minimizes the overall difference of the point pair set based on the optimization model; The third set of line features is corrected based on the geometric correction parameters to obtain the fifth set of line features. The fifth set of line features is then projected onto the environmental image to generate the annotation data of the environmental image.

[0006] In one embodiment of the present invention, the step of matching the third group of line features and the fourth group of line features according to geometric distance to obtain multiple line segment pairs includes: Based on the overlap interval between the line segments in the third group of line elements and the line segments in the fourth group of line elements in a preset first direction, a set of candidate line segment pairs is determined. The geometric similarity of each candidate line segment pair in the candidate line segment pair set is determined based on geometric distance. Construct a cost matrix based on the geometric similarity; Multiple line segment pairs in the cost matrix are determined based on the allocation algorithm.

[0007] In one embodiment of the present invention, obtaining at least one corresponding point pair for each line segment pair to obtain a point pair set includes: Within the overlapping interval, select the point that is closest to any line segment in the line segment pair and the other line segment in the line segment pair to determine the corresponding point pair.

[0008] In one embodiment of the present invention, obtaining the geometric correction parameters that minimize the overall difference of the set of point pairs based on the optimization model includes: Assign weights to the corresponding point pairs in the set of point pairs to obtain weighted point pairs; The weighted points are aligned with the first point set from the lane-level map and geometrically transformed according to preset candidate parameters to determine the residual function between the geometrically transformed first point set and the second point set from the vector map. Based on preset parameter boundary constraints and the least squares method, the residual function is solved to determine the geometric correction parameter that minimizes the residual function value.

[0009] In one embodiment of the present invention, the step of correcting the third group of line features based on the geometric correction parameters to obtain a fifth group of line features, and projecting the fifth group of line features onto the environmental image to generate annotation data for the environmental image includes: Based on the geometric correction parameters, the points in the third group of line features are geometrically transformed to determine the fifth group of line features; Based on the calibration parameters of the vehicle camera, the fifth set of line features is transformed from the vehicle coordinate system to the camera coordinate system to determine the camera coordinate system point set; The camera coordinate system point set is pixelated to determine the pixel coordinates of the fifth set of line features on the environmental image; Based on the pixel coordinates, annotation data for the environmental image is generated.

[0010] In one embodiment of the invention, before matching the third group of line features and the fourth group of line features according to geometric distance, the method further includes: Based on the geometric proximity and directional relationship between any two line segments in the first group of line elements and the second group of line elements, determine whether the distance between the endpoints of any two line segments in the first group of line elements and the second group of line elements is less than a preset endpoint threshold and whether the directional angle is less than a preset angle threshold. If the distance between the endpoints of any two line segments in the first group of line elements and the second group of line elements is less than a preset endpoint threshold and the angle between their directions is less than a preset angle threshold, the line segments are clustered and merged to generate a merged line segment set. Based on a preset length threshold, the line segments in the merged line segment set are filtered to determine the first group of line elements and the second group of line elements used for line segment matching.

[0011] In one embodiment of the present invention, the step of converting the first set of line features to the vehicle coordinate system to obtain the third set of line features, and converting the second set of line features to the vehicle coordinate system to obtain the fourth set of line features, includes: Transform the first latitude and longitude coordinates of each point in the first group of line features into the first planar coordinates; Acquire the vehicle's attitude data and position data, and based on the attitude data and position data, transform the first planar coordinates to the vehicle coordinate system to obtain the third set of line elements; Based on the second latitude and longitude coordinates of each point in the second group of line elements, convert them into second plane coordinates; The second plane coordinates are transformed to the vehicle coordinate system to obtain the fourth set of line elements.

[0012] In one embodiment of the present invention, the geometric correction parameters include at least one of yaw angle, lateral translation, pitch angle, and roll angle.

[0013] Secondly, this application proposes an image annotation system, the system comprising: a data acquisition module, a line segment matching module, and an image annotation module; The data acquisition module is configured to acquire the first set of line elements in the lane-level map and the second set of line elements in the vector map. The lane-level map is obtained based on the vehicle's environment, and the vector map is obtained by annotating the environmental image of the vehicle's environment based on a vectorized map annotation model. The line segment matching module is configured to: convert the first group of line elements to the vehicle coordinate system to obtain the third group of line elements, and convert the second group of line elements to the vehicle coordinate system to obtain the fourth group of line elements; match the third group of line elements and the fourth group of line elements according to geometric distance to obtain multiple line segment pairs, each line segment pair including: a first line segment in the third group of line elements and a second line segment in the fourth group of line elements that match each other; The image annotation module is configured to: obtain at least one corresponding point pair for each line segment pair to obtain a point pair set; obtain geometric correction parameters that minimize the overall difference of the point pair set based on an optimization model; correct the third group of line features based on the geometric correction parameters to obtain a fifth group of line features; project the fifth group of line features onto the environmental image to generate annotation data for the environmental image.

[0014] Thirdly, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program stored in the memory to implement the steps of an image annotation method as described in any of the first aspects above.

[0015] Fourthly, this application also proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of an image annotation method according to any one of the first aspects.

[0016] In summary, the image annotation method of this application utilizes a widely available and low-cost lane-level map as an initial reference; it matches the first set of line features of the lane-level map with the second set of line features generated by real-time vehicle perception in the vehicle coordinate system based on geometric distance to obtain multiple line segment pairs; it obtains at least one corresponding point pair for each line segment pair to obtain a point pair set; it obtains the geometric correction parameter that minimizes the overall difference of the point pair set based on an optimization model; it then corrects the first set of line features according to the geometric correction parameter; and it backprojects the corrected high-precision result onto the original environmental image, resulting in higher accuracy of the final generated image annotation data.

[0017] The image annotation method proposed in this application, along with other advantages, objectives, and features of this application, will be partly apparent from the following description and partly understood by those skilled in the art through study and practice of this application. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating an image annotation method provided in an embodiment of this application. Figure 2 This is a schematic diagram of the projection region in an image annotation method provided in an embodiment of this application; Figure 3This is a schematic diagram of an image annotation system structure provided in an embodiment of this application; Figure 4 This is a schematic diagram of an image annotation electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To better understand the technical solutions provided in the embodiments of this specification, the technical solutions of the embodiments of this specification will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this specification and the specific features in the embodiments are detailed descriptions of the technical solutions of the embodiments of this specification, rather than limitations on the technical solutions of this specification. In the absence of conflict, the embodiments of this specification and the technical features in the embodiments can be combined with each other.

[0020] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The term "two or more" includes two or more cases.

[0021] Please see Figure 1 This is a flowchart illustrating an image annotation method provided in an embodiment of this application, specifically including: S110. Obtain the first set of line elements in the lane-level map and the second set of line elements in the vector map. The lane-level map is obtained based on the vehicle's environment, and the vector map is obtained by annotating the environmental image of the vehicle's environment based on the vectorized map annotation model. For example, lane-level maps (LDMaps) are generated based on the actual surrounding environment in which a vehicle is driving. They include geographic features such as lane lines and road boundaries. The first set of line features extracted from this lane-level map is typically represented as a set of points in latitude and longitude coordinates. Vector maps, on the other hand, are generated by automatically annotating environmental images collected by the vehicle, such as surround-view images, using a Vector Map Annotation Model (VMA). The second set of line features included in a vector map is the result of directly perceiving the road geometry in the environmental image; these are features directly extracted from the image. This ensures that the annotations have both a geographic spatial basis and fit the actual scene depicted in the image.

[0022] Before extracting the first and second sets of line features, data integrity is verified. This mainly involves checking the existence of necessary fields in the LDMap and VMA model annotation results, such as coordinates, timestamps, and semantic labels. Missing or abnormal records are marked and listed in the output report. Based on the timestamp, the location or map that best matches the current frame is selected. If a significant time offset occurs, exceeding the tolerable threshold, it is marked as a risk factor for subsequent QC (Quality Control). Specifically, the system needs to process multi-source data such as lane-level maps, VMA annotation results, and vehicle positioning information. This data must be time-synchronized, meaning the same timestamp corresponds to the same frame scene, to ensure the accuracy of subsequent coordinate transformations and line segment matching. If the timestamp of a frame of data shows a significant offset, such as the difference between the LDMap timestamp and the camera image timestamp exceeding a preset threshold (e.g., 0.5 seconds), it indicates that this set of data may have a spatiotemporal misalignment, such as matching a map at time t1 with an image at time t2. Instead of discarding the data directly to avoid accidentally deleting valid data, the abnormality is marked first and then included in the subsequent QC process. The QC stage will focus on checking the annotation results of this frame of data, such as checking whether the line segment matching is misaligned, whether the annotation position does not match the actual scene of the image, and finally determine whether the annotation result is qualified or whether it needs to be manually reviewed and corrected.

[0023] Semantic tags from different sources are mapped to a unified category vocabulary for subsequent matching, improving accuracy and efficiency. After extracting the first and second sets of line features, basic filtering is performed on extreme noise points to remove isolated points, and the line features are then organized into a continuous point sequence in their original order. The same real road feature, such as a complete lane line, may be split into multiple short segments due to detection errors or occlusion. These short segments are not discarded or modified directly; instead, their original form is temporarily preserved to facilitate merging into complete line segments later using clustering algorithms. For each processed line feature, key auxiliary information, such as metadata, is added and stored in a unified format in the output data structure, facilitating subsequent data source tracing and problem investigation.

[0024] S120. The first group of line elements is transformed into the vehicle coordinate system to obtain the third group of line elements, and the second group of line elements is transformed into the vehicle coordinate system to obtain the fourth group of line elements. The third group of line elements and the fourth group of line elements are matched according to the geometric distance to obtain multiple line segment pairs. Each line segment pair includes: a first line segment in the third group of line elements and a second line segment in the fourth group of line elements that are matched with each other. For example, two sets of line features that might originally be in different coordinate systems, such as latitude and longitude coordinate systems or map pixel coordinate systems, are uniformly transformed to the vehicle coordinate system. The transformation process requires first converting latitude and longitude coordinates to planar coordinates, such as the ENU (East-North-Up) coordinate system, and then combining vehicle attitude and position data to complete the coordinate system mapping, ensuring that the two types of line features are under the same spatial reference and are comparable. In the line segment matching stage, the similarity of each line segment in the two sets of line features is quantified based on geometric distance, such as Chamfer distance. Line segments with similar spatial positions and shapes are selected, forming multiple one-to-one line segment pairs. Coordinate unification eliminates the spatial reference differences between different data sources, ensuring that line features are geometrically calculated under the same coordinate system, avoiding matching errors caused by spatial misalignment. The geometric distance-based matching method objectively quantifies line segment similarity, which is more accurate and efficient than manual matching, reducing subjective errors. S130. Obtain at least one corresponding point pair for each line segment pair to obtain a point pair set. Based on the optimization model, obtain the geometric correction parameter that minimizes the overall difference of the point pair set. For example, in the point pair extraction stage, for each valid line segment pair, a corresponding point is selected within its spatial overlap interval according to the principle of uniform sampling or nearest neighbor. That is, a point is selected from the line segment in the third group of line features, and the point closest to this point is found on the line segment in the fourth group of line features. The two form a corresponding point pair. The corresponding point pairs of all line segment pairs form a point pair set, which is the basis for quantifying the geometric differences between the two groups of line features. Then, an optimization model, such as the least squares model, is used to solve for the geometric correction parameters that minimize the overall difference of the point pair set, such as the sum of Euclidean distances, as the optimization objective. These parameters are usually used to compensate for the geometric deviation between the two groups of line features. The optimization model aims to minimize the overall difference, ensuring that the solved geometric correction parameters have global optimality and can fully compensate for the deviation between line features. It does not rely on high-precision positioning data and solves the parameters through pure geometric operations, overcoming the labeling error problem caused by positioning drift when GNSS (Global Navigation Satellite System) signals are interfered with.

[0025] S140. Based on the geometric correction parameters, the third group of line features is corrected to obtain the fifth group of line features. The fifth group of line features is projected onto the environmental image to generate the annotation data of the environmental image.

[0026] For example, the geometric correction parameters obtained from the solution are applied to the third group of line features. Through geometric transformations such as rotation and translation, the original geometric deviation of the third group of line features is eliminated to obtain the fifth group of line features. Then, the calibration parameters of the vehicle camera are used to transform the fifth group of line features from the vehicle coordinate system to the camera coordinate system, and then to the image pixel coordinate system. Finally, the five groups of line feature annotations are rendered on the environmental image to form standardized annotation data.

[0027] In summary, the image annotation method proposed in this application utilizes widely available and low-cost lane-level maps as an initial reference. It matches the first set of line features from the lane-level map with the second set of line features generated by real-time vehicle perception in the vehicle coordinate system based on geometric distance. The corresponding point pairs generated by the matching are then optimized using least squares to obtain geometric correction coefficients. The first set of line features is then corrected based on these geometric correction parameters, and the corrected high-precision results are back-projected onto the original environmental image, resulting in higher accuracy of the final generated image annotation data.

[0028] In some examples, the matching of the third group of line features and the fourth group of line features based on geometric distance yields multiple line segment pairs, including: Based on the overlap interval between the line segments in the third group of line elements and the line segments in the fourth group of line elements in a preset first direction, a set of candidate line segment pairs is determined. The geometric similarity of each candidate line segment pair in the candidate line segment pair set is determined based on geometric distance. Construct a cost matrix based on the geometric similarity; Multiple line segment pairs in the cost matrix are determined based on the allocation algorithm.

[0029] For example, in the line segment matching process, firstly, corresponding roads are screened at the road level, such as candidate pairs of left and right boundaries and the same road segment. Then, similar line segment pairs are screened at the line level to reduce computation. Next, the projection direction of two line segments detected by the vehicle's forward direction or local reference axis is used as the first direction. Projections are made onto each line segment in the third group of line elements and the fourth group of line elements, and the projection interval of each line segment in this first direction is calculated. By detecting any two line segments from different groups of line elements, whether their projection intervals intersect and the length of the intersection, the length of the intersection is compared with a preset overlap length threshold, thereby filtering out line segment combinations with excessively short overlap lengths, thus forming a candidate line segment pair set. For example: Figure 2 The diagram illustrates the projection interval in an image annotation method proposed in this application. The preset overlap length threshold is 5 meters. First, line segments from the third and fourth sets of line elements with an overlap exceeding 5 meters in the vehicle's direction of travel are selected as candidate line segment pairs. Alternatively, line segments from the third and fourth sets of line elements with different x-axis values ​​but partially overlapping y-axis values ​​can be selected, where the partially overlapping portion must exceed 5 meters. Specifically, an 8-meter-long road boundary line from the third set of line elements and a 10-meter-long road boundary line from the fourth set of line elements overlap by 6 meters in the direction of travel, thus becoming a candidate line segment pair.

[0030] After obtaining the candidate line segment pair set, for each candidate line segment pair in the set, their geometric similarity is quantified by calculating the chamfer distance between their respective point sets. Commonly used geometric similarity metrics include Chamfer distance and the average distance between the two nearest points. The smaller the distance value, the closer the shapes and spatial positions of the two line segments are. Subsequently, a cost matrix is ​​constructed based on the geometric similarity, with the line segments of the third group of line elements as rows and the line segments of the fourth group of line elements as columns. Each element in the cost matrix represents the similarity cost of the corresponding line segment pair. Finally, based on this cost matrix, the Hungarian assignment algorithm is used to find the global optimum, assigning each line segment a unique matching object with the minimum overall cost, thereby determining the final multiple one-to-one line segment pairs.

[0031] Candidate line segment pairs are selected based on overlapping intervals to reduce invalid matching calculations and improve matching efficiency. Geometric distance is used to quantify similarity, making the evaluation criteria for line segment matching more objective and accurate. An allocation algorithm is used to determine the optimal line segment pair, avoiding the subjectivity and error of manual matching.

[0032] In some examples, obtaining at least one corresponding point pair for each line segment pair to obtain a set of point pairs includes: Within the overlapping interval, select the point that is closest to any line segment in the line segment pair and the other line segment in the line segment pair to determine the corresponding point pair.

[0033] For example, within the overlapping interval, the Euclidean distance from each point on any line segment in a line segment pair to all points on the other line segment is calculated, and the point with the smallest distance is selected. These two closest points form a corresponding point pair. All such point pairs are aggregated to form multiple point pairs. The matching results are quantitatively evaluated using three types of indicators: point-level matching rate, which divides the number of matched points by the number of predicted points, reflects the accuracy of the point set fitting for a single line segment. For example, the higher the percentage of valid point pairs in a matching line segment pair, the closer the spatial fit between the two point sets; line-level matching rate, which divides the number of matched line segments by the number of candidate line segments, reflects the completeness of the overall matching and the matching coverage of the main road line elements; and overall matching confidence, which combines similarity and coverage to give an overall quality score, serving as the basis for judging whether the matching result is qualified. These evaluation indicators can effectively filter out mismatches and mark risk items, ensuring that high-quality matching data is used subsequently. Matching details are written into the output structure and used as input for least squares optimization. Point pairs are extracted only within the overlapping interval to avoid interference from points in irrelevant areas.

[0034] In some examples, the geometric correction parameters obtained based on the optimization model that minimize the overall difference of the set of point pairs include: Assign weights to the corresponding point pairs in the set of point pairs to obtain weighted point pairs; The weighted points are aligned with the first point set from the lane-level map and geometrically transformed according to preset candidate parameters to determine the residual function between the geometrically transformed first point set and the second point set from the vector map. Based on preset parameter boundary constraints and the least squares method, the residual function is solved to determine the geometric correction parameter that minimizes the residual function value.

[0035] For example, zero values ​​or parameters smoothed from historical frames are used as initial values. Weights are assigned to each corresponding point pair in the point pair set to obtain weighted point pairs. The weight assignment can be determined based on factors such as the confidence level of the point pair or its distance to the vehicle center. If there is no special prior information, the weights are initialized to equal weights, allowing important point pairs to play a greater role in the optimization process. Then, the first point set from the lane-level map in the weighted point pairs is geometrically transformed according to preset candidate parameters. The difference between the transformed first point set and the second point set from the vector map is then calculated to form a residual function. The geometric transformation includes rotation and translation along the vertical axis, and the residual uses two-dimensional Euclidean distance. Finally, under preset parameter boundary constraints (e.g., heading angle within ±30 degrees, lateral translation within ±5 meters), the residual function is solved using the least squares method to obtain the geometric correction parameters that minimize the residual function value. During the solution process, to suppress the negative impact of outliers, a robust loss function, such as the Cauchy loss function, is used, and outlier removal is performed using RANSAC (Random Sample Consensus Algorithm) before and after optimization. Assigning weights to corresponding point pairs highlights the role of key point pairs and suppresses the influence of minor or error-prone point pairs. Combining parameter boundary constraints and the least squares method ensures that the solved geometric correction parameters conform to physical reality and minimize point set differences, thereby improving correction accuracy.

[0036] In some examples, the step of correcting the third group of line features based on the geometric correction parameters to obtain the fifth group of line features, projecting the fifth group of line features onto the environmental image, and generating the annotation data of the environmental image includes: Based on the geometric correction parameters, the points in the third group of line features are geometrically transformed to determine the fifth group of line features; Based on the calibration parameters of the vehicle camera, the fifth set of line features is transformed from the vehicle coordinate system to the camera coordinate system to determine the camera coordinate system point set; The camera coordinate system point set is pixelated to determine the pixel coordinates of the fifth set of line features on the environmental image; Based on the pixel coordinates, annotation data for the environmental image is generated.

[0037] For example, a precise geometric transformation is performed on the third set of line features in the vehicle coordinate system based on geometric correction parameters to generate a fifth set of line features that conforms to the current actual scene. The correction process specifically involves: for each coordinate point in the third set of line features, performing a preset geometric transformation operation according to the geometric correction parameters; this operation involves sequentially rotating and translating around the vertical axis, where rotation is defined by the yaw angle, and translation is defined by the lateral and longitudinal translation amounts. The height value is queried or fitted for each point based on the nearest point cloud to obtain 3D coordinates; points with unreliable height data can be labeled with low confidence.

[0038] After obtaining the fifth set of line features, they are projected from the vehicle coordinate system onto the environmental image to generate pixel-level annotation data. Specifically, the camera calibration parameters include camera extrinsic and intrinsic parameters. Based on the camera extrinsic parameters, the fifth set of line features are transformed from 3D points in the vehicle coordinate system to the camera coordinate system (Xc, Yc, Zc), eliminating points with Zc ≤ 0 (i.e., points behind the camera or outside the visible depth range); points exceeding the image boundary are cropped or discarded. Subsequently, based on the camera intrinsic parameters and lens distortion model, perspective projection and distortion correction calculations are performed on the 3D points located in the camera coordinate system, mapping them to the 2D image pixel plane to determine the pixel coordinates corresponding to each 3D point. During this process, necessary pixel interpolation or line width estimation is required to generate visual lines. Finally, based on the pixel coordinates, annotation data for the environmental image is generated, wherein continuous sampling points are connected into lines and rendered as annotation layers on the original image, while the original image is preserved for manual verification. Using standardized formats such as GeoJSON, Extensible Markup Language (XML), or internal proto (protocol buffer), the final pixel-level annotations and corresponding metadata are exported. The metadata includes confidence levels, source information, and correction parameters. Line segments are precisely corrected based on geometric correction parameters to ensure consistency between line features and the actual environment. Coordinate system transformation and pixelation are performed using camera calibration parameters, achieving accurate mapping of line features from 3D space to the image plane. The generated annotation data is standardized and highly visualized, facilitating manual review and model training. The entire annotation generation process is automated, eliminating the need for manual drawing and significantly improving annotation efficiency and accuracy.

[0039] In some examples, the process of matching the third group of line features and the fourth group of line features based on geometric distance is further included: Based on the geometric proximity and directional relationship between any two line segments in the first group of line elements and the second group of line elements, determine whether the distance between the endpoints of any two line segments in the first group of line elements and the second group of line elements is less than a preset endpoint threshold and whether the directional angle is less than a preset angle threshold. If the distance between the endpoints of any two line segments in the first group of line elements and the second group of line elements is less than a preset endpoint threshold and the angle between their directions is less than a preset angle threshold, the line segments are clustered and merged to generate a merged line segment set. Based on a preset length threshold, the line segments in the merged line segment set are filtered to determine the first group of line elements and the second group of line elements used for line segment matching.

[0040] For example, before matching the third group of line features and the fourth group of line features based on geometric distance, the first group of line features and the second group of line features are first subjected to line segment clustering and merging processing to eliminate fragmented line segments caused by detection errors or discontinuous segmentation, thereby improving the accuracy of subsequent matching. Specifically, for any two line segments in the first group of line features or the second group of line features, their geometric proximity and directional relationship are calculated: the geometric proximity is quantified by calculating the shortest distance between the endpoints of the two line segments; if the shortest distance is less than a preset endpoint distance threshold, the two line segments are determined to be spatially adjacent; the directional relationship is quantified by calculating the angle between the direction vectors of the two line segments; if the angle is less than a preset angle threshold, the directions of the two line segments are determined to be basically consistent. Two line segments are deemed to meet the merging condition if and only if the endpoint distance is less than the endpoint threshold and the directional angle is less than the angle threshold. Based on connectivity (connected components) or hierarchical clustering merging strategy, the two line segments that meet the condition are combined into a cluster, and the merged continuous line (splicing in geometric order and aligning the endpoints at the seams) is output, generating a set of merged line segments.

[0041] Based on a preset length threshold, line segments in the merged line segment set are filtered. This means that line segments shorter than the threshold are deleted or marked as noise. Simple topology checks, such as self-intersection detection and duplicate coverage detection, are performed on the merged line segments, and anomalies are corrected or marked. The first and second sets of line features used for line segment matching are then determined. Clustering and merging fragmented line segments forms continuous and complete line features, improving the integrity and usability of line features; filtering out excessively short line segments reduces noise interference, improving the accuracy and efficiency of subsequent line segment matching.

[0042] In some examples, the process of transforming the first set of line features to the vehicle coordinate system to obtain the third set of line features, and transforming the second set of line features to the vehicle coordinate system to obtain the fourth set of line features, includes: Transform the first latitude and longitude coordinates of each point in the first group of line features into the first planar coordinates; Acquire the vehicle's attitude data and position data, and based on the attitude data and position data, transform the first planar coordinates to the vehicle coordinate system to obtain the third set of line elements; Based on the second latitude and longitude coordinates of each point in the second group of line elements, convert them into second plane coordinates; The second plane coordinates are transformed to the vehicle coordinate system to obtain the fourth set of line elements.

[0043] For example, for the first set of line features from the lane-level map, each point is typically represented by latitude and longitude coordinates. These need to be converted to first-plane coordinates, such as the ENU coordinate system, through a preset map projection transformation. Then, vehicle attitude and position data are acquired in real time. Based on this data, the first-plane coordinates are further transformed to the vehicle coordinate system, resulting in the third set of line features. If a point contains height information (Z), its 3D height is retained; if the height is missing, it can be retrieved and filled from the nearest point cloud. Linear interpolation or spline interpolation is performed along the line segment at user-defined fixed intervals to generate a uniform sampling point sequence, ensuring consistent sampling density and point distribution for line features from different sources. The interpolation interval can be set according to the scene resolution, with a preferred range of 0.3–1 meter. Only points falling within the analysis BEV (bird's-eye view) range, such as within 50 meters in front and behind and 25 meters to the left and right, are retained, excluding distant or irrelevant points to reduce interference.

[0044] For the second set of line features from the vector map annotation model, each point coordinate is also in latitude and longitude format. These are converted to second planar coordinates using the same method, and then transformed to the vehicle coordinate system using a known transformation or approximate transformation to obtain the fourth set of line features. This ensures that both sets of line features are in the same spatial coordinate system, meeting subsequent matching requirements. Converting latitude and longitude coordinates to planar coordinates facilitates geometric distance calculations and matching operations; unifying them to the vehicle coordinate system eliminates spatial differences between line features from different sources.

[0045] In some examples, the geometric correction parameters include at least one of yaw angle, lateral translation, pitch angle, and roll angle.

[0046] For example, geometric correction parameters include at least one of yaw angle, lateral translation, pitch angle, and roll angle. These parameters correspond to geometric deviation corrections in different dimensions. Yaw angle corrects heading deviation of line features in the horizontal plane, lateral translation corrects lateral position deviation in the horizontal direction, pitch angle corrects pitch deviation in the vertical plane, and roll angle corrects roll deviation in the vertical plane. One or more parameters can be selected for line feature correction according to the actual scenario requirements. Multiple parameter combinations are supported, adapting to geometric deviation correction needs in different scenarios and improving the versatility and flexibility of the method.

[0047] like Figure 3 As shown, this application proposes an image annotation system, which includes: a data acquisition module 21, a line segment matching module 22, and an image annotation module 23; The data acquisition module 21 is configured to acquire a first set of line elements in the lane-level map and a second set of line elements in the vector map. The lane-level map is obtained based on the vehicle's environment, and the vector map is obtained by annotating the environmental image of the vehicle's environment based on a vectorized map annotation model. The line segment matching module 22 is configured to: convert the first group of line elements to the vehicle coordinate system to obtain the third group of line elements, and convert the second group of line elements to the vehicle coordinate system to obtain the fourth group of line elements; match the third group of line elements and the fourth group of line elements according to geometric distance to obtain multiple line segment pairs, each line segment pair including: a first line segment in the third group of line elements and a second line segment in the fourth group of line elements that match each other; The image annotation module 23 is configured to: obtain at least one corresponding point pair for each line segment pair to obtain a point pair set; obtain geometric correction parameters that minimize the overall difference of the point pair set based on an optimization model; correct the third group of line features based on the geometric correction parameters to obtain a fifth group of line features; project the fifth group of line features onto the environmental image to generate annotation data for the environmental image.

[0048] The effects of applying the aforementioned method in the above system can be found in the description of the aforementioned method embodiments, and will not be repeated here.

[0049] like Figure 4 As shown, this application embodiment also provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, it implements the steps of any of the above-described image annotation methods.

[0050] Since the electronic device described in this embodiment is a device used to implement an image annotation device in the embodiments of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in the embodiments of this application. Therefore, how the electronic device implements the method in the embodiments of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiments of this application falls within the scope of protection of this application.

[0051] In practical implementation, when the computer program 311 is executed by the processor, it can achieve the following: Figure 1 Any of the corresponding implementation methods in the embodiments.

[0052] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0053] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-readable program code.

[0054] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0055] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0056] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0057] This application also provides a computer program product, which includes computer software instructions that, when executed on a processing device, cause the processing device to execute the LDPC decoding method of a solid-state drive controller.

[0058] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0059] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0060] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0061] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0062] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0063] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0064] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0065] Although preferred embodiments have been described in this specification, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this specification.

[0066] Obviously, those skilled in the art can make various modifications and variations to this specification without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims and their equivalents, this specification is also intended to include such modifications and variations.

Claims

1. An image annotation method, characterized in that, The method includes: The first set of line features in the lane-level map and the second set of line features in the vector map are obtained. The lane-level map is obtained based on the vehicle's environment, and the vector map is obtained by annotating the environmental image of the vehicle's environment based on the vectorized map annotation model. The first set of line features is transformed into the vehicle coordinate system to obtain the third set of line features, and the second set of line features is transformed into the vehicle coordinate system to obtain the fourth set of line features. The third set of line features and the fourth set of line features are matched according to the geometric distance to obtain multiple line segment pairs. Each line segment pair includes: a first line segment in the third set of line features and a second line segment in the fourth set of line features that are matched with each other. Obtain at least one corresponding point pair for each line segment pair to obtain a point pair set, and obtain the geometric correction parameter that minimizes the overall difference of the point pair set based on the optimization model; The third set of line features is corrected based on the geometric correction parameters to obtain the fifth set of line features. The fifth set of line features is then projected onto the environmental image to generate the annotation data of the environmental image.

2. The image annotation method according to claim 1, characterized in that, The matching of the third group of line features and the fourth group of line features based on geometric distance yields multiple line segment pairs, including: Based on the overlap interval between the line segments in the third group of line elements and the line segments in the fourth group of line elements in a preset first direction, a set of candidate line segment pairs is determined. The geometric similarity of each candidate line segment pair in the candidate line segment pair set is determined based on geometric distance. Construct a cost matrix based on the geometric similarity; Multiple line segment pairs in the cost matrix are determined based on the allocation algorithm.

3. The image annotation method according to claim 2, characterized in that, The step of obtaining at least one corresponding point pair for each line segment pair to obtain a point pair set includes: Within the overlapping interval, select the point that is closest to any line segment in the line segment pair and the other line segment in the line segment pair to determine the corresponding point pair.

4. The image annotation method according to claim 1, characterized in that, The geometric correction parameters obtained based on the optimization model that minimize the overall difference of the set of point pairs include: Assign weights to the corresponding point pairs in the set of point pairs to obtain weighted point pairs; The weighted points are aligned with the first point set from the lane-level map and geometrically transformed according to preset candidate parameters to determine the residual function between the geometrically transformed first point set and the second point set from the vector map. Based on preset parameter boundary constraints and the least squares method, the residual function is solved to determine the geometric correction parameter that minimizes the residual function value.

5. The image annotation method according to claim 1, characterized in that, The process of correcting the third group of line features based on the geometric correction parameters to obtain the fifth group of line features, and projecting the fifth group of line features onto the environmental image to generate the annotation data of the environmental image includes: Based on the geometric correction parameters, the points in the third group of line features are geometrically transformed to determine the fifth group of line features; Based on the calibration parameters of the vehicle camera, the fifth set of line features is transformed from the vehicle coordinate system to the camera coordinate system to determine the camera coordinate system point set; The camera coordinate system point set is pixelated to determine the pixel coordinates of the fifth set of line features on the environmental image; Based on the pixel coordinates, annotation data for the environmental image is generated.

6. The image annotation method according to claim 1, characterized in that, Before matching the third group of line features and the fourth group of line features based on geometric distance, the process also includes: Based on the geometric proximity and directional relationship between any two line segments in the first group of line elements and the second group of line elements, determine whether the distance between the endpoints of any two line segments in the first group of line elements and the second group of line elements is less than a preset endpoint threshold and whether the directional angle is less than a preset angle threshold. If the distance between the endpoints of any two line segments in the first group of line elements and the second group of line elements is less than a preset endpoint threshold and the angle between their directions is less than a preset angle threshold, the line segments are clustered and merged to generate a merged line segment set. Based on a preset length threshold, the line segments in the merged line segment set are filtered to determine the first group of line elements and the second group of line elements used for line segment matching.

7. The image annotation method according to claim 1, characterized in that, The step of converting the first set of line features to the vehicle coordinate system to obtain the third set of line features, and converting the second set of line features to the vehicle coordinate system to obtain the fourth set of line features, includes: Transform the first latitude and longitude coordinates of each point in the first group of line elements into the first planar coordinates; Acquire the vehicle's attitude data and position data, and based on the attitude data and position data, transform the first planar coordinates to the vehicle coordinate system to obtain the third set of line elements; Based on the second latitude and longitude coordinates of each point in the second group of line elements, convert them into second planar coordinates; The second plane coordinates are transformed to the vehicle coordinate system to obtain the fourth set of line elements.

8. The image annotation method according to claim 1, characterized in that, The geometric correction parameters include at least one of yaw angle, lateral translation, pitch angle, and roll angle.

9. An image annotation system, characterized in that, The system includes: a data acquisition module, a line segment matching module, and an image annotation module; The data acquisition module is configured to acquire the first set of line elements in the lane-level map and the second set of line elements in the vector map. The lane-level map is obtained based on the vehicle's environment, and the vector map is obtained by annotating the environmental image of the vehicle's environment based on a vectorized map annotation model. The line segment matching module is configured to: convert the first group of line elements to the vehicle coordinate system to obtain the third group of line elements, and convert the second group of line elements to the vehicle coordinate system to obtain the fourth group of line elements; match the third group of line elements and the fourth group of line elements according to geometric distance to obtain multiple line segment pairs, each line segment pair including: a first line segment in the third group of line elements and a second line segment in the fourth group of line elements that match each other; The image annotation module is configured to: obtain at least one corresponding point pair for each line segment pair to obtain a point pair set; obtain geometric correction parameters that minimize the overall difference of the point pair set based on an optimization model; correct the third group of line features based on the geometric correction parameters to obtain a fifth group of line features; project the fifth group of line features onto the environmental image to generate annotation data for the environmental image.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of an image annotation method as described in any one of claims 1-8.

Images