A data set labeling method and apparatus

Abstract

This invention relates to a dataset annotation method and apparatus, comprising: acquiring initial GPS positioning information, inertial navigation information, and a surround view image of a vehicle; matching a front view image in the surround view image with a high-precision map to obtain current vehicle positioning information; fusing the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information to output a vehicle positioning result; determining whether the surround view image and the vehicle positioning result are aligned; if aligned, locating the vehicle positioning result on the high-precision map based on the initial GPS positioning information; selecting a preset image region on the high-precision map based on the surround view image, corresponding the surround view image with the preset image region, and then annotating it based on map information on the high-precision map to obtain annotated data, thereby solving the problems of low data annotation efficiency and high manual annotation costs.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a dataset annotation method and apparatus. Background Technology

[0002] In the field of intelligent vehicles, with the development of vehicle networking and autonomous driving technologies, the use of autonomous driving technology is becoming more and more widespread. Among them, BEV imaging solutions are gradually becoming one of the mainstream methods in the current field of autonomous driving.

[0003] BEV image processing is a data fusion technology that uses the vehicle's own vision as the center to perceive and fuse multi-view images or multi-modal data. Depending on the type of data fused, it can be categorized into different directions. While BEV image processing has its unique advantages, like other models, it requires model training. Training this model requires a large amount of labeled datasets. Currently, related technologies use manual annotation of datasets, but this method is inefficient and costly. Summary of the Invention

[0004] One objective of this invention is to provide a dataset annotation method to solve the problems of low data annotation efficiency and high manual annotation costs; another objective is to provide a dataset annotation device.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A dataset annotation method includes: acquiring initial GPS positioning information, inertial navigation information, and a surround view image of a vehicle; matching a front view image in the surround view image with a high-precision map to obtain current vehicle positioning information; fusing the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information to output a vehicle positioning result; determining whether the surround view image and the vehicle positioning result are aligned; if aligned, locating the vehicle positioning result on the high-precision map based on the initial GPS positioning information; selecting a preset image region on the high-precision map based on the surround view image, corresponding the surround view image with the preset image region, and then annotating it based on map information on the high-precision map to obtain annotation data.

[0007] Based on the above technical means, vehicle positioning results are obtained through initial GPS positioning information, inertial navigation information, and surround view images. The corresponding dataset annotation is then obtained by fusing the vehicle positioning results and surround view images with a high-precision map. Due to the introduction of GPS positioning information and inertial navigation information, the annotation accuracy is high, and the annotation results can be automatically generated at once without manual intervention, thus saving annotation costs.

[0008] Furthermore, determining whether the surround view image and the vehicle positioning result are aligned includes: obtaining a first timestamp corresponding to the surround view image and obtaining a second timestamp corresponding to the vehicle positioning result; calculating the difference between the first timestamp and the second timestamp; if the difference is less than a set threshold, then the surround view image and the vehicle positioning result are aligned.

[0009] The above-mentioned technical means are used to ensure the accuracy of annotation when using high-precision map data, and also to solve the problem of high-precision map annotation distortion caused by GPS signal loss.

[0010] Furthermore, fusing the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information to output the vehicle positioning result includes: adjusting the weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information according to the working state of the GPS to obtain adjusted weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information; fusing the adjusted weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information to obtain the vehicle positioning result.

[0011] The accuracy of vehicle positioning results is ensured by the aforementioned technical means.

[0012] Furthermore, adjusting the weights of the initial GPS positioning information, the vehicle positioning information, and the inertial navigation information based on the GPS's operating status includes: if the GPS is operating normally, the weight of the initial GPS positioning information is greater than the weights of the current vehicle positioning information and the inertial navigation information; if the GPS is not operating normally, the weight of the initial GPS positioning information is less than the weights of the current vehicle positioning information and the inertial navigation information.

[0013] The accuracy of vehicle positioning results is ensured by the aforementioned technical means.

[0014] Furthermore, the method also includes: calibrating the inertial sensor and the vehicle's surround-view camera; using the calibrated inertial sensor to collect the inertial navigation information and using the calibrated vehicle's surround-view camera to collect surround-view images; initializing the GPS so that the initial solution output by the GPS is a fixed solution, and recording the initial positioning information of the GPS.

[0015] The above-mentioned technical methods ensured the accuracy and correctness of the annotation.

[0016] Furthermore, the calibration of the vehicle's surround-view camera includes: calibrating the surround-view camera using a camera calibration board;

[0017] Calculate the reprojection error of the calibrated surround-view camera; if the reprojection error is less than a set error threshold, the calibration is complete.

[0018] The above-mentioned technical means ensured the accuracy of the annotation.

[0019] A dataset annotation device, comprising:

[0020] The acquisition module is used to acquire the vehicle's initial GPS positioning information, inertial navigation information, and surround view image;

[0021] The matching module is used to match the front view image in the surround view image with the high-precision map to obtain the current vehicle positioning information;

[0022] The fusion module is used to fuse the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information, and output the vehicle positioning result;

[0023] The judgment module is used to determine whether the surround view image is aligned with the vehicle positioning result. If they are aligned, the vehicle positioning result is located on the high-precision map according to the initial positioning information of the GPS.

[0024] The annotation module is used to select a preset image area on the high-precision map based on the panoramic image, match the panoramic image with the preset image area, and then annotate it according to the map information on the high-precision map to obtain annotation data.

[0025] Furthermore, the judgment module includes: a timestamp unit, used to obtain a first timestamp corresponding to the surround view image and a second timestamp corresponding to the vehicle positioning result; and a first calculation unit, used to calculate the difference between the first timestamp and the second timestamp, wherein if the difference is less than a set threshold, the surround view image is aligned with the vehicle positioning result.

[0026] Furthermore, the fusion module includes: an adjustment unit, configured to adjust the weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information according to the working state of the GPS, to obtain the adjusted weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information; and a fusion unit, configured to fuse the adjusted weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information to obtain a vehicle positioning result.

[0027] Furthermore, the adjustment unit is specifically used to: if the GPS is in normal working condition, then the weight of the initial GPS positioning information is greater than the weight of the current vehicle positioning information and the weight of the inertial navigation information; if the GPS is not in normal working condition, then the weight of the initial GPS positioning information is less than the weight of the current vehicle positioning information and the weight of the inertial navigation information.

[0028] Furthermore, the device also includes: a calibration module for calibrating the inertial sensor and the vehicle's surround-view camera; an acquisition module for acquiring the inertial navigation information using the calibrated inertial sensor and acquiring surround-view images using the calibrated vehicle's surround-view camera; and an initialization module for initializing the GPS so that the initial solution output by the GPS is a fixed solution and recording the initial positioning information of the GPS.

[0029] Furthermore, the calibration module for calibrating the surround-view camera of the vehicle includes: a camera calibration unit for calibrating the surround-view camera using a camera calibration board; a second calculation unit for calculating the reprojection error of the calibrated surround-view camera; if the reprojection error is less than a set error threshold, the calibration is completed.

[0030] The beneficial effects of this invention are:

[0031] This invention obtains vehicle positioning results using initial GPS positioning information, inertial navigation information, and current vehicle positioning information, ensuring the accuracy of vehicle positioning. Then, the vehicle positioning results are aligned with a surround-view image, and the vehicle positioning result is located on a high-precision map based on the initial GPS positioning information. A preset image area is selected on the high-precision map based on the surround-view image, and the surround-view image is matched with the preset image area. Annotations are then made based on map information on the high-precision map to obtain annotation data. Because GPS positioning information, inertial navigation information, and current vehicle positioning information are incorporated, the annotation accuracy is high, and the annotation results can be automatically generated in one go without manual intervention, saving annotation costs. Attached Figure Description

[0032] Figure 1 This is a flowchart of a dataset annotation method according to the present invention;

[0033] Figure 2 This is a flowchart of a dataset annotation method according to the present invention;

[0034] Figure 3-a This is a schematic diagram of the high-precision map of the present invention;

[0035] Figure 3-b This is a schematic diagram of the GPS positioning output of the present invention;

[0036] Figure 3-c Annotated data generated using a data annotation method of the present invention;

[0037] Figure 4 This is a schematic diagram of the structure of a dataset annotation device according to the present invention. Detailed Implementation

[0038] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0039] Reference Figure 1 The document illustrates a flowchart of a data set annotation method according to an embodiment of the present invention, the method specifically including:

[0040] Step 101: Obtain the vehicle's initial GPS positioning information, inertial navigation information, and surround view image.

[0041] In practical applications, before acquiring the vehicle's initial GPS positioning information, inertial navigation information, and surround view image, it is necessary to calibrate the GPS, inertial sensor, and camera. Then, the calibrated GPS, inertial sensor, and camera are used to collect the vehicle's initial GPS positioning information, inertial navigation information, and surround view image, respectively.

[0042] The inertial sensor can be calibrated in the following way: Specifically, the inertial sensor is calibrated using a calibration board, and then the calibrated inertial sensor is used to collect inertial navigation information, which includes: acceleration information and attitude change information during vehicle operation.

[0043] The front-view camera and surround-view camera can be calibrated separately using the following methods. Specifically, the front-view camera is calibrated using a camera calibration board. The intrinsic parameters and extrinsic parameters relative to the vehicle body are calibrated using the calibration board. The reprojection error of the calibrated front-view camera is calculated. If the reprojection error is less than a set error threshold, the calibration is complete, and the calibrated front-view camera of the vehicle is used to acquire front-view images.

[0044] To ensure the final annotation accuracy, the reprojection results must be within a good range. Generally, the reprojection error requirement is between 0.5 and 1 pixel; however, this is also determined by calculation based on the specific camera and BEV requirements used. The specific calculation method is shown below:

[0045]

[0046] Wherein, the unit is meters; L represents the longitudinal distance required by BEV; n represents the number of images required by BEV in the longitudinal direction; p represents the number of pixels in a single image in the longitudinal direction; and 0.2 is the error threshold.

[0047] The error threshold can be set by any appropriate method by those skilled in the art. For example, the error threshold can be set by human experience, or the error threshold can be set based on the difference value of historical data. For example, the error threshold is 0.5 pixels, which can be appropriately relaxed to an error threshold of 1 pixel. This invention does not limit this.

[0048] The surround view camera can be calibrated in the following way: Specifically, the surround view camera is calibrated using a camera calibration board, and the reprojection error of the calibrated surround view camera is calculated. If the reprojection error is less than a set error threshold, the calibration is completed, and the calibrated vehicle surround view camera is used to collect surround view images.

[0049] The initialization of the GPS system is also crucial because the output of the GPS system varies depending on the current signal. Therefore, it is necessary to ensure that the output of the GPS system is on a fixed solution at least during initialization. In other words, the positioning accuracy of the GPS system is required to be high. Only when the output of the GPS system is on a fixed solution will the GPS initialization positioning information be recorded.

[0050] The vehicle's location can be determined on a high-precision map using the initial positioning information from GPS. Since the accuracy error of GPS is within 20 cm, and based on our previous calibration results, when using a 6-megapixel camera to construct a 3×2 BEV image, the error can be controlled within 1 pixel within a perception range of 120 m, thus ensuring accuracy.

[0051] In practical applications, GPS positioning involves measuring the distance between satellites at known locations and the user's receiver, and combining data from multiple satellites to determine the receiver's specific location. This allows for the real-time acquisition of the vehicle's initial GPS positioning information, which includes the vehicle's latitude and longitude.

[0052] The vehicle is equipped with an inertial sensor (INS), which collects inertial navigation information, including acceleration information and attitude change information during vehicle operation, enabling real-time positioning of the target vehicle.

[0053] Inertial sensors (INS) generally perform positioning in two ways: one is to obtain a relatively accurate attitude using accelerometers and gyroscopes; the other is to obtain a relatively accurate attitude using sensors, gyroscopes, and magnetometers, and then obtain specific positioning information through double integration.

[0054] The vehicle is equipped with cameras with multiple perspectives. This invention can mainly use surround-view cameras to capture surround-view images. The surround-view cameras can collect road information of the vehicle, including: drivable area, center line of the same lane, center line of the opposite lane, other lane lines, pedestrian area or obstacles, or one or more of these.

[0055] The present invention can also primarily use images captured by a front-view camera, which can collect road information of the vehicle, including: drivable area, center line of the same lane, center line of the opposite lane, other lane lines, pedestrian area or obstacles.

[0056] Step 102: Match the front view image in the surround view image with the high-precision map to obtain the current vehicle's location information.

[0057] Specifically, the front view image in the surround view image is matched with a high-precision map to obtain the current vehicle's location information, that is, the latitude and longitude information of the current vehicle on the high-precision map.

[0058] Step 103: Fuse the initial GPS positioning information, the vehicle positioning information, and the inertial navigation information to output the vehicle positioning result.

[0059] In practical applications, the GPS positioning information, the vehicle positioning information, and the inertial navigation information are fused using Kalman filtering to output the vehicle positioning result.

[0060] Kalman filtering is an algorithm that uses the state equations of a linear system, taking GPS positioning information, vehicle positioning information, and inertial navigation information as inputs, to make an optimal estimation of vehicle positioning and obtain the vehicle positioning result.

[0061] During vehicle operation, the front view image from the surround view camera is matched with the high-precision map in real time to obtain the current vehicle positioning information on the high-precision map. Unlike traditional track extrapolation, the fused positioning will perform a real-time GPS "check" operation during the positioning process to ensure the reliability of the GPS signal. That is, it will continuously check whether the GPS is working properly. At the same time, it will also use the front view image taken by the front view camera and the high-precision map to confirm the current positioning effect. Finally, the Kalman filter method is used for fusion output to make the vehicle positioning result always robust.

[0062] Step 104: Determine whether the surround view image is aligned with the vehicle positioning result. If aligned, locate the vehicle positioning result on the high-precision map based on the initial positioning information of the GPS.

[0063] Determine if the surround view image is aligned with the vehicle positioning result. If they are aligned, locate the vehicle on a high-precision map based on the initial GPS positioning information. If they are not aligned, the surround view image does not belong to the vehicle positioning result, so discard the data, do not perform any further operations, and end the process.

[0064] Since the control signals of the image acquisition system and the vehicle positioning system are different, it is necessary to align the surround view image with the vehicle positioning result in time. Specifically, this can be achieved through a vision module and a positioning module. When the vision module and the positioning module receive time signals from the same source and are assigned values, the first time stamp of the surround view image and the second time stamp of the vehicle positioning result are obtained through the first time calculation module corresponding to the vision module and the second time calculation module corresponding to the positioning module, respectively. The first time stamp and the second time stamp are compared. If the difference between the first time stamp and the second time stamp is less than the threshold H, it means that the surround view image and the vehicle positioning result are aligned. If aligned, a Gaussian projection is performed on the high-precision map based on the initial positioning information of the GPS to locate the vehicle positioning result.

[0065] Because the control signals of the image acquisition system and the vehicle positioning system are different, it is necessary to calibrate the timestamp signal. Using the timestamp information provided by the main controller and considering the time difference during transmission, a threshold H is set to 100-200ms. If the difference between the first timestamp corresponding to the surround view image and the second timestamp corresponding to the vehicle positioning result is within the threshold H, then the surround view image and the vehicle positioning result are considered misaligned. If the difference between the first timestamp corresponding to the surround view image and the second timestamp corresponding to the vehicle positioning result is not within the threshold H, then the surround view image and the vehicle positioning result are considered misaligned, and no further operations are performed.

[0066] The threshold can be set by anyone skilled in the art in any appropriate way, such as by setting the threshold based on human experience or by setting the threshold based on the difference value of historical data. This invention does not limit this.

[0067] Step 105: Select a preset image area on the high-precision map based on the panoramic image, match the panoramic image with the preset image area, and then mark it according to the map information on the high-precision map to obtain the annotation data.

[0068] In practical applications, around the vehicle positioning result on the high-precision map, a preset image area is selected according to the size of the surround view image, and the surround view image is matched with the preset image area. Then, annotations are made according to the map information on the high-precision map to obtain annotation data.

[0069] After the above steps, the latitude and longitude information of the vehicle positioning result is determined by Gaussian projection based on the initial GPS positioning information. The vehicle's location is then found on a high-precision map based on the latitude and longitude information of the vehicle positioning result, thus associating the vehicle positioning result with the high-precision map data. Then, a preset image area larger than or equal to the size of the surround view image is cropped around the vehicle positioning result on the high-precision map. The surround view image is then matched with the preset image area, completing the surround view image processing. Finally, corresponding annotations are made according to the map information on the high-precision map to obtain the annotation data.

[0070] In this embodiment, vehicle positioning results are obtained through GPS positioning information, inertial navigation information, and surround view images. The vehicle positioning results and surround view images are then fused with a high-precision map to provide corresponding dataset annotations. Due to the introduction of GPS positioning information and inertial navigation information, the annotation accuracy is high, and the annotation results can be automatically generated in one go without manual intervention, saving annotation costs.

[0071] Reference Figure 2 The document illustrates a flowchart of a data set annotation method according to an embodiment of the present invention, the method specifically including:

[0072] Step 201: Calibrate the inertial sensor, calibrate the vehicle's surround view camera, and initialize the GPS.

[0073] In practical applications, in order to ensure the accuracy and precision of the annotation, it is necessary to calibrate the inertial sensors, front-view camera and surround-view camera installed on the vehicle. Specifically, the inertial sensors are calibrated using a calibration board, and then the calibrated inertial sensors are used to collect inertial navigation information, which includes: acceleration information and attitude change information of the vehicle during operation.

[0074] To ensure the accuracy and precision of the annotation, there are also very strict requirements for camera calibration. Generally, the requirements for image vehicle calibration should be appropriately relaxed, and the reprojection error of camera calibration should not exceed the error threshold T.

[0075] Specifically, the front-view camera of the vehicle is calibrated in the following way: First, the front-view camera is calibrated using a camera calibration board.

[0076] The intrinsic parameters and extrinsic parameters relative to the vehicle body were calibrated using a calibration board.

[0077] Next, the reprojection error of the calibrated front-view camera is calculated. If the reprojection error is less than a set error threshold, the calibration is completed, and the calibrated front-view camera of the vehicle is used to acquire a front-view image.

[0078] To ensure the final annotation accuracy, the reprojection results must be within a good range. Generally, the reprojection error requirement is between 0.5 and 1 pixel; however, this is also determined by calculation based on the specific camera and BEV requirements used. The specific calculation method is shown below:

[0079]

[0080] Wherein, the unit is meters; L represents the longitudinal distance required by BEV; n represents the number of images required by BEV in the longitudinal direction; p represents the number of pixels in a single image in the longitudinal direction; and 0.2 is the error threshold.

[0081] The error threshold can be set by any appropriate method by those skilled in the art. For example, the error threshold can be set by human experience, or the error threshold can be set based on the difference value of historical data. For example, the error threshold is 0.5 pixels, which can be appropriately relaxed to an error threshold of 1 pixel. This invention does not limit this.

[0082] This invention can collect road information of vehicles using a front-view camera. The road information includes one or more of the following: drivable area, center line of the same-direction lane, center line of the opposite lane, other lane lines, pedestrian area or obstacles.

[0083] In a specific application, the intrinsic parameters, extrinsic parameters, and distortion parameters of the aforementioned front-view camera are obtained; the acquired multiple initial images are subjected to distortion correction based on the aforementioned distortion parameters; the homography matrix of the aforementioned front-view camera is calculated based on the aforementioned intrinsic parameters, extrinsic parameters, and distortion parameters; and the affine transformation of the distortion-corrected multiple initial images is performed based on the aforementioned homography matrix to obtain multiple front-view images.

[0084] Before acquiring the vehicle's GPS positioning information, inertial navigation information, and surround view images, the surround view camera needs to be calibrated. Specifically, the surround view camera is calibrated using a camera calibration board, and the reprojection error of the calibrated surround view camera is calculated. If the reprojection error is less than a set error threshold, the calibration is complete, and the calibrated vehicle surround view camera is used to acquire surround view images.

[0085] To ensure the final annotation accuracy, the reprojection results must be within a good range. Generally, the reprojection error requirement is between 0.5 and 1 pixel; however, this is also determined by calculation based on the specific camera and BEV requirements used. The specific calculation method is shown below:

[0086]

[0087] Wherein, the unit is meters; L represents the longitudinal distance required by BEV; n represents the number of images required by BEV in the longitudinal direction; p represents the number of pixels in a single image in the longitudinal direction; and 0.2 is the error threshold.

[0088] The error threshold can be set by any appropriate method by those skilled in the art. For example, the error threshold can be set by human experience, or the error threshold can be set based on the difference value of historical data. For example, the error threshold is 0.5 pixels, which can be appropriately relaxed to an error threshold of 1 pixel. This invention does not limit this.

[0089] The initialization of the GPS system is also crucial because the output of the GPS system varies depending on the current signal. Therefore, it is necessary to ensure that the output of the GPS system is on a fixed solution at least during initialization. In other words, the positioning accuracy of the GPS system is required to be high. Only when the output of the GPS system is on a fixed solution will the GPS initialization positioning information be recorded.

[0090] Step 202: Obtain the vehicle's initial GPS positioning information, inertial navigation information, and surround view image.

[0091] Inertial navigation information is collected by inertial sensors (INS) installed on the vehicle. This inertial navigation information includes the vehicle's acceleration information and attitude change information, which enables real-time positioning of the target vehicle.

[0092] A forward-view camera installed on the vehicle captures a forward-view image, which includes road information about the vehicle, including one or more of the following: drivable area, center line of the same-direction lane, center line of the opposite lane, other lane lines, pedestrian area, or obstacles. A surround-view camera installed on the vehicle captures a surround-view image of the area around the vehicle, which includes one or more of the following: drivable area, center line of the same-direction lane, center line of the opposite lane, other lane lines, pedestrian area, or obstacles.

[0093] Step 203: Match the front view image in the surround view image with the high-precision map to obtain the current vehicle positioning information.

[0094] Step 204: Fuse the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information to output the vehicle positioning result.

[0095] In practical applications, the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information are fused using Kalman filtering to output the vehicle positioning result.

[0096] Specifically, the working status of the GPS is continuously monitored, i.e., a continuous real-time "check" operation is performed on the GPS. Then, based on the working status of the GPS, the weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information are adjusted to obtain the adjusted weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information. The adjusted weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information are then fused using Kalman filtering to obtain the vehicle positioning result.

[0097] If the GPS is functioning normally, the weight of the initial GPS positioning information is greater than the weight of the current vehicle positioning information and the weight of the inertial navigation information. In other words, the weight of the initial GPS positioning information is greater than the weight of the current vehicle positioning information, and the weight of the current vehicle positioning information is greater than the weight of the inertial navigation information. This means that the GPS is functioning normally, indicating that the initial GPS positioning information is more accurate. At this time, the weight of the initial GPS positioning information is the highest, and the weight of the vehicle positioning information is less than the weight of the initial GPS positioning information. The weight of the inertial navigation information can be adjusted or left unchanged, depending on the actual application scenario.

[0098] If the GPS is not functioning properly, the weight of the initial GPS positioning information is less than the weight of the vehicle positioning information and the weight of the inertial navigation information. That is, the weight of the current vehicle positioning information is greater than the weight of the inertial navigation information, and the weight of the inertial navigation information is greater than the weight of the initial GPS positioning information. In other words, if the GPS is not functioning properly, it means that the initial GPS positioning information is inaccurate. At this time, the weight of the initial GPS positioning information can be the lowest, then the weight of the vehicle positioning information can be the highest, and the weight of the inertial navigation information is between the weight of the initial GPS positioning information and the weight of the vehicle positioning information.

[0099] It should be noted that the weights of the initial GPS positioning information, the vehicle positioning information, and the inertial navigation information are adjusted according to the working status of the GPS. The specific weight ratios can be adjusted according to different application scenarios, or set based on historical data. This invention does not impose any restrictions on this.

[0100] Step 205: Determine whether the surround view image is aligned with the vehicle positioning result. If aligned, locate the vehicle positioning result on the high-precision map based on the initial positioning information of the GPS.

[0101] In practical applications, the alignment between the surround view image and the vehicle positioning result can be determined by the following method: obtaining the first timestamp corresponding to the surround view image and obtaining the second timestamp corresponding to the vehicle positioning result; calculating the difference between the first timestamp and the second timestamp; if the difference is less than a set threshold, then the surround view image and the vehicle positioning result are aligned.

[0102] The threshold can be set by anyone skilled in the art in any appropriate way, such as by setting the threshold based on human experience or by setting the threshold based on the difference value of historical data. This invention does not limit this.

[0103] Step 206: Select a preset image area on the high-precision map based on the panoramic image, and match the panoramic image with the preset image area. Then, mark the area according to the map information on the high-precision map to obtain the annotation data.

[0104] In practical applications, after aligning the timestamps of the surround view image with the vehicle positioning results, the starting position on the high-precision map is determined based on the initial GPS positioning information. Then, the vehicle positioning results are processed to obtain the vehicle's latitude and longitude information. Based on this information, the vehicle's location can be determined on the high-precision map. Next, based on the size of the surround view image, a preset image region of the same size as the surround view image is extracted around the vehicle positioning results on the high-precision map. The surround view image is then placed within this preset image region. Finally, annotations are made based on map information on the high-precision map to obtain annotation data, such as... Figure 3-c As shown.

[0105] The preset image area can be selected based on the size of the panoramic image. In practical applications, the preset image area is generally slightly larger than the size of the panoramic image.

[0106] Since the mapping table for high-precision maps provides corresponding meanings for each piece of map information, annotations are made on the high-precision map based on these meanings to obtain annotation data. For example... Figure 3-c As shown.

[0107] In this embodiment, the vehicle positioning result is first obtained through GPS positioning information, inertial navigation information and front view image. The vehicle positioning result and the surrounding view image are then fused with a high-precision map to give the corresponding dataset annotation. Since GPS positioning information and inertial navigation information are introduced, the annotation accuracy is high, and the annotation result can be automatically generated at one time without manual intervention, saving annotation costs.

[0108] Secondly, continuous real-time GPS "check" operations are performed and matched and fused with corresponding images to ensure the positioning accuracy on high-precision maps, thus ensuring the accuracy of annotation when using high-precision map data. This operation also solves the problem of high-precision map annotation distortion caused by GPS signal loss.

[0109] To enable those skilled in the art to better understand the technical solutions defined in this invention Figure 3-a , Figure 3-b and Figure 3-c This invention provides a detailed explanation of a dataset annotation method using an example.

[0110] in, Figure 3-a It is a high-precision map; Figure 3-b This is the output of the vehicle location results; Figure 3-c This indicates the labeled data.

[0111] exist Figure 3-a The cylinders visible in the image represent the vehicle's location output on the high-precision map. Figure 3-b The bouncing cylinders you can see represent GPS positioning output; the two overlapping cylinders represent semantic and fusion outputs from the positioning; the cylinders intersecting with lane lines on the high-precision map represent semantic matching results. Specifically, according to... Figure 3-b The output of the vehicle positioning results, in Figure 3-a The vehicle's location information is determined on a high-precision map. Since the vehicle's location information is already defined on the high-precision map, it is then determined whether the timestamps of the surround view image and the vehicle's location result are aligned. If they are aligned, a preset image area is selected within the vehicle's location information based on the size of the surround view image, and the surround view image is placed within the preset image area. Then, a lot of map information will be displayed on the high-precision map. Based on the meaning of the map information in the corresponding table, annotations are made on the high-precision map to obtain annotation data, such as... Figure 3-c As shown in Figure 3, the explanation uses six panoramic images as an example, namely images 1-6. Figure 3-b As can be seen, GPS information is fluctuating, therefore, the positioning and high precision of this invention must be combined to achieve good data annotation results.

[0112] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0113] Based on the description of the above method embodiments, the present invention also provides corresponding device embodiments to implement the content described in the above method embodiments.

[0114] Reference Figure 4 The diagram illustrates a structural schematic of a dataset annotation device according to an embodiment of the present invention, the device comprising:

[0115] The acquisition module 401 is used to acquire the vehicle's initial GPS positioning information, inertial navigation information, and surround view image.

[0116] The matching module 402 is used to match the front view image in the surround view image with the high-precision map to obtain the current vehicle's positioning information.

[0117] The fusion module 403 is used to fuse the initial GPS positioning information, the current vehicle positioning information and the inertial navigation information, and output the vehicle positioning result.

[0118] The judgment module 404 is used to determine whether the surround view image is aligned with the vehicle positioning result. If they are aligned, the vehicle positioning result is located on the high-precision map according to the initial positioning information of the GPS.

[0119] The annotation module 405 is used to select a preset image area on the high-precision map based on the panoramic image, and to match the panoramic image with the preset image area, and to annotate it according to the map information on the high-precision map to obtain annotation data.

[0120] Furthermore, the determination module includes:

[0121] The timestamp unit is used to obtain the first timestamp corresponding to the surround view image and the second timestamp corresponding to the vehicle positioning result.

[0122] The first calculation unit is used to calculate the difference between the first timestamp and the second timestamp. If the difference is less than a set threshold, the surround view image is aligned with the vehicle positioning result.

[0123] Furthermore, the fusion module includes:

[0124] An adjustment unit is configured to adjust the weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information according to the working state of the GPS, so as to obtain the adjusted weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information.

[0125] The fusion unit is used to fuse the adjusted weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information to obtain the vehicle positioning result.

[0126] Furthermore, the adjustment unit is specifically used for:

[0127] If the GPS is in normal working condition, the weight of the initial GPS positioning information is greater than the weight of the current vehicle positioning information and the weight of the inertial navigation information.

[0128] If the GPS is not functioning properly, the weight of the initial GPS positioning information is less than the weight of the current vehicle positioning information and the weight of the inertial navigation information.

[0129] Furthermore, the device also includes:

[0130] The calibration module is used to calibrate inertial sensors and the vehicle's surround-view camera;

[0131] The acquisition module is used to acquire the inertial navigation information using a calibrated inertial sensor and to acquire surround view images using a calibrated vehicle surround view camera;

[0132] An initialization module is used to initialize the GPS, so that the initialization solution output by the GPS is a fixed solution, and to record the initial positioning information of the GPS.

[0133] Furthermore, the calibration module is used to calibrate the front-view camera of the vehicle, including:

[0134] A camera calibration unit is used to calibrate the surround-view camera using a camera calibration plate;

[0135] The second calculation unit is used to calculate the reprojection error of the calibrated surround-view camera; if the reprojection error is less than a set error threshold, the calibration is completed.

[0136] This invention obtains vehicle positioning results through GPS initial positioning information, inertial navigation information, and current vehicle positioning information, ensuring the accuracy of vehicle positioning. Then, the vehicle positioning results are aligned with a surround view image, and the vehicle positioning result is located on a high-precision map based on the initial GPS positioning information. A preset image area is selected on the high-precision map according to the size of the surround view image, and the surround view image is matched with the preset image area. Annotations are then made based on map information on the high-precision map to obtain annotation data. Because GPS positioning information and inertial navigation information are incorporated, the annotation accuracy is high, and the annotation results can be automatically generated in one go without manual intervention, saving annotation costs.

[0137] The above-described apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple. For relevant details, please refer to the description of the method embodiments shown.

[0138] It will be readily apparent to those skilled in the art that any combination of the above embodiments is feasible, and therefore any combination of the above embodiments is an implementation scheme of the present invention. However, due to space limitations, this specification will not describe them in detail here.

[0139] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the present invention. The above embodiments are merely preferred embodiments given to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A data set labeling method, characterized by, include: Acquire the vehicle's initial GPS positioning information, inertial navigation information, and surround view image; The front view image in the surround view image is matched with the high-precision map to obtain the current vehicle positioning information; The initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information are fused together to output the vehicle positioning result. Determine whether the surround view image is aligned with the vehicle positioning result. If they are aligned, locate the vehicle positioning result on the high-precision map based on the initial positioning information of the GPS. On the high-precision map, a preset image area is selected based on the panoramic image, and the panoramic image is matched with the preset image area. Then, annotations are made based on the map information on the high-precision map to obtain annotation data. The initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information are fused to output the vehicle positioning result, which includes: Based on the working status of the GPS, the weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information are adjusted to obtain the adjusted weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information. The adjusted weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information are fused to obtain the vehicle positioning result.

2. The method of claim 1, wherein, The step of determining whether the surround view image is aligned with the vehicle positioning result includes: Obtain the first timestamp corresponding to the surround view image and obtain the second timestamp corresponding to the vehicle positioning result; Calculate the difference between the first timestamp and the second timestamp. If the difference is less than a set threshold, then the surround view image is aligned with the vehicle positioning result.

3. The method of claim 1, wherein, Adjusting the weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information based on the GPS operating status includes: If the GPS is in normal working condition, the weight of the initial GPS positioning information is greater than the weight of the current vehicle positioning information and the weight of the inertial navigation information. If the GPS is not functioning properly, the weight of the initial GPS positioning information is less than the weight of the current vehicle positioning information and the weight of the inertial navigation information.

4. The method of claim 1, wherein, The method further includes: Calibrate the inertial sensors and the vehicle's surround-view camera; The inertial navigation information is acquired using a calibrated inertial sensor, and the surround view image is acquired using a calibrated vehicle surround view camera. The GPS is initialized so that the initial solution output by the GPS is a fixed solution, and the initial positioning information of the GPS is recorded.

5. The method of claim 4, wherein, The calibration of the vehicle's surround-view camera includes: The surround-view camera was calibrated using a camera calibration board; Calculate the reprojection error of the calibrated surround-view camera; If the reprojection error is less than the set error threshold, the calibration is complete.

6. A data set labeling apparatus characterized by comprising: include: The acquisition module is used to acquire the vehicle's initial GPS positioning information, inertial navigation information, and surround view image; The matching module is used to match the front view image in the surround view image with the high-precision map to obtain the current vehicle positioning information; The fusion module is used to fuse the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information, and output the vehicle positioning result; The judgment module is used to determine whether the surround view image is aligned with the vehicle positioning result. If they are aligned, the vehicle positioning result is located on the high-precision map according to the initial positioning information of the GPS. The annotation module is used to select a preset image area on the high-precision map based on the panoramic image, match the panoramic image with the preset image area, and then annotate it according to the map information on the high-precision map to obtain annotation data; The fusion module includes: An adjustment unit is configured to adjust the weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information according to the working state of the GPS, so as to obtain the adjusted weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information. The fusion unit is used to fuse the adjusted weights of the initial GPS positioning information, the current vehicle positioning information, and the inertial navigation information to obtain the vehicle positioning result.

7. The apparatus of claim 6, wherein, The judgment module includes: The timestamp unit is used to obtain the first timestamp corresponding to the surround view image and the second timestamp corresponding to the vehicle positioning result. The first calculation unit is used to calculate the difference between the first timestamp and the second timestamp. If the difference is less than a set threshold, the surround view image is aligned with the vehicle positioning result.

8. The apparatus according to claim 6, characterized in that, The adjustment unit is specifically used to: if the GPS is in normal working condition, then the weight of the initial GPS positioning information is greater than the weight of the current vehicle positioning information and the weight of the inertial navigation information; If the GPS is not functioning properly, the weight of the initial GPS positioning information is less than the weight of the current vehicle positioning information and the weight of the inertial navigation information.

9. The apparatus according to claim 6, characterized in that, The device further includes: The calibration module is used to calibrate the inertial sensor and the vehicle's surround-view camera; the acquisition module is used to acquire the inertial navigation information using the calibrated inertial sensor and to acquire surround-view images using the calibrated vehicle's surround-view camera. An initialization module is used to initialize the GPS, ensuring that the initialization solution output by the GPS is a fixed solution, and to record the initial positioning information of the GPS.

10. The apparatus according to claim 9, characterized in that, The calibration module is used to calibrate the surround-view camera for the vehicle, including: A camera calibration unit is used to calibrate the surround-view camera using a camera calibration plate; The second calculation unit is used to calculate the reprojection error of the calibrated surround-view camera; if the reprojection error is less than a set error threshold, the calibration is completed.

Images